Peer Review and Design Competition in the NNSA National Security Laboratories (2015)

4


The Future: Responding to Evolving Challenges

Because the nuclear deterrent remains a cornerstone of U.S. national defense policy, assuring the quality of the technical work that supports the nation’s stockpile is at least as important now as it has been since the introduction of nuclear weapons. However, owing to the U.S. adherence to the moratorium on nuclear weapons testing, the three NNSA laboratories must retain the design and engineering capability needed to maintain the nuclear stockpile without designing and testing new weapons. Maintaining the stockpile under these constraints requires a deep understanding of how weapons’ surety and performance may be affected by aging or the substitution of components. At the same time, the laboratories must develop and exercise new skills to address evolving stockpile needs. For example, it would not be surprising if the nation were to someday need newly designed and engineered components to address some fundamental safety or reliability risk to the nuclear stockpile. The laboratories would not only have to design and engineer the new components but would also have to assess their effect on the overall weapons in which they are placed. Moreover, as other nations pursue new designs or strategies that could constitute serious threat evolutions, the United States could find itself in a precarious security situation were it not to maintain nuclear weapon design, development, and production skills to address such evolving demands.

During the Cold War era, the nation depended on an active program of designing and testing nuclear weapons because the threat from other nations was changing rapidly. Following the collapse of the Soviet Union,

and perhaps even in the later years of the Cold War, the nuclear threat landscape appeared more static, and the United States stabilized and reduced its stockpile. After the cessation of nuclear explosion testing, the United States and its allies put their main efforts into maintaining the existing stockpile.

Today the nation is again facing evolving threats as Russia and China modernize their nuclear stockpiles and their doctrines for use. Several countries are contemplating the development of nuclear capabilities, terrorism has spread across the world, and technology advances are accelerating.

NNSA is entrusted with the responsibility of ensuring unimpeachable confidence in the nation’s nuclear warheads through its nuclear complex, consisting of the three national security laboratories, the test site, and the production plants. Today the technical challenges to carrying out this mission in the global environment described above are unprecedented:

• For the near term, the average age of the warheads in the stockpile will continue to climb, increasing the challenge of surveillance, meant to assure the physical state of the weapons, and for the annual stockpile assessments.
• For the medium term, Life-Extension Programs (LEPs) for the warheads in all the enduring stockpile systems are planned over the next several decades. In an LEP, components that are known to suffer from deterioration or obsolescence within the time frame considered in the LEP are replaced; other components that may degrade more slowly with time or fail abruptly after a time longer than that examined may or may not be replaced. As a result, the “aging clock” of these latter components continues to tick. In some LEPs, components may need to be changed to improve the safety and security of the warhead. Assessment of the performance of changed components is a significant challenge, especially because the skills associated with designing and developing nuclear explosive packages (NEPs)—and, in the process, strengthening understanding of the linkages between design and performance—have not been thoroughly exercised in the complex for more than 20 years.
• For the longer term, the nation must have in place the capability to anticipate and respond to the potential new threats the country could face in the future. To attempt this with a future workforce without validated experience in weapon design and development would be very risky.

These technical challenges all require a science and engineering enterprise of high quality and technical staff of high competence and good judgment. Peer review and design competition contribute essentially to that quality.

CONCLUSIONS AND RECOMMENDATIONS

Ultimately, the nuclear weapons program can only be effectively maintained (and a rigorous system of peer review preserved) if many of the nation’s best scientists and engineers choose to commit their time and talents to the NNSA laboratories. The most effective way of encouraging this is to maintain technical vitality at the laboratories through state-of-the-art research, exciting work, challenge, and reward. In support of their core mission—to sustain the nation’s nuclear deterrent—the laboratories should seek to develop and support the people who execute that mission. While a comprehensive treatment of this challenge is beyond the scope of the current report, it has been the subject of numerous recent reports dealing with the quality of science and engineering at the laboratories¹ and governance of the laboratories.²

Below, the tasks that constitute the committee’s charge are taken from Chapter 1, repeated one by one, and set in italics. Each task is followed by the committee’s relevant conclusions and recommendations.

The examples cited in Chapter 3 illustrate how technical programs within the NNSA national security laboratories have benefited from peer review of several kinds. While the approaches used by all three laboratories have proven successful in the main—providing high-quality, effective peer reviews of designs, development plans, and engineering and scientific activities—the committee finds that there are areas that need to be strengthened in these approaches in order to improve the assurance of stockpile reliability and surety going forward.

________________

¹ National Research Council (NRC), 2012, Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories, The National Academies Press, Washington, D.C.; NRC, 2013, The Quality of Science and Engineering at the NNSA National Security Laboratories, The National Academies Press, Washington, D.C.

² Congressional Advisory Panel on the Governance of the Nuclear Security Enterprise, 2014, A New Foundation for the Nuclear Enterprise, November; NRC, 2015, Aligning the Governance Structure of the NNSA Laboratories to Meet 21st Century National Security Challenges, The National Academies Press, Washington, D.C.

In particular, some of the case studies presented to the committee illustrate that relying too heavily on in-house peers and experts can limit the value of reviews because a review group composed solely of insiders may be so close to the technology and program under review that it fails to recognize subtle weaknesses in components, systems, or methods of analysis. In contrast, the engagement of peers with different experience bases, perspectives, and technical skill sets can strengthen a review process and avoid groupthink. The committee recognizes that involving some experts from outside the laboratory can add some costs and perhaps entail a learning curve, but it believes the value gained from outside perspectives offsets those downsides. The case studies presented to the committee also demonstrated the value of having documented frameworks for reviews. Such frameworks codify best practices and in essence capture the insights gained during the days of nuclear testing, when laboratory staff saw the many ways in which nature confounded the best predictions of any single team of scientists and engineers. Codifying the peer review approaches developed in recent years with the insights from the days of nuclear testing can be very beneficial to future generations of weapons designers after the current generation has retired.

In a world where the nuclear threat is evolving, peer review becomes even more important to NNSA and its laboratories (in the absence of testing of the NEP and with pressure on budgets for testing of non-nuclear subsystems and systems) for addressing technical challenges encountered in carrying out their mission. The primary value and incentive for peer review is that it reduces risk—of overlooking a technical option, of relying on suboptimal data or methods, or of simply making a mistake. In essence, peer review improves the quality of the work. As noted in Chapter 3, a process by which two technical teams compare their approaches, expertise, and results helps each team improve its own understanding of a weapon and its safety, security, and effectiveness. That knowledge is essential and intrinsically valued by staff at all levels in the laboratories. All three laboratories have an ingrained culture that sets a high standard for quality; this gives staff members an incentive to use peer review.

Furthermore, peer review is clearly valued by laboratory management. As an example, the NEP design laboratory directors were directly involved in establishing and advocating for INWAP, which has improved understanding of the stockpile and reinforces confidence in the warhead annual assessments. The value senior leadership places on peer review provides a strong incentive for peer review among the middle management and staff. The committee notes that independent peer reviews are regularly used by laboratory directors to assess early design feasibility studies as well as to review surveillance and SFI results, even though such reviews are not mandated and may expend their limited resources. In summary, the committee concludes the following:

• Peer review reduces the risk of overlooking a technical option, of relying on suboptimal data or methods, or of simply making an embarrassing mistake, and thereby supports the ingrained culture at all three laboratories, which sets a high standard for quality.
• Peer review is visibly valued by laboratory management.

• With only archival nuclear explosion test data available, LANL and LLNL rely on vigorous, deep-dive reviews by truly competitive peers and other subject-matter experts to critique the results of calculations and subcritical experiments relating to NEP performance.
• With testing data available to verify the performance of components and systems and to validate modeling and simulation tools, peer review at SNL is driven more by the need to assure cost-effective performance of stockpile hardware under all anticipated conditions and by budget pressures to reduce the number of expensive tests.

• With the exception of major reviews associated with the Annual Assessment Report or Life-Extension Programs, LLNL and LANL lack written guidance for conducting peer reviews to determine in general when a review is needed, how the review is to be conducted, who should participate in the review, or how to address review findings.
• SNL has developed useful written guidance for conducting peer reviews; however, during its visit to SNL and its probing of the presentations made, the committee determined that SNL could profitably make greater use of outside experts in its peer reviews, as called for in its written guidance.

• Los Alamos National Laboratory and Lawrence Livermore National Laboratory should ensure they have short, written guidance for a graded approach to peer review, the rigor of which is appropriate to the stage of work and range of technical activities being reviewed.
• Sandia National Laboratories should strengthen and broaden its use of outside experts on its peer review teams, as articulated in written guidance that Sandia recently finalized.

Recommendation 1, which calls for the NEP laboratories to develop additional peer review guidance and for SNL to ensure that outside experts are fully utilized in peer reviews, could entail some additional costs, though these should be minor. The call of Recommendation 2 for the maintenance of capabilities to conduct independent analyses does involve additional costs, but the benefits are likely to outweigh these.

There are not many examples where peer review has resulted in major course corrections; some of the more important ones are listed in Chapter 3. These include an improved understanding of the plutonium equation of state and of how plutonium’s characteristics change as it ages.

The limited number of examples in part reflects the fact that the research and development teams at the laboratories are really quite good and that they get it right most of the time. The other factor that is missed entirely by making lists of successes and failures is that a large part of the benefit of the review comes in the preparation for the review. The people to be reviewed have to get their thoughts in order about what they are doing and how to present what they have done and what they need to do next. This may be the most important benefit. The (expert) designers who are being reviewed by (expert) peer reviewers are more likely to themselves realize they have made a mistake than are the reviewers, especially since most mistakes are subtle and are found only when designers are recalculating everything that is being reviewed and staring at the calculation frames or movies that do not seem quite right. The committee looked for instances in which peer review failed to find problems, but it did not find any clear examples of this.

The three laboratories currently carry out high-quality, effective peer reviews of designs, development plans, and engineering and scientific activities. However, in the areas of design studies and design competition—which often include interlaboratory technical reviews—the committee found situations in need of improvement (see Conclusions 2 and 3 of Chapter 3, repeated below).

The Reliable Replacement Warhead (RRW) design study, discussed in Chapter 3, was one such particularly difficult example from which many lessons should be learned. The technical review format with broad participation early in the process engendered deep-seated and negative concerns between the two laboratories and a mistrust with NNSA that still exists. While the RRW was successful in motivating the nuclear weapon design staff and in generating unique designs, the competition process

had faults, and it did not offer any validation of the design because there was no opportunity to explore the viability of the “winning” design. It is not the right model to follow for future efforts to exercise the laboratories’ design capabilities.

Senior staffers from both LANL and LLNL told the committee³ that they support the concept of true design competitions as a necessary means of maintaining the laboratories’ capabilities in nuclear weapons design. But they emphasized the need for constructively managed competitions that are initialized with well-elucidated guidelines and a clearly envisioned outcome. In addition, they agreed that paper studies alone are not enough to challenge and maintain the skills of the weapon designers.

Recommendation 3 calls for a process change in future design competitions that would have no effect on costs.

As the threat faced by them continues to evolve, the United States and its allies will face new challenges. Since the end of the Cold War, the perception has been that the threats facing the United States have not required any basic change in the capability of the nuclear deterrent. Over the past two decades, Congress has restricted new NEP design studies and DOD has not required any fundamentally new warhead designs, nor have there been any of the associated design competitions that were so

________________

³ Jas Mercer-Smith, during committee discussions on September 24, 2014, and Charles Verdon, during committee discussions on November 14, 2014.

successful and valuable to the health of the complex during the Cold War. Because of this lack of new NEP design work, essential capabilities have not been exercised in a generation and are at risk. The committee’s most significant concern is that the capability for a full, integrated, end-to-end design, including of the integrated NEP—from the design concept for a different device to address a threat through the production of an engineering prototype at a level of confidence that it could, in principle, be considered for the stockpile—is not being exercised. The approved programs for the future (e.g., the W76-1 and the B61-12 LEPs) have not included the full physics design challenges because these programs are largely just a rebuild of the original NEP design with some changes of materials where necessary. The few “design competitions” since 1992 have been largely extensive design studies that did not move beyond the study phase to the later phases of full engineering development and prototype hardware and an evaluation of whether the design might be acceptable for the stockpile. The fraction of the NEP laboratories’ science and engineering personnel with hands-on experience in nuclear weapons design or testing continues to decrease and will reach zero in the next decade or so. Once this experience is lost, it could seriously compromise the nation’s defensive posture and will be difficult to reestablish.

To keep this from happening, the NNSA complex needs a way to exercise the full suite of nuclear weapons design, development, and production capabilities. A true design competition would exercise the full spectrum of skills and activities needed to produce a weapon that qualifies for inclusion in the stockpile. Such capabilities might be needed, for example, if evolving military requirements require an adjustment to an NEP.

• There have been no full NEP design competitions since the 1992 nuclear explosion testing moratorium. Recent design studies have been good analysis and modeling exercises, but they did not result in the actual engineering and fabrication of components and systems; thus, they did not exercise the complete set of skills required in the NNSA complex to design nuclear weapons that would be an effective deterrent, nor was the credibility of any design assessed by fabricating a device or by non-nuclear testing.
• At SNL, the need to continually replace aging or obsolete nonnuclear components in stockpile weapons, as well as the large Life-Extension Programs for the W76 and B61, have indeed exercised designers’ skills. However, these exercises do not stimulate

Looking to the future, maintaining nuclear weapon design skills at the NEP laboratories—as well as production skills within the NNSA complex—is essential to achieve three objectives:

1. Maintaining a credible nuclear deterrent workforce that is capable of designing and building weapons to meet evolving threats;
2. Understanding the status and direction of foreign nuclear weapon programs and thus strengthening the nonproliferation regime;
3. Determining the best and most cost-effective approaches to resolving problems that arise during stockpile weapon surveillance and Life-Extension Programs.

These design competitions should be of modest cost and managed so as to impact neither the cost of nor the schedule for delivering LEPs or the 3 + 2 program plan. The projects should be full design competitions that involve LANL, LLNL, SNL, and representatives from the plants and an applicable military service. Such design competitions should be initiated periodically (perhaps once every 5 years) to allow learning from mistakes and for the continuous development of judgment and skills of the nuclear weapon enterprise workforce. This recommendation is not unprecedented. In 2005, an Advisory Board (SEAB) Task Force of the Secretary of Energy recommended that a new version of the RRW, “incorporating new design concepts and surety features, be initiated on planned 5-year cycles.”⁴

The committee recognizes that if the 3 + 2 program was implemented,

________________

⁴ U.S. Department of Energy. Secretary of Energy Advisory Board, Nuclear Weapons Complex Infrastructure Task Force, 2005, Recommendations for the Nuclear Weapons Complex of the Future, July 13, p. 13.

there would be some warhead redesign work associated with the interoperable warheads (e.g., if conventional high explosive was to be replaced by insensitive high explosive).⁵ In addition, prototype interoperable warheads could be built. However, because of the constraints associated with the requirements for component reuse and compatibility with existing delivery systems, the committee’s view is that the 3 + 2 program does not involve fundamentally new designs and therefore does not exercise the same NEP design skills as the “clean slate” design competitions recommended here. To the extent that the 3 + 2 program does turn out to involve extensive warhead redesign, it could fulfill in part the purpose of Recommendation 4.

The committee realizes that Recommendation 4 will be controversial, particularly its call for NNSA to hold periodic competitions at its laboratories that produce a prototype nuclear weapon. The committee’s concept of the characteristics of such a prototype is laid out in Box 4.1.

Recommendation 4 might be seen by some critics as promoting an aggressive posture that would put the United States in a position to manufacture new nuclear weapons quickly and thus fuel a new global nuclear arms race. These same arguments were made in the vigorous

________________

⁵ JASON, 2015, “Technical Considerations for the Evolving U.S. Nuclear-Weapons Stockpile Executive Summary,” JSR14-Task-006E, January.

debate over the wisdom of the RRW program and its potential effects on nonproliferation efforts.⁶ And, as noted earlier, a similar recommendation was made by the 2005 SEAB Task Force.

Congress has authorized NNSA to “develop and carry out a plan for the national security laboratories and nuclear weapons production facilities to design and build prototypes of nuclear weapons to further intelligence estimates with respect to foreign nuclear weapons activities.”⁷ More recently, Congress called for “the directors of the national security laboratories [to] jointly develop a multiyear plan to design and build prototypes of nuclear weapons to further intelligence estimates with respect to foreign nuclear weapons activities and capabilities.”⁸ Recommendation 4, which is aimed at the preservation of nuclear weapon design capabilities at the NNSA laboratories, is consistent with the spirit of these authorizations.

Recommendation 4 calls for alternative design competitions that would be much more effective than the recent design studies, but it would entail costs. The committee’s view is that the laboratory staff activities that would take place during its recommended competitions would be focused applications of science and engineering skills that should happen anyway under the laboratories’ science campaign (~$412 million in FY2015) and engineering campaign (~$136 million in FY2015), so that the incremental cost of the competitions should be modest, while the benefit to the nation would be immense.

Roughly speaking, the committee imagines a design competition as involving a few dozen laboratory staff members, with a larger number in the first year of each competition, plus some prototype development and experiments up to and including hydrodynamic tests.⁹ These parameters suggest a scale for the endeavor that the committee deems appropriate.

Over the past 60 years, the mission of the NNSA laboratories has

________________

⁶ Jonathan Medalia, Congressional Research Service, 2009, The Reliable Replacement Warhead Program: Background and Current Developments, July 27.

⁷ Section 3115 of the 2013 National Defense Authorization Act (P.L. 112-239).

⁸ Section 3111 of the 2015 National Defense Authorization Act (P.L. 113-291).

⁹ “In a hydrodynamic test, inert material (e.g., ²³⁸U or a simulant for plutonium) is imploded to determine how well the high-explosive system functions. The testing program for an unboosted implosion device primarily ensures that the hydrodynamic behavior of the implosion (particularly of a hollow pit) is correct.” Federation of American Scientists, http://fas.org/nuke/intro/nuke/test.htm.

evolved from an exclusive focus on designing, engineering, testing, and maintaining nuclear weapons to a broader and more diverse mission of advancing national security generally. In response to funding uncertainty and the needs of other government agencies, the laboratories have worked to build a set of clients beyond DOE. This broadened funding base supports the core capabilities needed to perform the laboratories’ primary nuclear security mission. From modest beginnings in the 1960s, when DOD asked one of the laboratories to develop sensors for a specific application in the Vietnam War, what was formerly called Work for Others (WFO) and is now termed a Strategic Partnership Project (SPP) has grown to become a significant portion of the laboratories’ technical portfolios. In FY2013, for instance, the NNSA weapons complex received $1.656 billion in research funding from other federal agencies,¹⁰ accounting for 36 percent of SNL’s budget, 18 percent of LLNL’s, and 9 percent of LANL’s.

The largest sponsors of SPP at the NNSA laboratories are DOD and the Intelligence Community. Other sponsors include the Department of Homeland Security, the National Institute of Standards and Technology, the Food and Drug Administration, the Centers for Disease Control and Prevention, and the National Aeronautics and Space Administration. In 2008, Secretary of Energy Samuel Bodman formally articulated a vision for the future of the NNSA laboratories as national security laboratories charged with conducting research and development to address a range of national security threats facing the nation.¹¹

Several recent reports have noted the benefits that SPP brings to the nuclear weapons mission of the laboratories. For example, SPP provides challenging problems to the laboratory scientists that help attract, develop, and retain key personnel.¹² Furthermore, as illustrated by several presentations during the committee’s meeting at SNL, in many areas there are direct correlations between technical projects pursued in the context of SPP and nuclear weapons work (e.g., on radar or other sensor technologies). This results in an intellectual ferment that enriches the weapons program.

In the present context of assessing the effects of the evolving national security mission of the NNSA laboratories on peer review in the nuclear weapons program, the committee concludes that SPP will expand the base of technical experts available for peer review by involving (1) expert personnel inside the laboratories but outside the direct weapons programs

________________

¹⁰ NRC, 2014, Aligning the Governance Structure of the NNSA Laboratories to Meet 21st Century National Security Needs, The National Academies Press, Washington, D.C.

¹¹ “Transforming the Nuclear Weapons Complex into a National Security Enterprise,” signed on June 19, 2008.

¹² NRC, 2012, Managing for High-Quality Science and Engineering at the NNSA National Security Laboratories, The National Academies Press, Washington, D.C.

and (2) SPP customers in DOD, the Intelligence Community, and their industry partners, who will add substantially to the pool of qualified peer reviewers for the weapons program. These new sources of expertise could help broaden and diversify laboratory peer reviews, as called for in Recommendation 1.

SUMMARY COMMENTS

Implementation of the above four recommendations would help ensure that the most important asset—a competent workforce with demonstrated skills and judgment—is being developed and maintained and that all stakeholders (including our adversaries) have confidence in that workforce. There is no better way to learn and to develop judgment than to evaluate and test one’s ideas and to understand how implementation compares with expectations. The judgment of this workforce is the basis for all stakeholders’ confidence in the nuclear deterrent and the NNSA laboratories’ ability to respond to evolving national threats.