managers and the Director of the LLNL. Reports of the panels are circulated within DOE but are restricted.

Sandia National Laboratories is operated under a contract between NNSA and Sandia Corporation, a wholly owned subsidiary of the Lockheed Martin Corporation. There are six Lockheed employees on the Sandia Corporation Board of Directors. The Corporation’s S&T Committee is chaired by the chief technical officer of Lockheed. About half of the more than $2 billion budget at Sandia National Laboratories is for S&T. Peer review at Sandia is divided between S&T and nuclear weapons work. For S&T, the reviews are conducted under contract to the University of Texas (UT), which has two positions on the board of Sandia Corporation. Under supervision by the Sandia Corporation Board of Directors Subcommittee on S&T, the UT selects and convenes review panels. Both Sandia management and Sandia Corporation have input into the final selections. Each panel has 6 to 12 members from various disciplines, including physical sciences, computation, electronics, and materials. The UT draws some panel members from its faculty; other members are drawn from other sources. There is an external panel for each area of scientific and technical competence. Panels meet for multiple day sessions and cover individual projects and programs. They meet with staff members to assess morale and the research environment. They also may meet with groups of principal investigators at the program level, as well as with individual project leaders. The review results are reported to laboratory management and the board of directors’ S&T Committee. Principal investigators receive the reports and must respond to panel critiques.

Nuclear weapons-related peer review at Sandia includes three kinds of internal reviews: design, management, and internal peer review (members from entities other than the design team under review). Reviews are assisted by a full-time office of assessment that reports directly to the laboratory director. The assessment staff is internal but separate from the program areas. There is also a standing panel for external independent review of these Sandia assessments. These assessments are a critical part of the regular certification of the U.S. nuclear weapons stockpile.

In 2010, the Congress mandated that the NNSA contract with the National Research Council to conduct a review of the quality of the science and engineering, as well as its management, at LANL, LLNL, and Sandia. That review is ongoing at the time of the publication of this report.

Entities that award grants—for example, the NSF, the Army Research Office (ARO), the ONR, and the AFOSR—do not operate laboratories and, therefore, conduct quality reviews in a different manner. The NSF is overseen by the National Science Board, which reports to the Congress. Each NSF directorate is monitored by a formal advisory committee that meets regularly to review performance. Periodically, under the auspices of the committee the grant folders are reviewed to ensure that procedures have been followed. Grant proposals are sent out to experts for evaluation; subsequently the folders are evaluated by NSF staff before the decision is made to award or not to award. For work in progress, grantees are visited on-site by NSF program managers. Quality is judged by these reviews, regular reports, and examination of publications. The ultimate indication of how well a grantee is doing is the renewal or termination of the grant. This is true for all of the granting agencies.

The ARO conducts two kinds of peer review concerning single investigator (SI) proposals for new work. One review evaluates technical merit. Typically the proposal is sent to external reviewers, mostly university faculty. The other review focuses on military relevance and is done by Army and DOD scientists and engineers. The SI grants are typically for 3 years. Two or more site visits are usually conducted during this time. Program managers audit grantees’

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement