1

Introduction

The National Academies of Sciences, Engineering, and Medicine¹ has, starting in 1959, annually assembled panels of experts—from academia, industry, medicine, and other scientific and engineering communities of practice—to assess the quality and effectiveness of the National Institute of Standards and Technology (NIST) measurements and standards laboratories, of which there are now seven,² as well as the adequacy of the laboratories’ resources. These reviews are conducted under contract at the request of the NIST.

In 2018, at the request of the NIST Director, the National Academies formed the Panel on Review of the Information Technology Laboratory of the National Institute of Standards and Technology (the “panel”) and established the following statement of work:

NIST specified that the following four divisions of the Information Technology Laboratory (ITL) would be reviewed: the Applied and Computational Mathematics Division (ACMD); the Advanced Network Technologies Division (ANTD); the Computer Security Division (CSD); and the Applied Cybersecurity Division (ACD). All four of these divisions are located in Gaithersburg, Maryland, or nearby Rockville, Maryland (at the National Cybersecurity Center of Excellence [NCCoE]), and were visited by the panel on June 12-14, 2018.

The divisions described their purposes as follows: perform research in the mathematical sciences to nurture trust in NIST metrology and scientific computing (ACMD);³ establish the technical basis for trustworthy networking via standards, measurement science, test methods, reference implementations, and guidance (ANTD);⁴ cultivate information technology’s roots of trust (CSD);⁵ and improve the management of cybersecurity and privacy risk (AMD).⁶

___________________

¹ Effective July 1, 2015, the institution is called the National Academies of Sciences, Engineering, and Medicine (NASEM). References in this report to the National Research Council (NRC) are used in a historical context to refer to activities before July 1.

² The seven National Institute of Standards and Technology (NIST) laboratories are the Engineering Laboratory, the Physical Measurement Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Communication Technology Laboratory, the Center for Nanoscale Science and Technology, and the NIST Center for Neutron Research.

³ NIST website, https://www.nist.gov/itl/math, accessed October 15, 2018.

⁴ Abdella Batou, NIST, “Advanced Network Technologies Division,” presentation to the panel, June 12, 2018.

⁵ Matthew Scholl, NIST, “Computer Security Division,” presentation to the panel, June 12, 2018.

⁶ Kevin Stine, NIST, “Applied Cybersecurity Division,” presentation to the panel, June 12, 2018.

TABLE 1.1 Personnel Working at the Information Technology Laboratory (ITL)

NOTE: The column ITL (Total) includes all divisions, not only the four under review. NOTE: ACMD = Applied and Computational Mathematics Division, ANTD = Advanced Network Technology Division, CSD = Computer Security Division, ACD = Applied Cybersecurity Division.

As of May 2018, there were 760 personnel working in ITL, roughly three-fifths of them in the four divisions reviewed, as shown in Table 1.1.

In 2015, a National Academies panel reviewed the other three divisions of the ITL—the Information Access Division, the Software and Systems Division, and the Statistical Engineering Division—and summarized its findings in a 2015 report.⁷ In 2011, a National Academies panel reviewed all seven of the divisions of the ITL and summarized its findings in a 2011 report.⁸

For the current review, the NIST Director requested that the panel consider the following factors: the quality of the research and its relevance to the purposes of the divisions; the adequacy of the scientific expertise within the divisions; the adequacy of the divisions’ facilities, equipment, and human resources; and the effectiveness by which the organization disseminates its program outputs. The panel’s approach to the assessment relied on the experience, technical knowledge, and expertise of its members. The panel reviewed selected examples of the technical research performed at the four divisions of ITL; because of time constraints, it was not possible to review the programs and projects of these four divisions exhaustively. The examples reviewed by the panel were selected by the ITL. The panel’s goal was to identify and report salient examples of accomplishments, challenges, and opportunities for improvement with respect to the factors suggested above by the NIST Director. These examples are intended collectively to portray an overall impression of the laboratory, while preserving useful suggestions specific to the projects and programs that the panel examined. Given the necessarily broad and nonexhaustive nature of the review, omission in this report of any particular ITL program or project should not be interpreted as implying any negative reflection on the omitted program or project.

___________________

⁷ NASEM, 2015, Review of Three Divisions of the Information Technology Laboratory at the National Institute of Standards and Technology: Fiscal Year 2015, The National Academies Press. Washington, D.C.

⁸ NRC, 2011, An Assessment of the Information Technology Laboratory at the National Institute of Standards and Technology: Fiscal Year 2011, The National Academies Press, Washington, D.C.