1
Introduction
The National Academies of Sciences, Engineering, and Medicine has, starting in 1959, annually assembled panels of experts—from academia, industry, medicine, and other scientific and engineering communities of practice—to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now six,1 as well as the adequacy of the laboratories’ resources. These reviews are conducted under contract at the request of the NIST.
In 2019, at the request of the Director of NIST, the National Academies formed the Panel on Review of the Communications Technology Laboratory of the National Institute of Standards and Technology (the “panel”), having earlier established the following statement of work:
The NASEM shall form a panel of experts who shall perform an independent technical assessment of the quality of the National Institute of Standards and Technology’s (NIST’s) Communications Technology Laboratory (CTL). The panel shall prepare a final report that reflects the expert consensus opinion of the panelists based on materials provided by NIST before and during a site visit to NIST laboratories in Boulder, CO. During this site visit, the panel shall conduct an in-depth technical assessment and draft a consensus report addressing the topics requested by the NIST Director in his or her charge to the panel. Subsequent to the site visit, the panel shall finalize the report draft and the NASEM shall conduct any necessary reviews to ensure that the final report is of high quality and is impartial and objective in its assessment of the NIST CTL.
The assessment shall be responsive to the charge from the NIST Director. The following draft criteria for the assessment are proposed by the NIST sponsor and are expected to be formalized in the charge of the NIST Director that will be provided at contract award, expected November 1, 2018.
- The technical merit of the current laboratory program relative to current state-of-the-art programs worldwide;
- The portfolio of scientific expertise as it supports the ability of the organization to achieve its stated objectives;
- The adequacy of the laboratory budget, facilities, equipment, and human resources, as they affect the quality of the laboratory’s technical programs; and
- The effectiveness by which the laboratory disseminates its program outputs.
For this review, the CTL director asked that the panel focus on two main categories: public safety communications and metrology for advanced communications.2 Public safety communications addresses the CTL priority area of the same name. Metrology for advanced communications addresses three of the
___________________
1 The six National Institute of Standards and Technology (NIST) laboratories are the Communications Technology Laboratory, the Engineering Laboratory, the Information Technology Laboratory, the Material Measurement Laboratory, the Physical Measurement Laboratory, and the Center for Neutron Research.
2 NIST, Undated, “This is CTL,” U.S. Department of Commerce, Boulder, Colo., p. 1.
four CTL priority areas: (1) trusted spectrum testing, (2) fundamental metrology for communications, and (3) Next Generation Wireless (5G and Beyond).3
The four organizational units of CTL are located in Boulder, Colorado, and in Gaithersburg, Maryland. The Boulder facilities were visited by the panel on June 25-27, 2019.
The panel’s approach to the assessment relied on the experience, technical knowledge, and expertise of its members. Time constraints did not allow the panel to explore all aspects of CTL. Rather, the panel focused on the research that the leadership of the CTL chose to present to it and on a number of issues related to laboratory development that the panel identified as requiring particular attention. The panel’s report includes recommendations that specify “who should do what” to address any determinations as to what might warrant action. The panel underpins the Recommendations with salient examples of programs and projects that are intended collectively to portray an overall impression of the laboratory, while preserving useful suggestions specific to projects and programs. Key recommendations are those that the panel considers especially worthy of attention.
To accomplish its mission, the panel reviewed the material provided by the CTL prior to and during the review meeting. The choice of projects to be reviewed was made by the CTL. The panel applied a largely qualitative approach to the assessment. Given the nonexhaustive nature of the review, the omission in this report of any particular CTL project should not be interpreted as a negative reflection on the omitted project.
___________________
3 Marla Dowell, Ph.D., Director CTL, “Communications Technology Laboratory Overview,” presentation to the panel on June 25, 2019.