5

Crosscutting Conclusions and Recommendations

TECHNICAL QUALITY OF THE WORK

The technical quality of the work is generally excellent. Information Access Division, Software and Systems Division, and Statistical Engineering Division continue to make significant contributions to address the Information Technology Laboratory (ITL) and NIST missions, national needs, and the needs of government, industry, and academic stakeholders. Amidst continuing technological and societal changes, there are opportunities for increased collaborations across projects to address common challenges in such areas as artificial intelligence, machine learning, health information technology, data science, and statistical methodology.

TECHNICAL EXPERTISE OF THE STAFF AND ADEQUACY OF STAFFING

Technical staff generally possess expertise adequate to perform their task. Some are among the best in the world in their areas of research. However, evolving needs in some areas (e.g., artificial intelligence, machine learning, and data science) will produce expertise gaps that will have to be filled.

In terms of human resources, recruitment of highly qualified full-time staff has been identified as an overall challenge at ITL due to competition with industry and academia, which offer salaries that exceed the limits available to the ITL. It may be difficult for small teams to be effective due to lack of critical mass in some areas. The relatively large proportion of the permanent workforce at retirement age in some areas raises the potential need for succession planning to ensure maintenance of competence in core areas while meeting the demand for new areas of competency. Current and future ITL needs cannot be met without the addition of new competencies to complement existing strengths.

RECOMMENDATION: ITL should apply an aggressive, imaginative focus on hiring to replace retiring staff, to address important growth areas such as artificial intelligence, machine learning, and data science, and to fill specific gaps in the divisions. This effort should aspire to diversity targets.

RECOMMENDATION: ITL should plan and implement effective ways to recruit and retain a diverse workforce to ensure the appropriate staffing in areas of significant interest to national welfare and security, and to address severe competition from industry in areas such as artificial intelligence, cybersecurity, and the Internet of Things.

RECOMMENDATION: ITL should establish exchange programs with relevant government laboratories, academic institutions, and industry consortia to stimulate new ideas and problem areas, enhance competencies, and facilitate collaboration.

Properly managed, ITL’s diversity, equity, and inclusion strategy can add a helpful element to the recruiting process. The staff has performed commendably during the COVID-19 pandemic, collaboratively addressing critically important responses to the pandemic.

ADEQUACY OF FACILITIES AND EQUIPMENT

Post-pandemic planning gives NIST opportunities to continue some remote work, making NIST a more attractive environment for staff and giving NIST a needed edge in recruiting. Also, creative thinking about the new work environment can lead to more productive use of facilities, which could be considered during NIST’s ongoing laboratory and office renovations.

ITL facilities are generally adequate and support well the activities of the divisions. There is a critical need, however, for improved computing capabilities that are needed to support complex computation and analysis. This is particularly the case for the growing number of projects that involve state-of-the-art machine learning models, which call for ever larger training sets and computational resources.

RECOMMENDATION: ITL should take steps to insure adequate resources, especially computing to support artificial intelligence, machine learning, and data science at sufficient scale.

RECOMMENDATION: To get access to the most modern resources, ITL should seek collaborations with other organizations in the public and private sectors, including other government agencies

EFFECTIVENESS OF THE DISSEMINATION OF OUTPUTS

Each division disseminates its outputs widely, but the divisions vary in the relative emphasis placed on dissemination vehicles (e.g., publications, workshops, data repositories, standard reference data, and educational programs). Careful, systematic, and continuing analysis of the needs of specific stakeholder communities would improve the effectiveness of the dissemination of ITL’s outputs.

RECOMMENDATION: ITL should broaden its impact to non-technical stakeholders, policy makers, and the public.

RECOMMDNATION: ITL should improve messaging aimed at non-technical audiences such as policy makers, media, and the general public for the outputs of the Information Access, Software and Systems, and Statistical Engineering Divisions.

This would benefit NIST by providing greater acceptance of and support for NIST efforts.