Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
8 Information Technology Laboratory 71
72 AN ASSESSMENT OF THE NIST MEASUREMENT AND STANDARDS LABORATORIES: FY 2003 PANEL MEMBERS Albert M. Erisman, Institute for Business, Technology, and Ethics, Chair C. William Gear, NEC Research Institute, Inc. (retired), Vice Chair Michael Angelo, Hewlett-Packard Corporation Robert Blakley, Tivoli Systems John Boot, Motorola Corporation Linda Branagan, Secondlook Consulting Jack Brassil, Hewlett-Packard Laboratories James A. Calvin, Texas A&M University Susan T. Dumais, Microsoft Research John R. Gilbert, Xerox Palo Alto Research Center Cyndi Jung, Motorola Corporation Sallie Keller-McNulty, Los Alamos National Laboratory Stephen T. Kent, BEN Technologies Isaac S. Kohane, Children's Hospital, Boston Lawrence O'Gorman, Avaya Labs David R. Oran, Cisco Systems Jeffrey D. Ullman, Stanford University Stephen A. Vavasis, Cornell University Samaradasa Weerahandi, TDS Research, AOL/Time Warner Mary Ellen Zurko, IBM Software Group Submitted for the panel by its Chair, Albert M. Erisman, and its Vice Chair, C. William Gear, this assessment of the fiscal year 2003 activities of the Information Technology Laboratory is based on visits by members of the panel to the ITL divisions, a site visit by the panel on March 24-25, 2003, in Gaithersburg, Maryland, and documents provided by the laboratory. 1U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technology, Information Technology Laboratory Technical Accomplishments 2002, NISTSP 6909, National Institute of Standards and Technology, Gaithersburg, Md., November 2002; U.S. Department of Commerce, Technology Administration, National Institute of Stan- dards and Technology, Report to the ITL Assessment Panel, National Institute of Standards and Technology, Gaithersburg, Md., March 2003; U.S. Department of Commerce, Technology Administration, National Institute of Standards and Technol- ogy, Information Technology Laboratory Publications 2002, National Institute of Standards and Technology, Gaithersburg, Md., March 2003.
INFORMATION TECHNOLOGY LABORATORY 73 LABORATORY-LEVEL REVIEW The mission of the Information Technology Laboratory (ITL) is to develop and promote measure- ment, standards, and technology for information technology (IT) to enhance productivity, facilitate trade, and improve the quality of life. To carry out this mission, the laboratory is organized in seven divisions (see Figure 8.1~: Mathematical and Computational Sciences, Advanced Networking Tech- nologies, Computer Security, Information Access, Convergent Information Systems, Software Diagnos- tics and Conformance Testing, and Statistical Engineering. This chapter presents an assessment of the laboratory overall, discussing some highlights and overarching issues. A selection of the activities of these units is commented on at length in the division reviews in Chapter 15. Software Diagnostics and Conformance Testing Division · Software Quality · Interoperability · Standards and Conformance Testing | Information Technology Laboratory Mathematical and Formation Technology Laboratory Advanced Net, ore Computational Sciences l l l | Technologies ~ ·v~s~ ;; l l Division · High Speed Networking · Mathematical Modeling Technologies · Mathematical Software · Wireless Communications · Optimization and Technologies Computational Geometry · Internetwork · S i tif A it ti~ I Technologies ~ ~ Information Access Divis ion _ Convergent Information · Speech Systems Division · Retrieval · Distributed Systems · Image Technologies · Visualization and Usability · Information Storage and ~ | I Integrated Systems Computer Security Division · Security Technology · Security Management and Guidance · Systems and Network Security · Security Testing and Metrics Statistical Engineering Division · Measurement Process Evaluation · Statistical Modeling and Analysis · Boulder Statistics FIGURE 8.1 Organizational structure of the Information Technology Laboratory. Listed under each division are its groups.
74 AN ASSESSMENT OF THE NIST MEASUREMENT AND STANDARDS LABORATORIES: FY 2003 Major Observations The panel presents the following major observations from its assessment of the Information Tech- nology Laboratory: · The overall technical quality and the merit, relevance, and effectiveness of the Information Technology Laboratory's programs and staff remain strong. Examples of high-quality, meritorious, relevant, and effective projects are many and exist in all divisions. · There is ample evidence of outstanding work in leveraging technology ideas across customer areas for industry, academia, government, and within NIST. · Collaboration across ITL divisions and between ITL and other NIST laboratories is increasing, and projects increasingly are demanding multidisciplinary team approaches. In the current flat or declin- ing funding environment, such projects (e.g., in pervasive computing) demand especially careful man- agement. · Progress has been made in terminating good but less important projects. This skill needs to become even more widespread as demands for work expand without commensurate expansion of resources. · It is important that ITL staff take into account not only the scientific and technical aspects of projects but also their psychosocial context, so that the laboratory avoids the mistake of endorsing products that cause psychosocial problems (e.g., lack of integrity leading to users' mistrust). A particu- larly important new project that raises this concern involves electronic voting. · The transfer of IT support functions to the Office of the Chief Information Officer (CIO), which is separate from ITL, holds promise for garnering NIST-wide recognition of the importance of standard- ized, reliable IT service. Both the CIO office and the ITL should maintain effective, cooperative communication and activity to ensure the application of IT technology and practice appropriate to NIST. · ITL has worked hard and effectively to develop metrics for its performance. ITL should work with customers and the panel should assist to further develop means of assessing the effectiveness of ITL projects and products. · The panel commends ITL for its careful work with consortia, properly participating in the standards activity while avoiding the endorsement of vendor-specific products. The panel urges ITL to continue its efforts to refine and implement its policy to help divisions decide when participation in closed consortia is appropriate and to consider how NIST can encourage industry to utilize open, or at least inclusive, approaches to standards development. · Skip-level meetings with the panel and employee surveys confirm that morale within ITL is high: ITL members enjoy their work, take pride in its quality and effectiveness, and appreciate the atmosphere of mutual respect. They expressed some confusion about the plans to fill the director's position; about the import of Office of Management and Budget Circular A-76, which establishes federal policy for the competition of commercial activities; and about impacts of outsourcing explanation by management would allay this confusion. · The housing of part of the ITL staff at NIST North inhibits, to some extent, the ability of staff to collaborate (especially junior staff who have not established a collaborative network). It is time for ITL to apply its knowledge for the benefit of its own staff by providing effective IT workarounds such as electronic collaboration technologies to ameliorate this problem. Nevertheless, NIST should continue to work toward more consolidated facilities because of the power of face-to-face activity.
INFORMATION TECHNOLOGY LABORATORY 75 Technical Merit The technical merit of the work in ITL remains strong. As part of its onsite reviews, the panel had the opportunity to visit each of the divisions for a variety of presentations and reviews related to the projects currently under way. On the basis of its sampling of ITL projects, the panel has been consis- tently impressed with the technical quality of the work undertaken. Many examples of programs with especially strong technical merit are highlighted in the division reviews in Chapter 15 and include the following: · Within the Mathematical and Computational Sciences Division: . ~ ~ ~ The work on solidification modeling is a long-running, collaborative effort with the Materials Science and Engineering Laboratory that has yielded significant advances in modeling capability. The Digital Library of Mathematical Functions project is ambitious, excellently performed, and important. · Within the Advanced Networking Technologies Division: The Internet Telephony project has continued its considerable progress, with a focus on call signaling protocols; this project remains a model for industrial collaboration and balances the mainte- nance of existing software tools with the need to advance the division's research agenda. The first-responders project is a promising and ambitious exploratory project that is well matched with the division's research competencies. · Within the Computer Security Division: The Cryptographic Module Validation program is an important and well-conducted effort that continues to uncover and correct a large number of flaws in algorithm implementation and documenta- tion, providing a common definition of "assurance" for users of those modules. The system administration guidance for the Windows 2000 Professional project can have an impact on Microsoft's own secure configuration development efforts; can establish a configuration with known security properties for use by system administrators, application developers, and auditors; and represents exemplary cooperation with industry (Microsoft). · Within the Information Access Division: The efforts in support of the Text Retrieval Conference (TREC) continue to represent leader- ship and coordination of research, with a focus on key common and relevant problems in the field of information retrieval. Work in the area of evaluation methodologies and standards to support usability and accessi- bility is in good alignment with the division's strengths and represents good progress in this high-impact area. · Within the Software Diagnostics and Conformance Testing Division: Due to its outstanding track record in working successfully with industry and providing tech- nical leadership and unbiased feedback, the Standards and Conformance Testing Group has expanded its conformance testing beyond extensible markup language (XML) to include related technologies. Work in the area of health care information systems is well conceived, promising, and impor- tant. · Within the Statistical Engineering Division: The division continues to be the leading presence in the area of key statistical comparisons, developing new and effective methods to help determine the degree of equivalence among measurement standards of different nations.
76 AN ASSESSMENT OF THE NIST MEASUREMENT AND STANDARDS LABORATORIES: FY 2003 The division provides important statistical support to the scientific research conducted across all NIST laboratories. which ranges from short-term activities to extensive collaborations requiring the generation of new statistical methodology. · Within the Convergent Information Systems Division: The work on digital preservation (content storage) represents effective collaboration with industry, government, and universities to develop metrology methods, technologies, and standards that support efforts by the data storage industry and government efforts in digital preservation, a national issue. · The quantum communication testbed is an enabler of important first steps in exploring the viability of cryptographic key distribution as a practical technology for commercial and defense appli- cations. The testbed is also an enabler of test, calibration, and development efforts in the area of quantum communication. · Cross-divisional projects: Several cross-divisional efforts are being applied to the field of biometrics. The Information Access Division is performing good work in the accumulation and testing of large fingerprint and face test sets for the purpose of evaluating verification and identification rates. With the large databases, its Image Group is expected to be able to determine statistics and also to evaluate operational issues that will occur with very high volumes of enrollment and verification (for example, in border and visa security). There is new and productive cooperation between the Convergent Information Systems Divi- sion (which has been performing critical work on the development of standards, prototypes, and test methods and data for multimodal biometrics technologies) and the Information Access Division (which is expanding its role beyond support of government and law enforcement). Whereas interest and support came mainly from the FBI in the past, a wider range of government agencies are now supporting and making reference to NIST recommendations on biometrics. Pervasive computing efforts are also being conducted effectively by the Information Access Division and by the Advanced Networking Technologies Division, which is addressing wireless net- working and standards and performing analysis of the resource discovery protocols being developed for ubiquitous computing systems. An example of cross-laboratory interaction is the use of the Smart Space testbed, part of the pervasive computing infrastructure, for the Single Molecule Manipulation and Measurement project undertaken by NIST's Chemical Science and Technology Laboratory and the Physics Laboratory. The panel also continues to see progress in the divisions on rational, welljustified decisions about what projects to start and conclude and when to do so. The panel urges ITL to vigorously address two important issues that may help to improve the technical quality and merit of the laboratory' s work: 1. The psychosocial factors related to electronic voting should be analyzed so that potential prob- lems (e.g., issues pertaining to the verification of votes and related issues such as implications for trust in automated systems, amplified by private ownership of the voting system and related security con- cerns) are identified and so that NIST is not perceived as endorsing the use of a possibly problematic system. 2. The expanded interest in and support of NIST's biometrics work, as well as the importance of the timely development of biometrics for homeland security, encourage the development of a clear plan for integrating the efforts across divisions so that resources are applied effectively and priorities are ad- dressed, and for tapping relevant divisional expertise. For example, it will be important to expand the
INFORMATION TECHNOLOGY LABORATORY 77 Involvement of the Computer Security Division in this field, and the Statistical Engineering Division could make substantial contributions with appropriate experimental design and modeling support in many experimental and simulation contexts. Program Relevance and Effectiveness ITL has a very broad range of customers, including scientists and engineers, from industry, academia, and government and from within NIST, and the panel found that the laboratory serves these groups with distinction. The high level of outside funding is an indicator of highly relevant work. A related indicator is the high level of interaction between laboratory staff and their customers. Attendance is generally _ _ 1 , · 1 1 1 ,~ 1 1 1 1 1 ,1 1 1 , , Or ,~ good at seminars, workshops, and meetings led and sponsored by the laboratory; stats participation In standards organizations and consortia is strong; and laboratory staff have robust relationships with researchers and users from companies, governmental agencies, and universities. Another visible measure of the quality and relevance of ITL's work is the number of awards that laboratory staff receive from NIST, the Department of Commerce, and external sources. Examples of internal recognition include a Department of Commerce Gold Medal awarded to three staff members and Bronze Medals awarded to two staff members. Staff members received external recognition from the Federal Bridge Certification Authority and the Laboratory Consortium for Technology Transfer. External awards also included the National Committee for Information Technology Standards (NCITS) Merit and Chairman's Awards; American Physical Society and American Statistical Association fellow- ship awards; an International Committee for Information Technology Standards (INCITS) Merit Award; a Percy S. Julian Award; and an Arthur. S. Fleming Award. These honors, spread across the various divisions, recognize outstanding technical and program achievement at numerous levels. ITL's interactions with and impact on industrial customers continue to be strong, and the panel applauds the laboratory's ability to produce and disseminate results of value to a broad audience. ITL primarily serves two kinds of industrial customers: computer companies (i.e., makers of hardware and software) and the users of their products (which include companies from all sectors, the government, and the public). The division reviews in Chapter 15 contain many examples of how ITL makes a difference to industrial customers. For example: · NIST's familiarity with the networking community and its reputation for an unbiased technical approach are useful in defining the technical matters on which the standards bodies should focus. One recent impact is the Advanced Networking Technologies Division's leadership within the Internet Engineering Task Force (IETF's) investigations of Domain Name System Security (DNSSec) and Internet Protocol Security (IPSec), cumulatively leading to the publication of seven IETF requests for comments. · The usability reporting work done by the Information Access Division has involved strong industry participation in defining and using the Common Industry Format (CIF) standards. The certifi- cation by the American National Standards Institute (ANSI) and the ongoing ISO process mean that standards will be more widely adopted in the future. NIST's work contributing to the growing recogni- tion of usability as a key component of software procurement is important. · The Software Diagnostics and Conformance Testing Division leads several laboratory-wide initiatives in health informatics that promise to be of significant consequence to the health care delivery community. · The Statistical Engineering Division has been involved in the establishment of baseline data sets to be used in the assessment and evaluation of the computational accuracy of statistical software. In the
78 AN ASSESSMENT OF THE NIST MEASUREMENT AND STANDARDS LABORATORIES: FY 2003 area of key comparisons, the division is working directly with CIPM, national metrology institutes, and regional metrology organizations to establish sound statistical principles for the determination of the basis for global transactions using various measurement standards of different nations. · The Computer Security Division's Cryptographic Module Validation program has demonstrated its effectiveness in improving the security and quality of cryptographic products. About 50 percent of the cryptographic modules tested had security flaws, and over 95 percent had documentation errors. About 25 percent of the algorithms submitted for evaluation had security flaws, and over 65 percent had documentation errors. Detecting these problems enables vendors and implementers to correct their products before the modules and algorithms are put into production and bought and used by consumers. · Thousands of scientists use the mathematical resources of the Mathematical and Computational Sciences Division's Web site annually. The division's Digital Library of Mathematical Functions will be a very relevant tool for use by the scientific and engineering communities. The federal government relies significantly on ITL's products and expertise and often uses NIST standards and evaluation tools to guide its purchase and use of information technology. Salient ex- amples include these: · The Computer Security Division's Cryptographic Module Validation program has enabled pur- chasers, including the U.S. government, to be sure that the security attributes of the products they buy are as advertised and appropriate. · The Information Access Division's Retrieval Group has a wide range of customers. Government agencies (most notably the Advanced Research and Development Activity of the intelligence commu- nity and the Defense Advanced Research Projects Agency) work closely with the Retrieval Group to evaluate the success of new information access technologies funded by their programs. In addition, hundreds of participants from government agencies, industry, and academia take part in the annual TREC program. Participation in TREC continues to increase, and it evolves as new tracks are added and old ones are phased out to reflect emerging retrieval challenges. The Retrieval Group's expertise and experience in developing new evaluation frameworks are highly valued by government agencies and are critical in evaluating the success of new technologies. TREC's customers are more diverse, including industry and academia in addition to government agencies. In addition, the division' s Image Group has responded rapidly to address requirements of the USA PATRIOT Act of 2001 to design a test program and compile a large test set of fingerprint and face images. · The biometrics work done at NIST is contracted to it by government customers and is highly relevant to homeland security needs. The effectiveness of much of the current work related to homeland security cannot be measured as yet. However, NIST biometrics contracts demonstrate that the customers believe the group to produce effective results. Much of the Information Technology Laboratory's work supports multiple types of customers (e.g., industry and government). For example, the new equipment created by the Convergent Information Systems Division for image quality analysis, biometrics, and quantum laboratories is critical to the success of industry and government progress in these areas. Web site statistics also suggest the external relevance of the Computer Security Division. From January 2002 through February 2003, approxi- mately 1.3 million Web site requests were handled each month. In addition to strong relationships with customers in industry and in the federal government, ITL has traditionally placed significant emphasis on effectively serving its customers within NIST. Collabora- tive work is highlighted above and is discussed in detail in the division reports in Chapter 15. Beyond
INFORMATION TECHNOLOGY LABORATORY 79 these collaborative efforts, the Statistical Engineering Division conducts an outreach and education effort, providing courses on an ongoing basis as well as excellent resources, such as the NIST/ SEMATECH e-Handbook of Statistical Methods, supporting the NIST community and beyond. Challenges and Opportunities The examples mentioned above highlight the Information Technology Laboratory's high level of relevance and effectiveness, its very good customer connections, and its outstanding work in leveraging technology ideas across customer areas. Continued success should involve consideration of challenges, obstacles, and questions to be addressed, such as the following: · The Information Technology Laboratory has achieved a good strategic focus, which is shared and understood across staff. The laboratory should be alert to and avoid natural tendencies to rely on "analyst and favorite customer" relationships when such relationships undermine the strategic focus. · The laboratory should continue its efforts to align its strategic focus with the overall NIST strategic plan, especially in areas such as knowledge management and manufacturing optimization. This effort should be carried out at the strategic level with the engagement of the cognizant laboratory directors. · The laboratory engages in projects that are multidisciplinary, both across its divisions and in interaction with other NIST laboratories. Examples include biometrics, quantum computing, security, modeling, statistics, and pervasive computing. The laboratory staff and management have expressed a clear understanding of the multidisciplinary aspects of such work, and this understanding should be formalized in program management that shows methods and plans for inter- and intralaboratory multi- disciplinary collaboration. · Measures of effectiveness can be elusive and, without clearly expressed underlying assumptions, misleading. Not all measures are appropriate for all activities. The panel commends the efforts by laboratory staff to develop and try out such measures. The panel also suggests that the laboratory management and staff consider whether it would be useful, generally or for specific projects, for the panel to interact directly with customers, who might be able to provide useful feedback pertaining to the effectiveness of the laboratory. Customers might be able to suggest metrics that can be used to demon- strate the value of technology dissemination, and customer feedback might constitute pertinent data for a metric currently used by NIST. · The laboratory serves many customers. Which customers are most strategically important to the laboratory, and how should work and dissemination be tailored to the characteristics of specific types of customers? For example, the Mathematical and Computational Sciences Division is performing high- quality work in collaboration with the Building and Fire Research Laboratory in the area of modeling and visualization of concrete hardening; this work supports a fragmented industry that consists largely of small firms. It is important that such work be performed with cognizance of the mechanisms by which it will be introduced to and used by the community; the panel would derive benefit in future briefings from hearing about how plans for the deployment of results are influencing work. · By what rationale would an appropriate target be defined for the percentage of the laboratory's funds that should come from outside sources? · The panel commends the responsiveness of the ITL management and staff to the panel's sug- gested focus on improving the number and quality of publications. The laboratory has certainly done so. However, the panel recognizes that journal publication, a commonly accepted metric for effectiveness in scientific disciplines such as physics and chemistry, is likely not the most effective metric for some
80 AN ASSESSMENT OF THE NIST MEASUREMENT AND STANDARDS LABORATORIES: FY 2003 areas of computer science, for which presentation at top-level conferences is a more appropriate metric. This distinction needs to be widely understood within ITL and NIST to ensure that proper recognition for researchers is achieved. The panel also urges the Information Technology Laboratory to vigorously address three important issues highlighted in the 2002 report: 1. A primary traditional responsibility of ITL has been to provide IT support for all of NIST. The relevant activities which include the support and maintenance of campus networking, personal com- puters, administrative applications (such as accounting software), and telephones were traditionally performed by the Information Services and Computing Division of ITL. During the past year, the IT responsibility was transferred from ITL to a separate unit, Technology Services, headed by a chief information officer who will report directly to the NIST director. Since a significant challenge for the ITL involved convincing the NIST laboratories to embrace consistent, institution-wide standards for IT systems, raising the Technology Services unit to a level equivalent with that of the laboratories was expected to provide needed visibility for the issue. Achieving acceptance of this new unit and of centralized IT support across NIST remains a serious challenge; this approach will be a cultural shift for NIST. Making the IT services component of NIST a separate unit rather than a division of ITL may bring it closer to other laboratories: however. it is important that this unit maintain close ties with ITL --= - - - - - - - - - - - --- - - --- - - - -- - - -- - - 7 -- - - - - 7 - - -- ----I - - - ---- - ----- - ----- ----- - -------- - ----- - - - - - -- - - - --- - - ~ . ~ .. . . · . . .. ~ . ~ ., ~ . . . . . .. programs. For example, some of the work nemg clone In the computer Security L,lvlslon can and should be applied to the security of the NIST system. Work on technologies for meetings can be tested and used effectively throughout NIST. Applying the development work of ITL's research divisions to NIST as a whole will require the continued tracking by the CIO office of relevant, ongoing projects, as well as recognition in ITL of the potential for using NIST as a whole as a testbed. The Information Technolo~v Laboratorv should maintain interaction with and support the efforts of the now-separate CIO office. lo,, 2. Programs such as the work on biometrics, especially face recognition, highlight a question relevant to many information technology activities: In what context will technological advances be used? Information technology is often an enabling technology that will produce new capabilities with expected and unexpected benefits and costs. Although ITL focuses primarily on technical questions and technical quality, it is important that the laboratory demonstrate recognition of the context in which new technologies will be applied so that the results of its work will be taken seriously in the relevant communities. This context has two elements: the deployment of the technology and its social implica- tions. The deployment questions relate to the functionality of the systems in which new technical capabilities will be used. A testbed is not necessarily meant to determine the "best" technology but rather the one that works well enough to meet the needs for which the technology is being developed. Often, the process of considering the possible applications of a technology results in a broader apprecia- tion of the potential benefits. For example, appropriate security is actually an enabler that allows e- business, the globalization of work, and collaboration across geography. Understanding the goals for new technologies relates to addressing their social implications. For example, maintaining security has serious implications for privacy. The panel emphasizes that in many of the ongoing programs such as the work on the potential use of face-recognition technologies in security systems in public places ITL staff have made long and arduous efforts to comply with existing privacy legislation. However, when describing the capabilities and benefits of technological advances to public groups (such as the panel), staff should also be sure to take the time to acknowledge related privacy issues and describe potential social implications. This concern extends to many signifi-
INFORMATION TECHNOLOGY LABORATORY 8 cant ongoing areas of work at the laboratory, including biometrics (especially face and voice recogni- tion) and voting technology (with special concerns for system integrity and privacy). 3. In last year's assessment report,2 the panel expressed concerns about industry trends in standards development that would affect ITL's ability to effectively and openly help industry adopt the most appropriate standards for emerging technologies. The growing use of consortia and other private groups in standards development processes places a burden on ITL, which has to strike a balance between its obligation to support and encourage open processes and its need to be involved as early as possible in standards-setting activities so as to maximize the impact of ITL's experience and tools. In some cases, a delicate trade-off must be made between participating in a timely way in organizations that will set standards for the industry and avoiding endorsement of standards set by exclusive groups. ITL's role as a neutral third party and its reputation as an unbiased provider of technical data and tools have produced a significant impact in many areas and should not be squandered by association with organizations that unreasonably restrict membership. Though the situation has not changed materially since last year, there is some indication that consortia are, due to a proactive stance by NIST, addressing participation issues affecting NIST; the Java Specification Participation Agreement is cited as a model. The panel continues to urge ITL to refine and implement its policy to help divisions decide when participation in closed consortia is appropriate and to consider how NIST can encourage industry to utilize open, or at least inclusive, approaches to standards development. Given that consortia, in some form or another, are here to stay and that in some cases it will be vital for NIST to participate in them, the panel supports recent ITL and NIST efforts to work on the internal legal roadblocks to participation, and it urges continued and expanded efforts to educate external groups, such as consortia members and lawyers, on ways to facilitate NIST's timely participation and technical input. This effort requires customer outreach as well as resolution of legal issues. Laboratory Resources Funding sources for the Information Technology Laboratory are shown in Table 8.1. In January 2003, staffing for the laboratory included 247 full-time permanent positions. There were also 96 nonper- manent or supplemental personnel, such as postdoctoral research associates and temporary or part-time workers. The panel notes that the funding for the laboratory has been flat or declining in the face of growth in requirements, some of which like the unfunded homeland defense mandates are at a very high level. This situation, predicted to continue for the near future, demands careful strategic planning and prioritiza- tion and increases the importance of well-managed collaborative activities and of applying the tests of relevance and effectiveness to ongoing tasks. The Software Diagnostics and Conformance Testing Division has demonstrated particular adroitness at identifying and closing tasks that have effectively completed their usefulness, and this division's approach might be usefully examined by other divisions. The panel has observed and laboratory staff have explicitly stated that morale is at an all-time high in ITL, due in large part to the director's leadership style and direction. The panel reemphasizes its recommendation, offered in the FY 2002 assessment report, that NIST leadership focus on communicat- ing clearly with staff about the selection criteria for new hires and the progress being made in the search and hiring process. Sharing relevant information will certainly help ensure a smooth transition. 2National Research Council, An Assessment of the National Institute of Standards and Technology Measurement and Standards Laboratories: Fiscal Year 2002, National Academies Press, Washington, D.C., 2002.
82 AN ASSESSMENT OF THE NIST MEASUREMENT AND STANDARDS LABORATORIES: FY 2003 TABLE 8.1 Sources of Funding for the Information Technology Laboratory (in millions of dollars), FY 2000 to FY 2003 Fiscal Year Fiscal Year Fiscal Year Fiscal Year 2000 2001 2002 2003 Source of Funding (actual) (actual) (actual) (July 2003 estimate) NIST-STRS, excluding Competence Competence STRS Supercomputing ATP Measurement Services (SRM production) OA/NFG/CRADA Other Reimbursable Agency Overhead Total Full-time permanent staff (totally 31.9 1.6 12.0 2.4 0.0 9.9 1.6 16.4 75.8 381 44.4 1.1 11.9 2.3 0.1 12.2 1.0 18.4 91.4 368 46.7 1.7 42.0 1.1 1.9 0.2 12.2 16.1 29.4 92.1 401a 1.5 0.2 0.0 61.0 247 NOTE: Funding for the NIST Measurement and Standards Laboratories comes from a variety of sources. The laboratories receive appropriations from Congress, known as Scientific and Technical Research and Services (STRS) funding. Compe- tence funding also comes from NIST's congressional appropriations but is allocated by the NIST director's of lice in multiyear grants for projects that advance NIST's capabilities in new and emerging areas of measurement science. Advanced Technol- ogy Program (ATP) funding reflects support from NIST's ATP for work done at the NIST laboratories in collaboration with or in support of ATP projects. Funding to support production of Standard Reference Materials (SRMs) is tied to the use of such products and is classified as Measurement Services. NIST laboratories also receive funding through grants or contracts from other [government] agencies (OA), from nonfederal government (NFG) agencies, and from industry in the form of cooperative research and development agreements (CRADAs). All other laboratory funding, including that for Calibration Services, is grouped under "Other Reimbursable." aThe number of full-time permanent staff is as of January of that fiscal year, except in FY 2002, when it is as of March (due to a reorganization of ITL that year). The staff total for 2003 reflects the transfer of 257 full-time permanent staff members to the CIO organization, as well as staff changes in other divisions. The existence and use of NIST North is a perennial issue. The panel recognizes that the quality of the space in NIST North is significantly better than that available on campus; however, access to these improved facilities does not currently compensate for separating the Mathematical and Computational ~ . a_ . . . . .. ~ . .. .. . ~ . . a_ . . . .~ .. . .~ .. aim. .. Sciences L,lvlslon and the ~tatlstlca1 englneermg L,lvlslon from the rest ot the campus. l he separation inhibits informal interactions of the staff of these two divisions with their collaborators in the other laboratories on the main campus; this is particularly problematic for new staff, who are faced with the special challenge of developing collaborations. Although NIST management is addressing the issue, the panel continues to note that a mix of systems, taking into account technological and social factors, could help compensate for the separation. Information technology tools such as videoconferencing, Web collaboration packages, and Web broadcasting can support nonphysical interactions, although regular, scheduled, and subsidized opportunities for face-to-face meetings are necessary to make these technical solutions most effective. These approaches are applicable to the NIST North/main campus gap, as well as to the Gaithersburg/Boulder divide. The Information Technology Laboratory should be adept at demonstrating such tools for its own use and that of other laboratories.
INFORMATION TECHNOLOGY LABORATORY 83 A second facilities issue raised in the FY 2002 assessment report and repeated this year is the substandard network connectivity of NIST to the outside world; laboratory staff noted the advantages of Internet 2 connectivity, which should be explored. The panel met with staff in skip-level meetings (sessions in which management personnel were not present). These meetings confirmed the impression, also conveyed universally by staff and management who presented their work to the panel, that morale is very high: staff consider ITL an enjoyable place to work and appreciate its positive attributes, such as opportunities for career growth and training, respect for the individual, stability, an appropriate level of flexibility, a focus on visible results, and connection to a world-class bank of expertise at NIST. The panel applauds laboratory and division management for creating such a positive work environment. The panel did note a small set of staff uncertainties (specifi- cally, confusion about intellectual property processes, anxiety about potential staffing impacts of Office of Management and Budget Circular A-76, concern that outsourcing of infrastructure support may make it unreliable, and concern that junior staff be mentored effectively to establish connectivity with custom- ers) and recommends that laboratory management provide clear explanations to alleviate uncertainties. Laboratory Responsiveness The panel found that, in general, ITL has been very responsive to prior recommendations and observations. The panel's comments appear to be taken very seriously, and the suggestions made in the assessment reports are often acted on, especially as they relate to the redirection and conclusion of projects. When advice is not taken, ITL usually provides a good rationale for why a given action has not occurred. Examples of positive responses to suggestions made in last year's report include these: · The Convergent Information Systems Division addressed funding shortfalls, devoted additional attention to student interns and guest researchers, expanded involvement with external organizations, improved the flexibility of its business plan, performed impact studies to describe the impact on industry or society of several of the division's projects, and improved the content and usability of its Web site. · The Mathematical and Computational Sciences Division strengthened its strategic planning, expressing recognition that the demand for its work is increasing more rapidly than its resources. The division also tracked the life cycle of programming projects by creating ratings for projects depending on the maintenance level.
