National Academies Press: OpenBook
« Previous: Summary
Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×

1
The Charge to the Panel and the Assessment Process

At the request of the National Institute of Standards and Technology, the National Research Council has, since 1959, annually assembled panels of experts from academia, industry, medicine, and other scientific and engineering environments to assess the quality and effectiveness of the NIST measurements and standards laboratories, of which there are now nine,4 as well as the adequacy of the laboratories’ resources. In 2010, NIST requested that five of its laboratories be assessed: the Building and Fire Research Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics Laboratory. A separate panel of experts assessed each of these; the findings of the respective panels are summarized in separate reports. This report summarizes the findings of the Panel on Manufacturing Engineering.

For the fiscal year (FY) 2010 assessment, NIST requested that the panel consider the following criteria as part of its assessment:

  1. The technical merit of the current laboratory programs relative to current state-of-the-art programs worldwide;

  2. The adequacy of the laboratory budget, facilities, equipment, and human resources, as they affect the quality of the laboratory’s technical programs; and

  3. The degree to which laboratory programs in measurement science, standards, and services achieve their stated objectives and desired impact.

The panel adopted the following additional assessment criteria to make the broad factors more explicit. These criteria were only suggested to guide the assessment process and did not need to be individually addressed.

  1. Technical Quality and Merit Assessment Criteria

    • Relative Technical Caliber:

      • How does the technical quality of the laboratory programs compare to current state-of-the-art programs worldwide?

      • Does the laboratory work product consistently demonstrate evidence of high technical quality (including but not limited to papers in high-impact technical publications, invited talks to major scientific and industry conferences and workshops, and external awards and recognition)?

4

The nine NIST laboratories are the Building and Fire Research Laboratory, the Center for Nanoscale Science and Technology, the Chemical Science and Technology Laboratory, the Electronics and Electrical Engineering Laboratory, the Information Technology Laboratory, the Manufacturing Engineering Laboratory, the Materials Science and Engineering Laboratory, the NIST Center for Neutron Research, and the Physics Laboratory.

Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×
  • Distinctness:

    • Do the projects reflect mission focus?

    • Is the value proposition distinctive and in keeping with NIST’s role and strategy?

  • Relevance:

    • Is there a clear tie between projects and NIST and MEL strategic priority areas?

    • Do the projects reflect a broad understanding of comparable work being done elsewhere (in other government laboratories, universities, and industry)?

    • Are there demonstrable links between NIST researchers and the external community?

  • Balance:

    • Does the laboratory adequately balance anticipatory, longer-term research and activities that respond to immediate customer needs?

  1. Adequacy of Budget, Facilities, Equipment, and Human Resources Assessment Criteria

    • Available Tools:

      • Is the state of the equipment and facilities adequate to meet project objectives and customer needs?

    • Critical Mass:

      • Are the available scientific and technical competencies adequate to achieve success?

      • Is available funding adequate to achieve success?

    • Agility:

      • Is the laboratory sustaining the technical competencies and capacity to respond quickly to critical issues as they arise?

  1. Achievement of Stated Objectives Assessment Criteria

    • Technical Planning:

      • Are appropriate milestones identified? Do they appear feasible?

      • Are obstacles and challenges defined (technical, resources)?

      • Are the roles of the investigators clear, and are the individual project tasks and objectives clearly linked to those of other projects within a given strategic priority area?

    • Dissemination:

      • Is the laboratory regularly implementing sound and effective techniques and practices for delivering products and services?

      • Are the results of the laboratory projects readily available to stakeholders?

    • Impact:

      • How well do the laboratory programs in measurement science, standards, and services achieve their stated objectives and desired impact?

      • Will the laboratory products have a consequential, long-term impact?

Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×
  • Level of effort:

    • Are research projects scaled appropriately to meet the technical problems being addressed?

  • Responsiveness:

    • Are the research projects moving at a pace and in a direction that is well matched to current and emerging stakeholder needs?

  1. Other Considerations

    • Collaboration and crosscutting within a given laboratory

    • Project-level technical planning

The context of this technical assessment is the mission of NIST, which is to promote U.S. innovation and industrial competitiveness by advancing measurement science, standards, and technology in ways that enhance economic security and improve the quality of life. The NIST laboratories conduct research to anticipate future metrology and standards needs, to enable new scientific and technological advances, and to improve and refine existing measurement methods and services.

In order to accomplish the assessment, the NRC assembled a panel of 15 volunteers, whose expertise matches that of the work performed by the MEL staff.5 The panel members were also assigned to four subgroups (division review teams), whose expertise matched that of the work performed in the four divisions reviewed in the MEL: (1) Intelligent Systems, (2) Manufacturing Metrology, (3) Manufacturing Systems Integration, and (4) Precision Engineering.

The panel met at the NIST facilities in Gaithersburg, Maryland, on March 8-10, 2010. After the full panel had met for a session of welcoming remarks from the NIST Director’s representative and an overview presentation on the MEL by MEL management, the panel divided into its four review teams, and each team (led by a team leader chosen from within the panel) then visited with the staff of its respective MEL division for about a day. During these visits, the review team members attended presentations, tours, demonstrations, and interactive sessions with the MEL staff. Subsequently, the entire panel assembled for about a day-and-a-half meeting, during which it interacted with MEL and NIST management and also met in a closed session to deliberate on its findings and to define the contents of this assessment report.

The approach of the panel to the assessment relied on the experience, technical knowledge, and expertise of its members, whose backgrounds were carefully matched to the technical areas of MEL activities. The panel reviewed selected examples of the technological research covered by the MEL; because of time constraints, it was not possible to review the MEL programs and projects exhaustively. The MEL selected the examples reviewed by the panel. The panel’s goal was to identify and report salient examples of accomplishments and opportunities for further improvement with respect to the following: the technical merit of the MEL work, its perceived relevance to NIST’s own definition of its mission in support of national priorities, and specific elements of the MEL’s resource infrastructure that are intended to support the technical work. These examples are intended collectively to portray an overall impression of the laboratory,

Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×

while preserving useful suggestions specific to projects and programs that the panel examined. The assessment is currently scheduled to be repeated biennially, which will allow, over time, exposure to the broad spectrum of MEL activity. While the panel applied a largely qualitative rather than a quantitative approach to the assessment, it is possible that future assessments will be informed by further consideration of various analytical methods that can be applied.

The comments in this report are not intended to address each program within the MEL exhaustively. Instead, this report identifies key issues. Given the necessarily nonexhaustive nature of the review process, the omission of any particular MEL program or project should not be interpreted as a negative reflection on the omitted program or project.

Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×
Page 9
Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×
Page 10
Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×
Page 11
Suggested Citation:"1 The Charge to the Panel and the Assessment Process." National Research Council. 2010. An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010. Washington, DC: The National Academies Press. doi: 10.17226/13009.
×
Page 12
Next: 2 Intelligent Systems Division »
An Assessment of the National Institute of Standards and Technology Manufacturing Engineering Laboratory: Fiscal Year 2010 Get This Book
×
Buy Paperback | $21.00 Buy Ebook | $16.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The mission of the Manufacturing Engineering Laboratory (MEL) of the National Institute of Standards and Technology (NIST) is to promote innovation and the competitiveness of U.S. manufacturing through measurement science, measurement services, and critical technical contributions to standards. The MEL is organized in five divisions: Intelligent Systems, Manufacturing Metrology, Manufacturing Systems Integration, Precision Engineering, and Fabrication Technology. A panel of experts appointed by the National Research Council (NRC) assessed the first four divisions.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!