National Academies Press: OpenBook

2009-2010 Assessment of the Army Research Laboratory (2011)

Chapter: Appendix C: Assessment Criteria

« Previous: Appendix B: Membership of the Army Research Laboratory Technical Assessment Board and Its Panels
Suggested Citation:"Appendix C: Assessment Criteria." National Research Council. 2011. 2009-2010 Assessment of the Army Research Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/13137.
×

Appendix C
Assessment Criteria

The Army Research Laboratory Technical Assessment Board’s assessment considered the following general questions posed by the ARL Director:

  1. Is the scientific quality of the research of comparable technical quality to that executed in leading federal, university, and/or industrial laboratories both nationally and internationally?

  2. Does the research program reflect a broad understanding of the underlying science and research conducted elsewhere?

  3. Does the research employ the appropriate laboratory equipment and/or numerical models?

  4. Are the qualifications of the research team compatible with the research challenge?

  5. Are the facilities and laboratory equipment state of the art?

  6. Does the research reflect an understanding of the Army’s requirement for the research or the analysis?

  7. Are programs crafted to employ the appropriate mix of theory, computation, and experimentation?

  8. Is the work sufficiently unique and appropriate to the ARL niche?

  9. Are there especially promising projects that, with application of adequate resources, could produce outstanding results that could be transitioned ultimately to the field?

The Board applied the following metrics or criteria to the assessment of the scientific and technical work reviewed at the Army Research Laboratory (ARL):

  1. Effectiveness of Interaction with the Scientific and Technical Community

    1. Papers in quality refereed journals and conference proceedings (and their citation index)

    2. Presentations and colloquia

    3. Participation in professional activities (society officers, conference committees, journal editors)

Suggested Citation:"Appendix C: Assessment Criteria." National Research Council. 2011. 2009-2010 Assessment of the Army Research Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/13137.
×
  1. Educational outreach (serving on graduate committees, teaching or lecturing, invited talks, mentoring students)

  2. Fellowships and awards (external and internal)

  3. Review panel participation (Army Research Office, National Science Foundation, Multidisciplinary University Research Initiative)

  4. Recruiting new talent into the ARL

  5. Patents and intellectual property (IP) (and examples of how the patent or IP is used)

  6. Involvement in building an ARL-wide cross-directorate community

  7. Public recognition (e.g., in the press and elsewhere) for ARL research

  1. Impact on Customers

    1. Documented transfer or transition of technology, concepts, or program assistance from ARL to Research, Development, and Engineering Centers (RDECs) or RDEC contractors for both the long term and short term

    2. Direct funding from customers to support ARL activities

    3. Documented demand for ARL support or services (is there competition for ARL’s support?)

    4. Customer involvement in directorate planning

    5. Participation in multidisciplinary, cross-directorate projects

    6. Surveys of customer base (direct information from customers on value of ARL research)

  1. Formulation of Projects’ Goals and Plans

    1. Is there a clear tie to ARL Strategic Focus Areas, Strategic Plan, or other ARL need?

    2. Are tasks well defined to achieve objectives?

    3. Does the project plan clearly identify dependencies (i.e., successes depend on success of other activities within the project or outside developments)?

    4. If the project is part of a wider activity, is role of the investigators clear, and are the project tasks and objectives clearly linked to those of other related projects?

    5. Are milestones identified if they are appropriate? Do they appear feasible?

    6. Are obstacles and challenges defined (technical, resources)?

    7. Does the project represent an area where application of ARL strengths is appropriate?

  1. Research and Development Methodology

    1. Are the hypotheses appropriately framed within the literature and theoretical context?

    2. Is there a clearly identified and appropriate process for performing required analyses, prototypes, models, simulations, tests, etc.?

    3. Are the methods (e.g., laboratory experiment, modeling or simulation, field test, analysis) appropriate to the problems? Do these methods integrate?

    4. Is the choice of equipment or apparatus appropriate?

    5. Is the data collection and analysis methodology appropriate?

    6. Are conclusions supported by the results?

    7. Are proposed ideas for further study reasonable?

    8. Do the trade-offs between risk and potential gain appear reasonable?

    9. If the project demands technological or technical innovation, is that occurring?

    10. What stopping rules, if any, are being or should be applied?

Suggested Citation:"Appendix C: Assessment Criteria." National Research Council. 2011. 2009-2010 Assessment of the Army Research Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/13137.
×
  1. Capabilities and Resources

    1. Are the qualifications and number of the staff (scientific, technical, administrative) appropriate to achieve success of the project?

    2. Is funding adequate to achieve success of the project?

    3. Is the state of the equipment and facilities adequate?

    4. If staff, funding, or equipment is not adequate, how might the project be triaged (what thrust should be emphasized, what sacrificed?) to best move toward its stated objectives?

    5. Does the laboratory sustain the technical capability to respond quickly to critical issues as they arise?

  1. Responsiveness to the Board’s Recommendations

    1. Have the issues and recommendations presented in the previous report been addressed?

Suggested Citation:"Appendix C: Assessment Criteria." National Research Council. 2011. 2009-2010 Assessment of the Army Research Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/13137.
×
Page 117
Suggested Citation:"Appendix C: Assessment Criteria." National Research Council. 2011. 2009-2010 Assessment of the Army Research Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/13137.
×
Page 118
Suggested Citation:"Appendix C: Assessment Criteria." National Research Council. 2011. 2009-2010 Assessment of the Army Research Laboratory. Washington, DC: The National Academies Press. doi: 10.17226/13137.
×
Page 119
Next: Appendix D: Acronyms »
2009-2010 Assessment of the Army Research Laboratory Get This Book
×
Buy Paperback | $34.00 Buy Ebook | $27.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The charge of the Army Research Laboratory Technical Assessment Board (ARLTAB) is to provide biannual assessments of the scientific and technical quality of the research, development, and analysis programs at the Army Research Laboratory (ARL). The advice provided in this report focuses on technical rather than programmatic considerations.

The Board is assisted by six National Research Council (NRC) panels, each of which focuses on the portion of the ARL program conducted by one of ARL's six directorates. When requested to do so by ARL, the Board also examines work that cuts across the directorates. The Board has been performing assessments of ARL since 1996. The current report summarizes its finding for the 2009-2010 period, during which 96 volunteer experts in fields of science and engineering participated in the following activities: visiting ARL annually, receiving formal presentations of technical work, examining facilities, engaging in technical discussions with ARL staff, and reviewing ARL technical materials.

The Board continues to be impressed by the overall quality of ARL's technical staff and their work and applauds ARL for its clear, passionate concern for the end user of its technology--the soldier in the field--and for ARL's demonstrated mindfulness of the importance of transitioning technology to support immediate and longer-term Army needs. ARL staff also continue to expand their involvement with the wider scientific and engineering community. In general, ARL is working very well within an appropriate research and development (R&D) niche and has been demonstrating significant accomplishments.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!