Appendix C

Assessment Criteria

The Army Research Laboratory Technical Assessment Board’s assessment considered the following general questions posed by the Army Research Laboratory (ARL) Director:

1. Is the scientific quality of the research of comparable technical quality to that executed in leading federal, university, and/or industrial laboratories both nationally and internationally?

2. Does the research program reflect a broad understanding of the underlying science and research-conducted elsewhere?

3. Does the research employ the appropriate laboratory equipment and/or numerical models?

4. Are the qualifications of the research team compatible with the research challenge?

5. Are the facilities and laboratory equipment state of the art?

6. Does the research reflect an understanding of the Army’s requirement for the research or the analysis?

7. Are programs crafted to employ the appropriate mix of theory, computation, and experimentation?

8. Is the work appropriate to the ARL niche?

9. Are there especially promising projects that, with application of adequate resources, could produce outstanding results that could be transitioned ultimately to the field?

The Board applied the following metrics or criteria to the assessment of the scientific and technical work reviewed at ARL:

1. Effectiveness of Interaction with the Scientific and Technical Community

a. Papers in quality refereed journals and conference proceedings (and their citation index)

b. Presentations and colloquia

c. Participation in professional activities (society officers, conference committees, journal editors)



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 135
Appendix C Assessment Criteria The Army Research Laboratory Technical Assessment Board’s assessment considered the following general questions posed by the Army Research Laboratory (ARL) Director: 1. Is the scientific quality of the research of comparable technical quality to that executed in lead- ing federal, university, and/or industrial laboratories both nationally and internationally? 2. Does the research program reflect a broad understanding of the underlying science and research- conducted elsewhere? 3. Does the research employ the appropriate laboratory equipment and/or numerical models? 4. Are the qualifications of the research team compatible with the research challenge? 5. Are the facilities and laboratory equipment state of the art? 6. Does the research reflect an understanding of the Army’s requirement for the research or the analysis? 7. Are programs crafted to employ the appropriate mix of theory, computation, and experimentation? 8. Is the work appropriate to the ARL niche? 9. Are there especially promising projects that, with application of adequate resources, could pro- duce outstanding results that could be transitioned ultimately to the field? The Board applied the following metrics or criteria to the assessment of the scientific and technical work reviewed at ARL: 1. Effectiveness of Interaction with the Scientific and Technical Community a. Papers in quality refereed journals and conference proceedings (and their citation index) b. Presentations and colloquia c.  articipation in professional activities (society officers, conference committees, journal P editors) 135

OCR for page 135
136 APPENDIX C d.  ducational outreach (serving on graduate committees, teaching or lecturing, invited talks, E mentoring students) e. Fellowships and awards (external and internal) f. Review panel participation (Army Research Office, National Science Foundation, Multidis- ciplinary University Research Initiative) g. Recruiting new talent into the ARL h. Patents and intellectual property (IP) (and examples of how the patent or IP is used) i. Involvement in building an ARL-wide cross-directorate community j. Public recognition (e.g., in the press and elsewhere) for ARL research 2. Impact on Customers a.  ocumented transfer or transition of technology, concepts, or program assistance from ARL D to Research, Development, and Engineering Centers (RDECs) or RDEC contractors for both the long term and short term b. Direct funding from customers to support ARL activities c. Documented demand for ARL support or services (is there competition for ARL’s support?) d. Customer involvement in directorate planning e. Participation in multidisciplinary, cross-directorate projects f. Surveys of customer base (direct information from customers on value of ARL research) 3. Formulation of Projects’ Goals and Plans a. Is there a clear tie to ARL Strategic Focus Areas, Strategic Plan, or other ARL need? b. Are tasks well defined to achieve objectives? c.  oes the project plan clearly identify dependencies (i.e., successes depend on success of other D activities within the project or outside developments)? d.  f the project is part of a wider activity, is role of the investigators clear, and are the project I tasks and objectives clearly linked to those of other related projects? e.  re milestones identified if they are appropriate? Do they appear feasible? A f. Are obstacles and challenges defined (technical, resources)? g.  oes the project represent an area where application of ARL strengths is appropriate? D 4. Research and Development Methodology a. Are the hypotheses appropriately framed within the literature and theoretical context? b.  s there a clearly identified and appropriate process for performing required analyses, proto- I types, models, simulations, tests, etc.? c.  re the methods (e.g., laboratory experiment, modeling or simulation, field test, analysis) A appropriate to the problems? Do these methods integrate? d. Is the choice of equipment or apparatus appropriate? e. Is the data collection and analysis methodology appropriate? f. Are conclusions supported by the results? g. Are proposed ideas for further study reasonable? h. Do the trade-offs between risk and potential gain appear reasonable? i. If the project demands technological or technical innovation, is that occurring? j. What stopping rules, if any, are being or should be applied?

OCR for page 135
APPENDIX C 137 5. Capabilities and Resources a.  re the qualifications and number of the staff (scientific, technical, administrative) appropriate A to achieve success of the project? b. Is funding adequate to achieve success of the project? c. Is the state of the equipment and facilities adequate? d.  f staff, funding, or equipment is not adequate, how might the project be triaged (what thrust I should be emphasized, what sacrificed?) to best move toward its stated objectives? e.  oes the laboratory sustain the technical capability to respond quickly to critical issues as D they arise? 6. Responsiveness to the Board’s Recommendations a. Have the issues and recommendations presented in the previous report been addressed?