Appendix C
Panel Assessment Criteria
The ARLTAB applies the following metrics or criteria to the assessment of the scientific and technical work reviewed at the Army Research Laboratory (ARL):
-
Effectiveness of Interaction with the Scientific and Technical Community
-
Papers in quality refereed journals and conference proceedings (and their citation index)
-
Presentations and colloquia
-
Participation in professional activities (society officers, conference committees, journal editors)
-
Educational outreach (serving on graduate committees, teaching or lecturing, invited talks, mentoring students)
-
Fellowships and awards (external and internal)
-
Review panel participation (Army Research Office, National Science Foundation, Multidisciplinary University Research Imitative)
-
Recruiting new talent into the ARL
-
Patents and Intellectual Property (IP) (and examples of how the patent or IP is used)
-
Involvement in building an ARL-wide cross-directorate community
-
Public recognition (e.g., in the press and elsewhere) for ARL research
-
-
Impact on Customers
-
Documented transfer or transition of technology, concepts or program assistance from ARL to Research, Development, and Engineering Centers (RDECs) or RDEC contractors for both the long term and short term
-
Direct funding from customers to support ARL activities
-
Documented demand for ARL support or services (Is there competition for their support?)
-
-
-
Customer involvement in directorate planning
-
Participation in multidisciplinary, cross-directorate projects
-
Surveys of customer base (direct information from customers on value of ARL research)
-
-
Formulation of Projects’ Goals and Plans
-
Is there a clear tie to ARL Strategic Focus Areas, Strategic Plan, or other ARL need?
-
Are tasks well defined to achieve objectives?
-
Does the project plan clearly identify dependencies (i.e., successes depend on success of other activities within the project or outside developments)?
-
If the project is part of a wider activity, is role of the investigators clear, and are the project tasks and objectives clearly linked to those of other related projects?
-
Are milestones identified if they are appropriate? Do they appear feasible?
-
Are obstacles and challenges defined (technical, resources)?
-
Does the project represent an area where application of ARL strengths is appropriate?
-
-
R&D Methodology
-
Are the hypotheses appropriately framed within the literature and theoretical context?
-
Is there a clearly identified and appropriate process for performing required analyses, prototypes, models, simulations, tests, etc.?
-
Are the methods (e.g., laboratory experiment, modeling or simulation, field test, analysis) appropriate to the problems? Do these methods integrate?
-
Is the choice of equipment or apparatus appropriate?
-
Is the data collection and analysis methodology appropriate?
-
Are conclusions supported by the results?
-
Are proposed ideas for further study reasonable?
-
Do the tradeoffs between risk and potential gain appear reasonable?
-
If the project demands technological or technical innovation, is that occurring?
-
What stopping rules, if any, are being or should be applied?
-
-
Capabilities and Resources
-
Are the qualifications and number of the staff (scientific, technical, administrative) appropriate to achieve success of the project?
-
Is funding adequate to achieve success of the project?
-
Is the state of the equipment and facilities adequate?
-
If staff, funding, or equipment are not adequate, how might the project be triaged (what thrust should be emphasized, what sacrificed?) to best move toward its stated objectives?
-
Does the laboratory sustain the technical capability to respond quickly to critical issues as they arise?
-
-
Responsiveness to the Board’s Recommendations
-
Have the issues and recommendations presented in the previous report been addressed?
-