National Academies Press: OpenBook

Assessment of the Information Sciences Directorate at the Army Research Office (2019)

Chapter: Appendix: Possible Metrics for Assessment of Program and Project Accomplishments and Plans

« Previous: 5 Conclusions and Recommendations
Suggested Citation:"Appendix: Possible Metrics for Assessment of Program and Project Accomplishments and Plans." National Academies of Sciences, Engineering, and Medicine. 2019. Assessment of the Information Sciences Directorate at the Army Research Office. Washington, DC: The National Academies Press. doi: 10.17226/25426.
×

Appendix

Possible Metrics for Assessment of Program and Project Accomplishments and Plans

PROGRAM AND PROJECT ACCOMPLISHMENTS

  • Did the accomplishments represent significant scientific advances?
  • What are the major consequences for the science if the project succeeds?
  • Is the potential, long-term Army application of the research significant?
  • Is the research novel, leading the field in an important area, and does it have the appropriate level of risk and payoff?
  • Was related research being sponsored by other major players adequately summarized in terms of approach and goals? Were there areas of duplication?
  • When comparisons are made, are the comparison groups well defined?
  • How do the accomplishments map to the stated program goals and the Army Research Office (ARO) strategic plans?
  • Do the accomplishments reflect productivity and ingenuity on the part of the performers?
  • What portion of the accomplishments are attributable to the ARO funding?
  • What is the network of contacts involved in the project?
  • Were there appropriate examples of significant transitions, or anticipated transitions, of research to follow-on applied research or exploratory development either within industry or within an Army or a Department of Defense laboratory?
  • Are there any high-priority missed opportunities/areas relating to the program or project?
  • Was the speed of knowledge acceleration or transfer appropriate?
  • What are the number, type, and caliber of awards and recognition related to the program or project?
  • Has follow-up funding been awarded for the project or program?

PROGRAM PLANNING

  • How, specifically, does the project address one or more critical challenges that the Army of the Future will face?
  • Should research funding align topically with the current or the anticipated future focus of a division/program?
  • Is there a clear and cogent strategy regarding how each of the program managers’ major objectives are likely to make substantial and unique progress in advancing the scientific frontiers of their discipline?
  • Is there some reasonable basis (e.g., incipient breakthrough, new understanding, or novel theory) to believe that the scientific objectives might be met?
Suggested Citation:"Appendix: Possible Metrics for Assessment of Program and Project Accomplishments and Plans." National Academies of Sciences, Engineering, and Medicine. 2019. Assessment of the Information Sciences Directorate at the Army Research Office. Washington, DC: The National Academies Press. doi: 10.17226/25426.
×
  • Have the highest priority objectives been selected? If not, what is the potential that the accomplishments will lead to significant scientific advances?
  • What mixture of collaboration (within ARO and with outside teams/agencies) and competition is best? What percentage of projects should be unique to ARO?
  • If higher-priority areas are identified, what projects or programs should be reduced or eliminated to accommodate the new areas?
  • Are anticipated transitions within the Army likely to be greater than transitions to the commercial sector or other government agencies (e.g., Naval Research Laboratory or Office of Naval Research)?
  • Should funding portfolios be homogeneous or heterogeneous? Should funding of divergent or convergent works be preferred?
  • What diversified portfolio of short-/medium-/long-term; low-/high-risk; and collaborative versus ARO-only projects is best, and why?
  • What project type (e.g., preliminary exploration, longer-term research, workshop, or Multidisciplinary University Research Initiative) is best, and why?
  • What ratio of faculty, staff, and students is best for a given topic and type of project, and why?
  • What principal investigator (PI) turnover rate is best?
  • How closely should program managers work with PIs? Are research collaborations between both required for a given project?
  • What diversity ratios (e.g., gender, age, ethnicity) are best for a given project or program, and why?
  • What percentage of a given project or program should be undertaken by postdoctoral researchers and graduate students?
  • Are the most qualified applicants selected, independent of geospatial distribution?
  • What composition of researchers, practitioners, and entrepreneurs is best to ensure high-quality research results that can be translated into products?
  • What are the difficult aspects of a proposed project, and what are the major technical risks involved?
Suggested Citation:"Appendix: Possible Metrics for Assessment of Program and Project Accomplishments and Plans." National Academies of Sciences, Engineering, and Medicine. 2019. Assessment of the Information Sciences Directorate at the Army Research Office. Washington, DC: The National Academies Press. doi: 10.17226/25426.
×
Page 37
Suggested Citation:"Appendix: Possible Metrics for Assessment of Program and Project Accomplishments and Plans." National Academies of Sciences, Engineering, and Medicine. 2019. Assessment of the Information Sciences Directorate at the Army Research Office. Washington, DC: The National Academies Press. doi: 10.17226/25426.
×
Page 38
Assessment of the Information Sciences Directorate at the Army Research Office Get This Book
×
Buy Ebook | $9.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report summarizes the 2018 findings of the Panel on Review of Extramural Basic Research at the Army Research Laboratory, which reviewed the programs at the Army Research Office’s Information Sciences Directorate.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!