5
Conclusions and Recommendations
The Information Sciences Directorate (ISD) is producing high-quality research. In general, however, the ISD programs did not evince clear and consistent sets of metrics by which to evaluate program impact and effectiveness. It is necessary that metrics be counted by consistent and transparent methods (e.g., what is meant by a publication, how to count graduate students who are supported by multiple programs) to facilitate measurement of progress. Metrics should reflect accomplishments actually attributable to the Army Research Office (ARO) projects. The appendix of this report lists a broad set of metrics that ARO could consider for assessment of its programs.
Recommendation 1: The Information Sciences Directorate should develop and apply a set of clear and consistent metrics by which to evaluate program impact and effectiveness.
The division follows a system of establishing personal connections between the program managers and their principal investigators (PIs); this almost amounts to collaboration. This system is effective, but it runs the risk that research foci might not change on appropriate time scales and that promising alternative approaches to problems might be missed if they fall outside the knowledge and experience base of the program managers. The system would benefit from deliberate efforts to inject more competition among different research approaches. This would include more rapid turnover in the PI base.
Recommendation 2: The Information Sciences Directorate should consider ways to expand the knowledge base beyond that possessed by the program managers when formulating approaches to selecting programs for funding.
Recommendation 3: To the extent that program managers in the Information Sciences Directorate demonstrate management of successful programs and expanded knowledge of their discipline and of relevant opportunities to support research with potential application to Army needs, they should be encouraged to exercise their visions for basic science to meet Army needs and be encouraged to maintain their entrepreneurial style in program management.
Recommendation 4: The Information Sciences Directorate should consider shorter time scales and more rapid turnover of the principal investigator base for projects that are not jointly funded or targeted for long-term funding by collaborating Information Sciences Directorate divisions, Army Research Office (ARO) directorates, or other funding agencies. Consideration should include potential impacts on graduate students supporting funded projects, should ARO deem graduate student support a project goal.
The ARO supports research by PIs and by centers of multiple researchers. In contrast with single investigator programs, the Multidisciplinary University Research Initiative (MURI) programs at the ARO support centers whose efforts intersect more than one traditional research specialty, typically at $1.25
million per year for 5 years. Research topics increasingly benefit from such multidisciplinary participation, even in pairs or small sets of investigators and over shorter time periods. Including in such collaborations researchers with knowledge of transitions would be useful.
Recommendation 5: The Information Sciences Directorate should consider funding mechanisms to encourage pairs or small sets of researchers from divergent perspectives to work on the same problem.
ARO is organizationally a subset of the Army Research Laboratory (ARL), which in turn resides within the Army’s Research, Development, and Engineering Command (RDECOM) along with the Command’s other research and development centers. ARO program managers interact formally with their colleagues at ARL and RDECOM by collaborative reviews of research project proposals, status reports, and Army operational concept documents, which are also provided by the Army’s Training and Doctrine Command (TRADOC). The ARO Core Program also organizes and facilitates scientific and technical conferences, workshops, and symposia attended by Army, commercial, and other Department of Defense (DoD) agencies. This program provides a method for conducting scientific and technical meetings that facilitate the exchange of scientific information relevant to the long-term basic research interests of the Army and help define research needs, thrusts, opportunities, and innovation. In particular, workshops are a key mechanism that ARO uses to identify new research areas with the greatest opportunities for scientific breakthroughs that will revolutionize future Army capabilities. ARO program managers also establish and maintain less formal interactions with other DoD agencies.
The program managers in the ISD evinced varying levels of engagement with other DoD research, development, and funding agencies such as the Office of Naval Research, the Air Force Office of Scientific Research, the Defense Advanced Research Projects Agency, and other elements within the Army, such as the Research, Development, and Engineering Centers. Such engagement is important for the maintenance of shared situation awareness and is a key enabler for ARO to continue to “outpunch its weight.”
Recommendation 6: Program managers within the Information Sciences Directorate should maintain and seek to expand their engagement with other Department of Defense funding agencies such as the Office of Naval Research, the Air Force Office of Scientific Research, the Defense Advanced Research Projects Agency, and other elements within the Army.
Diversity of gender, age, and geographic location was acknowledged across the ARO as requiring attention.
Recommendation 7: The Information Sciences Directorate should continue encouraging the participation of females and minorities in research funded by the Army Research Office and should collect statistics to track diversity in the broad sense, including gender, age, and geographic location.
The material provided for this review described a limited set of projects funded by the programs. The review could have been augmented by providing copies of the broad agency announcements for all of the currently funded projects, a more complete and organized description of all of the projects, and abstracts of the projects.
There was a wide variety in the content and relevance of the presentations about individual projects. A consistent template for the description of projects could address in sufficient detail such questions as the following:
- How does the project map to the long-term vision for the program?
- What are major consequences for the science if the project succeeds?
- Why is the problem difficult and what are the major technical risks?
- What is the network of contacts involved in the project?
- How, specifically, does the project address one or more critical challenges that the Army of the future will face?
