7

Crosscutting Recommendations

The metrics through which ARL, as a research organization, internally measures and quantifies the quality of its S&T research across the spectrum of its mission space were not provided to the ARLTAB. The options could include the number and impact factor of publications or the number of transitions to operational use by the warfighter. Definition of such metrics and any relevant data could enhance the impact of the ARLTAB assessments.

The mix of low-risk and high-risk research to achieve an optimal balance continues to be a crosscutting issue for all of ARL’s S&T programs. ARL indicated that 30-50 percent of Director’s Research Initiative (DRI) projects and many Director’s Strategic Initiative (DSI) projects go on to become core efforts.1 While ARL is looking for ways to encourage innovation that will impact mission-critical programs, making it safe to fail is one such way to encourage innovative, high-risk projects. Internal research investments in successful innovations and high-risk research expectations are goals in conflict. Risky research is likely to fail most of the time. If success is expected of nearly all projects supported by discretionary funds, staff cannot be expected to propose risky ones. Strategic management discussion of the objectives and expectations for DRI and DSI projects and how these precious funds are aligned or feed longer-term programmatic efforts is encouraged.

The visibility of ARL staff in professional technical societies and technical conferences is not up to the level that their accomplishments and scientific expertise warrant. While it is clear that the sequestration and travel restrictions have negatively affected staff interactions with the outside R&D community, the long-term continuation of restrictions on external technical interactions will have a significant adverse impact. Lack of interactions normally fostered through conferences and professional associations will negatively impact both collaborative programmatic efforts and maintenance of an edge in areas of expertise. This has clearly already affected ARL staff morale, produced opportunity costs, and will seriously impact staff retention and hiring in the future. Moreover, ARL’s strategic focus on innovation through adoption and development of scientific ideas and insights from the scientific community cannot be applied to solve Army problems if the focus is solely inward. If sustained, a not-invented-here syndrome will be nearly impossible to avoid, leading to internal reinvention of wheels that would be better brought in from outside.

Steps to improve the overall ARL research enterprise include the following:

Recommendation 1. ARL should require researchers to clearly articulate the existing technical challenges in their research and how and why proposed tools and methods are likely to resolve those challenges.

Recommendation 2. To allow for setting meaningful goals and adopting a research agenda that targets nonincremental advances, ARL should require all researchers to identify precise metrics against which progress can be gauged.

__________________

1 ARL uses the DSI and DRI research projects to build new research capabilities in long-term, high-risk scientific areas with very high potential payoff to the Army mission. DSI projects are typically $500,000 to $1 million per year, while DRI projects are $250,000 per year.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 71
7 Crosscutting Recommendations The metrics through which ARL, as a research organization, internally measures and quantifies the quality of its S&T research across the spectrum of its mission space were not provided to the ARLTAB. The options could include the number and impact factor of publications or the number of transitions to operational use by the warfighter. Definition of such metrics and any relevant data could enhance the impact of the ARLTAB assessments. The mix of low-risk and high-risk research to achieve an optimal balance continues to be a crosscutting issue for all of ARL’s S&T programs. ARL indicated that 30-50 percent of Director’s Research Initiative (DRI) projects and many Director’s Strategic Initiative (DSI) projects go on to become core efforts. 1 While ARL is looking for ways to encourage innovation that will impact mission-critical programs, making it safe to fail is one such way to encourage innovative, high-risk projects. Internal research investments in successful innovations and high-risk research expectations are goals in conflict. Risky research is likely to fail most of the time. If success is expected of nearly all projects supported by discretionary funds, staff cannot be expected to propose risky ones. Strategic management discussion of the objectives and expectations for DRI and DSI projects and how these precious funds are aligned or feed longer-term programmatic efforts is encouraged. The visibility of ARL staff in professional technical societies and technical conferences is not up to the level that their accomplishments and scientific expertise warrant. While it is clear that the sequestration and travel restrictions have negatively affected staff interactions with the outside R&D community, the long-term continuation of restrictions on external technical interactions will have a significant adverse impact. Lack of interactions normally fostered through conferences and professional associations will negatively impact both collaborative programmatic efforts and maintenance of an edge in areas of expertise. This has clearly already affected ARL staff morale, produced opportunity costs, and will seriously impact staff retention and hiring in the future. Moreover, ARL’s strategic focus on innovation through adoption and development of scientific ideas and insights from the scientific community cannot be applied to solve Army problems if the focus is solely inward. If sustained, a not- invented-here syndrome will be nearly impossible to avoid, leading to internal reinvention of wheels that would be better brought in from outside. Steps to improve the overall ARL research enterprise include the following: Recommendation 1. ARL should require researchers to clearly articulate the existing technical challenges in their research and how and why proposed tools and methods are likely to resolve those challenges. Recommendation 2. To allow for setting meaningful goals and adopting a research agenda that targets nonincremental advances, ARL should require all researchers to identify precise metrics against which progress can be gauged. 1 ARL uses the DSI and DRI research projects to build new research capabilities in long-term, high-risk scientific areas with very high potential payoff to the Army mission. DSI projects are typically $500,000 to $1 million per year, while DRI projects are $250,000 per year. 71

OCR for page 71
Recommendation 3. As ARL continues to build its research staff, it should give some attention to bringing in mid-career and senior personnel to mentor the outstanding early-career scientists who have been recruited. Recommendation 4. ARL should look for additional ways to increase interaction of its researchers with leaders in industry and academia, given that limitations on travel have restricted this important function. Recommendation 5. ARL should focus on developing a mature framework to guide the conception, design, development, and testing of small, unmanned autonomous systems, including definitions of pertinent parameters and their domain (values). Recommendation 6. ARL should adopt a systems integration approach as a fundamental research thrust. As the intersection of modeling and simulation with experimental measurements grows, it requires a coherent treatment of verification and validation across ARL. Model validation was insufficiently defined and elucidated during the review for the majority of the projects presented. Some excellent examples of validation were shown, such as in the MOUT project, but this was not seen throughout the review. Too often, a computer-based visualization of a model was presented with little or no quantitative comparisons to data. Details of complex material and structural models matter, but these, along with the basis for choosing model parameter values, were seldom discussed. When geometry or assumptions of material behavior are considerably simplified, it is important to provide data justifying such approximations. The success of a model in reproducing a visual image of the overall phenomenology is not validation. To map out regions to define and limit experiments, delineation is needed on a project by project basis as to whether validation is sought via a detailed comparison with quantitative data or via the ability to predict trends in response or in performance. A rigorous formal internal validation program is needed within ARL to quantify whether the physics within the broad spectrum of ballistics models being developed accurately describes the operative physics. Given the importance of such models to the development of predictive design capability in support of current Army programs and future system, platform, and equipment development, increased emphasis on validation is warranted. In addition to the need for an ARL-wide strategic approach to model validation, methods are needed to quantify the margin of uncertainty (QMU) for these models. For example, it is not clear how the operational requirements-based casualty assessment (ORCA) and MUVES-S2 models are validated. The reviews often lack sufficient details on how ARL’s models are formulated and validated; the sensitivity, if known, to key parameters and variables; and the statistical variations to be expected. The details of how ARL is leveraging ARO’s 6.1 investment in support of the near-term and long-term Army strategic vision was not always clearly presented to the ARLTAB panels. Examples of how individual ARO projects fit into Army overall goals and relate to one another and to other ARL projects would facilitate the ARLTAB’s tasking, to assess the quality of ARL’s S&T. 72