Summary

At the request of the U.S. Army, the National Academies of Sciences, Engineering, and Medicine (National Academies) formed the Research Program Review and Analysis Committee (RPAC) to perform an assessment of the In-House Laboratory Independent Research (ILIR) conducted at the Research, Development, and Engineering Centers (RDECs) of the U.S. Army’s Research, Development, and Engineering Command (RDECOM).¹

The RPAC’s review was guided by the following task statement provided by the National Academies:

Eight panels performed reviews of the ILIR during the period of November 2018 through December 2018 in support of the RPAC’s assessment in the areas listed in the task statement. Table S.1 identifies for each panel the RDECs and the related U.S. Army Space and Missile Defense Command Technical Center (SMDC) whose ILIR was reviewed by that panel.

___________________

¹ Subsequent to this review, the RDECOM was, under the Army Futures Command (AFC), renamed the Combat Capabilities Development Command (CCDC), and the RDECs were renamed to clarify their roles within the AFC. The names in effect at the time of the review have been maintained in this report.

TABLE S.1 In-House Laboratory Independent Research in Research, Development, and Engineering Centers Reviewed by the Research Program Review and Analysis Committee Panels

NOTE: AMRDEC, Aviation and Missile Research, Development, and Engineering Center; ARDEC, Armament Research, Development, and Engineering Center; CERDEC, Communications–Electronics Research, Development, and Engineering Center; ECBC, Edgewood Chemical Biological Center; TARDEC, Tank Automotive Research, Development, and Engineering Center; NSRDEC, Natick Soldier Research, Development, and Engineering Center; SMDC, Space and Missile Defense Command Technical Center.

The Department of Defense (DoD) describes the ILIR program of basic research as follows: “Each DoD Component that operates an R&D laboratory or center will support an ILIR program. Participating DoD Component R&D laboratories or centers will use ILIR funds to initiate and support efforts judged to be important or promising in the accomplishment of missions assigned to that laboratory or center. Each R&D laboratory or center will have wide latitude in the use of ILIR funds, subject to the approval of overall funding levels, to enable performance of innovative, timely, and promising work without requiring formal and prior approval that might delay normal funding authorization. ILIR funds will be used to support basic research efforts. The programs will have as their primary goals the performance of highest quality research in support of laboratory missions and the enhancement of factors that contribute to recruitment and retention of outstanding scientists and engineers.”²

Considering the findings of the eight panels, the RPAC summarized the findings pertaining to the eight areas of ILIR reviewed and the findings and recommendations that were common across more than one of the areas reviewed.

SUMMARY FINDINGS FOR EACH AREA OF RESEARCH

Chemistry Research

All presentations focused on fundamental research that provides opportunities for advancing basic science. ILIR addresses an apparent need for the training of an expanded cadre of skilled scientific staff with backgrounds appropriate to address problems unique to the Army. Because of their specific research focus, the RDECs have access to unique sets of scientific questions that require the development of both new experimental and theoretical tools and the integration of tools already developed by the scientific community. Because of limited resources, this is a need and, often, a challenge for the RDECs. In this context, the ILIR programs in chemistry are a beneficial activity for the Army and represent an effec-

___________________

² Department of Defense Instruction Number 3201.04, October 18, 2013: In-House Laboratory Independent Research (ILIR) Program.

tive mechanism for addressing the challenges associated with advancing fundamental knowledge while, at the same time, providing specialized training. Given the basic nature of ILIR, there are significant opportunities for enhancing the benefits of each ILIR project by engaging the expertise of other RDECs and by promoting greater interactions with the Army, other government laboratories, and the broader scientific community. Besides providing access to state-of-the-art knowledge, equipment, and software, expanding relationships with the scientific community at large would serve as an effective means of developing greater cross-pollination that would be beneficial to advance individual ILIR projects and enhance the training experience of ILIR researchers. In this context, periodically presenting ILIR projects at conferences and publishing the research findings in peer-reviewed journals would serve as a method of greater scientific community engagement and oversight by subject matter experts.

Computational Sciences Research

The research efforts reviewed were of good technical merit, addressed foundational topics, and were well aligned with the Army’s enduring mission needs. The projects supported the development of advanced capabilities to ensure that the Army retains access to the most comprehensive and up-to-date set of computational science tools to respond to its emerging needs. Approaches were appropriate to address the technical challenges and achieve the objectives of the projects. Most projects reviewed resulted in publications in the peer-reviewed literature and/or in conference proceedings and in external presentations. Most teams of investigators were generally cognizant of the state of knowledge in their respective fields of research and, in some cases, had established some level of external partnerships with the broader research community, as appropriate, because most topics being evaluated are active areas of research within industry and academia. Partners included the Army Research Laboratory (ARL) and academic institutions. Consequently, the research teams appropriately leveraged the efforts of the external community and adapted this body of knowledge to the development of capabilities targeted to the Army’s specific mission needs.

The development of advanced computational science capabilities—including multiscale materials simulations, deep learning algorithms, multi-agent optimization, and vehicle autonomy—cuts across virtually all Army mission areas and supports all mission needs. Consequently, these efforts need to be supported across the spectrum of basic and applied research programs and across organizations within the RDECs. The ILIR program is an important source of support to foster innovation and to synergize basic and applied research efforts in response to enduring and emerging mission needs and to ensure that the Army remains in a state of technical readiness to respond to emerging technological surprises. ILIR resources can help enable the integration and coordination of the Army’s various computational science efforts toward the development of integrated and comprehensive predictive simulation capabilities. In addition to the development of advanced capabilities, the ILIR program is an important source of support for the recruiting and retention of strategic hires necessary for the Army to attract the leadership-class workforce of the future. Programs such as the ILIR are strategically critical in stimulating innovative and competitive research toward the development of next-generation capabilities and in fostering the pursuit of bold ideas with high-risk—and possibly high-payoff—technical elements.

Electronics Research

The ILIR electronic research and development (R&D) program could transition to advanced development and fielded systems in each of the research areas presented: ground vehicle radio frequency antennas with low observability; bioinspired beyond-visual sensors that may enable local navigation

to complement or replace global positioning systems; soldier energy harvesting boots; lighter but efficient electric generators that support critical operations; and optical technologies for real-time tracking, identification, and discrimination.

The program is evolving staff, oversight, and mentoring to promote well-tuned research, but there is a need for stronger research connections to ARL and its Army Research Office (ARO) and for evaluation by external reviewers. Given funding constraints, there is a need to regularly assess the number and lifetime of projects to optimize their innovation and impact, and a need to foster longer-term research. Several paths, including strategic planning and mission assessment, greater range of publications, and expanding networking and collaboration with other Army laboratories and joint service activities, need to be pursued to improve the effectiveness and stature of the researchers. There is a need to maintain doctoral candidate research, add career research and growth opportunities, and provide recognition incentives.

Life Sciences Research

The ILIR program awards are competitive, provide opportunity for independent work, and are likely beneficial for morale and retention. However, they are also relatively small, short-term grants, and the work is often performed in relative isolation from the larger scientific community. Most were focused on important military-relevant basic science problems that were also being pursued in academic and industrial laboratories. Developing new knowledge and publication of results are program goals.

ILIR projects engage the students at the Armament Research, Development, and Engineering Center’s Armament Graduate School. An ambitious, well-planned project involved multichannel collection and integration of psychophysiologic data to predict behavior—in this case, whether or not to fire a weapon. The project dealt with challenging contemporary research issues that will require a large number of subjects to obtain predictive data and would benefit from input from outside investigators.

The Edgewood Chemical Biological Center has several life sciences–oriented ILIR projects, including the following. The project studying effects of quorum-sensing molecules on production of bacterial nanocellulose, a biofilm material with useful properties, focused on identifying the ability of acylated homoserine lactones to promote production and develop different, potentially useful morphologies. Progress is being made, but this is a competitive area of materials science, and the investigators need to stay abreast of related research. Another project seeks to determine the effects of various chemical warfare agents on endothelial barrier function using an innovative impedance assay and macromolecular tracer assay with a mechanistic focus on tyrosine kinase signaling. The area is important, but it is complicated by abundant prior work on the topic and the need to translate in vitro results to the in vivo effects of these agents. A third project examines the existence of epigenetic memory during bacterial adaptation to environmental changes; it will perform tests using Salmonella typhimurium exposed initially to different levels of Mg++, then passaged and examined by quantitative mass spectrometry. Problems identified with that project included the unlikely success of identifying changes using measurement of protein abundance in bulk cultures. A fourth project aimed to use protein engineering to design and synthesize thermodynamically stable cystine knot miniproteins that can serve as reagents such as enzyme inhibitors. This was an important but difficult project that likely will require more resources and time than the ILIR funding permits.

Important problems are identified for these funded basic science projects, but research is done in relative isolation from the greater scientific community. In addition, the short-term, part-time, small-budget nature of the research means that it may not achieve its potential reward for the scientific/military research community.

Materials Science Research

Many excellent projects were presented—some freshly launched and some just completed. A few projects were underperforming. It appeared that the best projects benefitted from good planning and execution of the research as well as from significant external input from others at ARL and DoD laboratories, universities, and national laboratories. Some of the projects that were struggling appeared to lack such input and consequently made less progress and had problems identifying and attaining key goals.

Some of the projects found deficient were led by less experienced researchers, often government employees working toward their doctorates on a part-time basis with a local university. Projects led by either very seasoned researchers (e.g., a project on modeling and quantifying nanoenhanced scattering in polymers) or by newly hired Ph.D. researchers (e.g., a project on exploring graphene’s potential for enhanced chemical reactivity induced by regions of local conformational deformation on rough iron surfaces) demonstrated strong interactions within and outside the RDEC, which contributed to the projects’ success and access to excellent equipment and facilities.

Mechanical Sciences Research

The understanding of rotary-wing aerodynamics and performance is essential to Army functions and very challenging in scientific terms. Unsteadiness, including but not limited to turbulence, interactions among blades, and blade deformations, presents important problems to be addressed. Generally, the research presented in this area was well done. There were indications of broad and detailed attention to these challenges with research on stall, effects of blade number and solidity, wing–vortex interactions, corotating and counterrotating coaxial rotors, and laminar-to-turbulent transitions.

Projects on a measurement facility for rotor blade deformation, tomographic particle image velocimetry (PIV), and computational strand-mesh generation were reviewed. The development of a rotor-hub-based camera system will help the understanding of blade twist and other deformations. PIV can resolve vertical structures in three-dimensional flows. The strand-mesh generation potentially is easier to generate automatically and has lower computer memory requirements. The capabilities that are sought here can have important applications.

Some novel contributions were made on three-dimensional fluid actuators, although some clarification is needed on their application as control mechanisms for vorticity. The residual compressive stress in gears and coupons is created by a shot-peening process and is aimed to improve material ability to resist cracks and fatigue; the relaxation of that stress due to fatigue is an important issue under study.

Network Sciences Research

The principal investigators working on projects that apply machine learning techniques to practical problems need to interact extensively with one another to share ideas, best practices, and research results. Machine learning needs to be regarded as a key technique to be studied throughout the ILIR program. There are possible synergies across a number of the projects, with respect to how detailed modeling of antennas and communication systems needs to be reflected in higher-layer network models.

In broad terms, the research in network sciences needs to be strongly supported for several reasons, including attracting and retaining top-quality technical personnel, serving as a crucial link with the larger research community, and providing the seed corn leading to important applied research that addresses the Army’s technical challenges. To ensure the quality of the research program, there is a need to apply the peer-review process as much as possible to ILIR projects. To advance greater scientific understanding

that will improve ILIR projects, there is a need to promote interactions between the ILIR researchers and the broader scientific community and to promote greater collaboration with other RDECs and Army laboratories. To enhance the quality of the ILIR program, there is a need to aim for a balanced mix of junior and senior personnel, design projects with a 2-3 year duration, and promote mentoring of junior principal investigators by RDEC managers and senior ILIR principal investigators. The researchers need to enhance their interactions with one another to take advantage of potential synergies among their ILIR projects.

Physics Research

Overall, the physics activities included a broad spectrum from basic physics, including interaction of electromagnetic radiation with microscopic and macroscopic materials, improved performance detectors, simulation of performance of optoelectronic devices, novel high-power lasers, unbiased testing of electro-optic infrared systems, atmospheric propagation of laser radiation, and continuum electro dynamics and mechanical resonances in braided textile parachute cords. The ILIR program is a potentially empowering mechanism for individuals to make long-term fundamental impacts in support of institutional responsibilities.

However, most of the research cannot be classified as basic research in the context of ARO or the outside world. The described activities represented solid research that can be used for support of system development. Much of the research focused narrowly on specific devices and materials related to ongoing applied concerns. The principal investigators and the managers overseeing the research projects need to look at the broader picture and make efforts to develop models and concepts that could be generalized and widely used by others. In addition, the current narrow focus is possibly a result of the fact that the project reviews and selections are done only at the RDEC level. A small number of ILIR programs do seem to build research knowledge at the basic level within its center, and this is commendable.

Several of the reported research activities involved partnerships with colleagues from local universities; they appeared to be chosen because of geographical convenience and familiarity. Extensions of collaborations to universities at the national level to better interact with the best research groups would provide a significant opportunity for improving the overall quality and impact of research. Every effort needs to be made to expand the collaboration horizons to reach out to experts in the field and act as a bridge to develop strong ties with the relevant basic research world outside the particular RDEC.

CROSSCUTTING RECOMMENDATIONS

Almost all of the ILIR projects involve basic research related to other work conducted at various RDECs and ARL. Enhanced interactions with researchers in these organizations would help accelerate progress on ILIR projects and provide important sources of relevant knowledge, equipment, and software to Army ILIR researchers.

Recommendation 1: The RDECs should explore ways to create greater interactions with one another and with the Army Research Laboratory to enhance their projects.

Conducting basic research activities in an otherwise applied research environment can be very challenging because of a different philosophy and methodology with which such activities are executed. Attendance at national scientific meetings is essential for researchers to experience and assess the state of the art in their discipline and expand scientific relationships. Building relations beyond the local

universities to engage researchers across the United States can bring in additional expertise, potential experimental validation, and theoretical modeling as well as cross-pollination of ideas. For example, many of the projects increasingly employ advanced computational science, including multiscale materials simulations, deep learning algorithms, and multi-agent optimization, for which external expertise would be impactful and when expertise developed via ILIR projects could help others. It appeared that many of the best ILIR projects created synergies with and benefitted from external scientific input, while some of the projects that were less successful appeared to lack such helpful inputs and consequently made less progress.

Recommendation 2: The RDECs should facilitate evaluation and dissemination of ILIR through enhanced engagement with the broader scientific community, including increased presentation at relevant conferences and publication in peer-reviewed journals.

It is important for the Army to have high-caliber researchers who can bring critical and up-to-date thinking to technical challenges that are unique to the Army. One characteristic of such researchers is a demonstrated understanding of the current state of the art, especially when starting a new project, evaluating the novelty of a research result, or documenting a project for publication. State of the art in this sense includes what information is available from the open literature, what relevant tools are available for use in research, and what referable data sets are available for comparative purposes. Many of the high-quality ILIR projects demonstrated such an understanding. Projects that have encountered difficulties often suffered from missing some crucial understanding that has been documented in the literature.

Recommendation 3: The RDECs should ensure that the principal investigators of ILIR projects demonstrate an understanding of the state of the art in the project’s field at the project’s inception and at appropriate project reviews.

Some metrics facilitate internal evaluation of Army-specific research projects, and others represent scientific progress in general; the latter are external, usually quantitative, measures of success acknowledged by the broader research community. Metrics can be qualitative or quantitative. Metrics may include such questions as the following:

• What are the objectives achieved by a project as compared with a measure of feasibility, which requires consideration of available resources and constraints imposed by strategic or mission goals?
• Do the reported results address the stated hypotheses?
• Did the research uncover significant or high-impact findings?
• Are the findings unexpected?
• Are the findings beyond the scope of the original hypotheses?
• Are the results breakthroughs or innovations?
• Are the results broad or niche-specific?
• Is the quality of the technical work consistent with the project’s stated objectives and research standards of practice?
• Is the number of refereed publications, conference presentations, and patents appropriate?
• Were unforeseen obstacles overcome?
• Did the project meet schedule and level-of-effort targets?

Such metrics influence management decisions necessary to facilitate efficient conduct of overall ILIR activity at the RDECs. Metrics reflect to a significant degree issues of efficient research management, including setting strategic scope, planning and securing future funding, training staff, and evaluating staff progress in reaching goals by rational critical paths within consistent evaluation criteria. There is sometimes a competition for limited resources that drives the directions of research to meet niche Army needs without sacrificing broader scientific justification.

Recommendation 4: The RDECs should consider developing and implementing technical metrics to gauge progress in their research that are consistent across the entire ILIR program. Given that ILIR projects are restricted to basic research, these metrics should include the boldness and quality of scientific research as well as the possible transition to practical applications.

The ILIR program is a beneficial activity that can contribute to workforce development by allowing Army research personnel to think creatively and to aim for transformative enhancements in the Army’s capabilities. Because basic research focuses on fundamentals, there is a need to engage the expertise and insight of other investigators in the field. Many presentations did not acknowledge the broader scientific picture or make efforts to develop concepts that could be generalized. Many RDECs do not avail themselves of external reviewers for project selection and objective evaluation of progress, and the current narrow focus may be a result of reviews and selection done only within the RDECs.

Recommendation 5: The RDECs should enhance the quality of ILIR by using a broader external peer-review process for selection, review, evaluation, and termination of their projects.

The committee judged the amount of ILIR funding to be rather small. It varied between $1 million and $2.2 million across the RDECs that reported ILIR funding (AMRDEC, CERDEC, and SMDC did not report ILIR funding levels).

Within the ILIR budgets at each RDEC, the amount of resources allocated to some projects is limited; there needs to be a focus on getting the most impact from these available resources. To establish a balanced portfolio, the RDECs may consider conducting fewer projects at higher levels of support for the chosen projects. It is noted that if the level of ILIR funding is maintained and the number of projects is reduced to achieve improvement in research quality, this may reduce the opportunity to recruit and retain scientific and engineering personnel, which is a goal of the ILIR program. It was not apparent that sufficient oversight was provided to ensure effective utilization and leverage of these limited resources to maximize the overall impact of each project.

In addition, although future plans were discussed, it was not made clear how the RDECs assure successful execution to completion given changes in the investigators’ team membership or availability of research infrastructure. Although the Science, Mathematics and Research for Transformation (SMART) Scholarship program is available for recruiting talent, it does not seem to be effectively leveraged and broadly communicated to the workforce. The RDECs are not fully utilizing this opportunity.

The committee acknowledges that it is challenging to develop project portfolios that reflect consideration of the appropriate number and nature of projects, the goal to recruit and retain talents, resource availability and allocation, and the organization’s mission. Addressing this challenge will require careful examination of the values and priorities that the Army places on these factors, as well as analysis of the pragmatic constraints imposed by limited resources.

Recommendation 6: The RDECs should ensure that ILIR projects are provided with adequate resources, expertise, and researchers’ time to successfully accomplish the objectives of the projects.

In some cases, it was not made clear whether there is a process for researchers to receive feedback and oversight on their work and their thinking processes throughout their projects. Periodic (e.g., quarterly) status reviews by RDEC managers—in addition to feedback at the end of a project—would be beneficial in increasing the chance of project success.

It was also not made clear whether junior researchers receive systematic mentoring by senior personnel throughout the course of a project. Junior personnel would benefit greatly from mentorship with respect to the selection, planning, and execution of ILIR projects, as well as the presentation of the project results and the planning for future research directions.

As an example, in modeling and simulation it is critical that the developed models adequately represent the real world. Senior researchers can help facilitate an understanding of the implementation of best practices in verification, validation, and uncertainty quantification. It is also worthwhile to explicitly endorse mentorship activities as a positive contribution to the senior researchers’ annual performance evaluation.

Recommendation 7: The RDECs should establish a process for and devote resources to providing effective project oversight and mentorship for junior ILIR researchers.