5

Computational Sciences

The Panel on Computational Sciences at the Army Research Laboratory (ARL) conducted its review of ARL’s advanced computing architectures, data-intensive sciences, and predictive sciences at Aberdeen, Maryland, on June 12-13, 2017. This chapter provides an evaluation of that work.

ADVANCED COMPUTING ARCHITECTURES

The advanced computing architectures group has evolved from operating and managing high-performance computing (HPC) systems to serving the processing needs of the broader Department of Defense (DoD) community to performing research focusing on tactical HPC at the edge. This places innovative new systems at the points of need to support the soldier in complex operational environments. The group is continuing to advance this evolution by responding to the recommendations of the 2015-2016 ARLTAB report¹ and ARL strategic priorities as defined by the ARL essential research areas (ERAs). The group emphasizes research on innovative systems that match size, weight, power, and time (SWAPT) and SWAPTN (SWAPT plus network) constraints to support workflows to enable “compute: decide faster!”

The research portfolio comprises projects for evaluating and advancing new and emerging architectures to enable artificial intelligence and machine language (AI and ML) computing with extremely low size, weight, and power (SWAP) and real-time, large-scale data analytics at energy-constrained points of need. There are also projects on models for quantum and classical networks for secure communications. Many early-career researchers are advancing these areas, publishing in high-quality venues and being recognized through best paper awards. ARL is complimented for trying to look forward a good 30 years

___________________

¹ National Academies of Sciences, Engineering, and Medicine, 2017, 2015-2016 Assessment of the Army Research Laboratory, The National Academies Press, Washington, D.C.

and thinking about the consequences of Moore’s law hitting a final plateau, and also for thinking about performance-portable programming models for uncertain future hardware.

Accomplishments and Advancements

The group has made significant progress in evaluating the role of neuromorphic computing using IBM’s TrueNorth processor and its leaky “integrate and fire” framework to enable high-fidelity computation using many low-precision elements, and thus very low energy. TrueNorth uses simulated leaky integrate and fire units to simulate neural computing. A key highlight is the demonstrated capability to perform symbolic processing on neuromorphic processors, providing computing capabilities equivalent to von Neumann processors at 1 percent of the power required.

This is a great example of successfully addressing the recommendations in the 2015-2016 Army Research Laboratory Technical Assessment Board (ARLTAB) report. Highlights include the fact that the project has successfully developed an end-to-end application that uses deep learning to identify information in an image, and then processing that information using a rules-based table look-up to determine optimal action. The team has also established a strong collaboration with the IBM TrueNorth team and has emerged as one of the leading players in the TrueNorth ecosystem with a respectable set of publications. The next step is to prototype a solution for an important Army problem as soon as possible.

Hybrid distribution shared memory application programming interfaces (APIs) and C++ templates were developed with the Adapteva Epiphany processor to demonstrate programmability with a two-dimensional (2D) array of compute nodes connected by a low-latency mesh network-on-chip. Army needs inexpensive, low-power, and distributed computing by leveraging open hardware and software. The team has also shown technical leadership by having ARL achieve voting status in the OpenSHMEM community. This is important because OpenSHMEM is an effort to create a specification for a standardized API for parallel programming in the Partitioned Global Address Space.

To serve the long-term Army needs for quantum secure communication and networking, measurable progress has been made toward understanding the potential range of uses of quantum computing and networking through modeling and simulation. A highlight is the work, in collaboration with experimentalists, to demonstrate microwave-to-optical frequency conversion to connect superconductive quantum logic circuits to optical communication.

Other notable results include the work on quantum walks to simulate physical systems. This is a good example of a theoretical project, with two publications in 2017 in the prestigious Quantum Information Processing journal. A complementary project is the one on quantum control for quantum-enhanced distributed sensing. The group has developed a practical tool (NIDE) to make it easier to design and evaluate networks that can realize quantum-enhanced sensing. This project is important for keeping the Army abreast of the latest relevant work on quantum networks. Moreover, the project’s methodology of evaluation through simulation is a nice example of leveraging the HPC expertise at ARL.

Challenges and Opportunities

In advanced computing architectures, the Computational Sciences Campaign has identified the goals of advancing neuromorphic computing, many-core, co-processor, and ASIC-integrated architectures for data analytics and tactical HPC delivered to points of need, while meeting strict SWAPTN constraints. If a leadership role is the objective, then these goals are extremely ambitious. The space of design choices is extremely large at the nexus of these types of new architectures, and their match to the data analytic, ML, AI, scientific, and numerical workloads of Army relevance is not clear.

It would be advisable to consider a strategic approach that is grounded on a few critical end-to-end processing workflows that have been determined as representative of the gaps that needs to be covered for the success of the ARL science and technology (S&T) campaigns. Requirements emanating through these end-to-end workflows could in turn provide a roadmap for the end-to-end system designs for research and development (R&D). A portfolio planning approach could then be used to determine and prioritize the projects to be advanced.

The projects presented as part of advanced computing architectures need to be positioned within a larger framework so that they are motivated by the driving application or scenario to provide a complete picture. This is the key challenge for this area. Within this broader context, there are many opportunities to strengthen current projects, including those described below. ARL could use the driving applications to determine the key kernels to be used for evaluating new architectures, such as the Epiphany processor, and to advance system development as opposed to using generic community kernels, such as those for linear algebra, Graph 500, and so on. The work on neuromorphic computing with the TrueNorth architecture is commendable, but there needs to be at least another processor to enable more generalizable findings. The project on control of quantum entanglement in cascade networks has credible results. However, the presentation combined two or three disparate ideas that were not clearly delineated, in that the linear and star topologies were for completely different applications. The Epiphany evaluation is promising, but input/output (I/O) and full system design considerations are essential for successfully accommodating large streaming data analytics workloads (which are not to be conflated with those for AI and ML).

It will also be a useful exercise to consider some of these driving applications in a future context. For example, this might include one in which a large central server trains an ML application, broadcasts a trained model to small units in the field, monitors their output, and iteratively retrains the model on the fly based on reported results. What would those small units look like? What would the server look like? How will they communicate? What are the appropriate data formats and representations? What are the appropriate provisions for tolerating various failures? Would ML training be the only function for that server, or are more conventional “scientific HPC” calculations also relevant in the field in this scenario?

Programmability

This work described a framework for organizing communication among parallel processes for large-scale HPC applications. ARL studied multiple approaches, including threaded message passing interface (MPI), OpenSHMEM, hybrid OpenSHMEM/OpenCL, and distributed shared memory (DSM). ARL has put the OpenSHMEM project on GitHub and is now a member of a standardization forum.

Templated Metaprogramming

This work described a software stack based on C++ templates for array-based programming. There are optimizers tuned for various Intel architectures. However, it was unclear whether this is entirely in the library templates, or whether some of it required digging into the C++ compiler. ARL researchers have considered extending this to graphics processing unit (GPU) targets, but agreed that that may be a more difficult problem because GPU architectures typically have multiple memories, not just a cached memory hierarchy, and it is not yet clear how best to divide responsibility for memory allocation between compiler and programmer.

Both the programmability and templated metaprogramming activities represent small projects that seem to be approaching maturity and deployment; indications were that they would extend only to 2017

or 2018, and that seems appropriate. They are “internal tooling” projects, well worth the doing, that will enable easier programming and improved portability for future software projects.

Opportunities for Computational Overmatch

This work considered hardware and software opportunities and challenges, attempting to look forward 30 years and considering the consequences of Moore’s law hitting a final plateau. It also considered performance-portable programming models for uncertain future hardware. The team highlighted the need for large numbers of inexpensive, low-power devices that will be deployed in a distributed fashion. This contrasts with designs being studied by the Naval Research Laboratory (NRL) and the Air Force Research Laboratory (AFRL), where the Navy and Air Force are more likely to deploy a smaller number of higher powered devices that perform centralized computation.

Portions of the work concentrated on the Epiphany architecture. Although it is lower power than some other contemporary alternatives, it is designed for the construction of larger, more centralized systems. Work on the Epiphany architecture and associated software concentrated on computation (floating-point [FP] operations and template meta-programming) and internal communication (mesh network, OpenSHMEM, and improving communication by exploiting writes in preference to reads), but it entirely omitted any discussion or analysis of input/output (I/O) capabilities and requirements (latency and throughput) at either the hardware or software level. Putting a “petaflop in a cubic meter” on the battlefield to perform centralized processing may well be worthwhile, but the work failed to put this in the proper context. ARL needs to sketch a more complete application scenario to explore such requirements, even as a back-of-the-envelope computation.

As an example, imagine deploying 10,000 sensors on the battlefield (maybe based on neuromorphic processors). Sketch some idea, however rough, of how these sensors will communicate their data to a central server, how that data will be fed into the server, what value will be added by applying petascale computing to this data, and how results will be delivered and for what purpose. This scenario assessment might lend some insight into whether a specific Epiphany-based design would have the necessary I/O capability and capacity, and whether the software stack provides the necessary support for that capability and capacity.

Part of the work also discussed the Army’s role as a fast follower of the Department of Energy (DOE) in high-performance computing. As global competition accelerates and the United States faces budgetary challenges, it is important to track global leaders. If not DOE, is it the Chinese? Is it Amazon or Google? In this context, ARL appropriately mentioned the Google tensor processing unit (TPU).

The specific Google TPU design may or may not be the best architecture for an Army application, but its designers paid very careful attention to overall application requirements and the context in which the TPU would be deployed. Examination of Google’s recent white paper about the version 1 TPU shows that Google paid as much attention, or more, to the motion of data into and out of the chip as to internal data motion and computation. In the end, Google designed a systolic matrix multiplier with appropriately sized memory buffers so that data can constantly stream into and out of the chip as computation is performed.

Another lesson from very recent developments in high-performance chip design: both Google (in its version 2 TPU, announced only in 2017) and NIVIDIA (in its most recent GPU design, with “tensor units” included) appear to have concluded that the sweet spot for machine-learning applications (or a least training) may be 16-bit FP numbers rather than 32-bit or 64-bit. This is very much in contrast to scientific applications. Indeed, the version 1 Google TPU supports 8-bit integer arithmetic, and it

may well be that moving to 16-bit FP for version 2 is actually a slight compromise on performance in exchange for portability and ease of programming.

A plausible conjecture is that 5 years from now, the machine-learning community will have “standardized” on a computational structure of 16-bit FP activation values, 8- or 16-bit weights (or possibly 16-bit FP weights stored using some compression scheme and expanded on the fly), multiply-add operations that multiply 16-bit FP values into a 32-bit FP accumulator, and a selection of nonlinear operations that take 32-bit accumulations back into new 16-bit activation values.

Quantum Technologies

This work touched on “unique properties” of quantum physics—namely, coherence and entanglement—but failed to give any real sense of how those unique properties enable the proposed applications relative to classical approaches. How do these unique properties make quantum networking better than classical networking? How do they contribute to secure communications and sensing?

DATA-INTENSIVE SCIENCES

The data-intensive sciences program presented its goals and progress in applied ML, neuromorphic computing, and cooperative multiagent control using deep reinforcement learning. The data-intensive sciences overview and the two presentations on neuromorphic processing and cooperative reinforcement learning were excellent. The data-intensive team has made a strong start in the new research thrust in machine learning. ARL has not only hired new talent but also leveraged its existing researchers. This is important because the ARL researchers bring additional scientific and application expertise that will distinguish ARL’s research in machine learning.

Accomplishments and Advancements

Following up on recommendations from the 2015-2016 ARLTAB report, ARL has established an advanced computing laboratory. It includes IBM’s TrueNorth brain-inspired computing platform, the Adapteva Epiphany Network-on-Chip (NoC), and other emerging architectures. TrueNorth uses simulated leaky integrate and fire units to simulate neural computing, and is considered exciting for two reasons. First, the brain uses neurons that “fire” action potentials or “spikes.” Thus, TrueNorth can be seen as coming closer to real brain computations than many other computing platforms. Second, TrueNorth uses power only when “firing” events occur, making it a platform that uses very little power when it computes.

Exciting progress has occurred using the TrueNorth platform in two different ways. One approach has been to write computationally transparent computer algorithms that exploit the TrueNorth platform (a so-called top-down approach). This approach has been used to show how well-understood computational problems can be solved in novel ways using timing differences between the occurrences of spikes, including an award-winning paper that implemented a sorting algorithm that sorts lists in linear time on a neuromorphic computing platform. An award-winning integer factorization paper was also published that detailed the development and implementation or the algorithm on a neuromorphic computing platform. The second approach has port-trained neural networks to the TrueNorth platform, allowing demonstrations that previously learned computations can be carried out on a platform that consumes very little power. The laboratory’s efforts in neuromorphic computing have blossomed since the panel’s last visit into a significant and highly collaborative research effort.

In the Army High-Performance Computing Research Center, exciting progress has also been made using deep reinforcement learning methods for multiagent control. Stanford University researchers, funded through a cooperative agreement with ARL, have successfully extended the multiagent setting methods previously used to train a single agent using cooperative reinforcement learning to achieve state-of-the-art performance in several applications. The illustrated applications indicate the power of deep reinforcement learning to address mission-relevant tasks dependent on multiple artificial agents.

ARL plans to port an instance of one of these applications to TrueNorth. It promises to provide a low-power realization of a trained model. This is significant, promising work in a new problem area in machine learning. This could serve as a model for ARL’s internal research. The challenge is to build on this work within ARL and exploit it.

Several projects are worthy of particular mention, as discussed in the following. The keystroke biometrics project has made an impressive showing, winning multiple awards in an international machine learning competition establishing ARL’s visibility in this research area. Similarly, the impact of machine learning on the Army project has made a good step in articulating Army-relevant machine learning challenges. The machine learning for planetary gearbox analysis work demonstrated a proactive approach to preventative maintenance by automated generation of features. This approach could provide an AI-based preventative maintenance technology—evidently a tremendous advance for Army operations. ARL could benefit by exploiting theoretically based parameterized mathematical and statistical representation methods for automated feature generation. The large-scale data analytics project, which was presented 2 years ago, has now entered the transition phase and is rebasing the software using current commercial off-the-shelf (COTS) technology. Each of the three data-intensive computing projects (high-throughput electrolyte modeling, discovering and quantifying atomistic defects in large data sets for assessing nanocrystalline aluminum, and computational technologies for the reduction of highly nonlinear and multiscale solid mechanics and structural dynamics models) are exploiting the same ARL-developed software architecture for distributed simulation. Each of them represents different machine learning challenges in detail. They all demonstrate successful machine-learned approximations applied to simulation pioneered by ARL. Last, several other predictive simulation projects (Distributed Multiscale Computation for Modeling Energetics, Multiscale Transport Modeling of Optical Response in GaN, Development and Application of Multiscale and Multiresolution Models to Build Better Fuel Cell Membrane Electrolytes, and Modeling and Design of Macromolecules at Multiple Scales) demonstrated this same approach. ARL could consider writing a book on its approach.

Challenges and Opportunities

ARL has an outstanding opportunity to continue making progress exploiting and extending recent developments in machine learning. Two opportunities identified in the 2015-2016 review remain exciting avenues for further development. The first is use of ARL’s expertise and resources in high-performance computing as a home for large mission-relevant data sets. The second is exploiting the potential of neuromorphic computing platforms to allow the advances in deep learning to be implemented effectively in light, low-power devices for deployment in the field.

The first of these opportunities remains to be more fully exploited; this agenda lies at the heart of the data-intensive science mission and needs to be more vigorously pursued. Application of machine learning to Army problems is target rich and just beginning. The challenge ARL faces is how to expand the initial efforts to applications beyond the excellent application speeding simulation with learned models.

As noted earlier, ARL has stepped forward in the area of neuromorphic computing, and the efforts to date are applauded. That said, it is important to be aware that neuromorphic computing is a diverse

field of engineering whose potential is only beginning to be explored. Relying on the leaky integrate and fire model of the neuron is one important approach, but far from the only one, and it is not clear that it is ultimately advantageous from a computational point of view to make use of spiking neurons.

Today’s deep learning methods rely on the use of continuous functions to allow very deep continuous computational paths to be subject to gradient descent—their power depends on their differentiability, whereas a “spiking” event is discrete and not differentiable. It is an important basic science question to understand how the human brain succeeds in learning with spiking neurons, but from an AI/ML/robotics point of view, solving this problem, or relying on any particular characteristic of real neurons, may or may not turn out to be helpful. For these reasons, alternatives to the existing TrueNorth integrate and fire platform needs to be pursued.

Neuromorphic computing appears to be one of the best approaches to low-power computing for sensory data. The challenge is for the research not to be tied too closely to a particular neuromorphic architecture (e.g., the leaky integrate and fire model for neurons). ARL needs to explore other low-power realizations of neuromorphic computing.

Another opportunity exists for dramatically improving predictive modeling capabilities by augmenting traditional scientific simulation paradigms with machine learning. Such a physics-guided machine learning paradigm has the potential to bring the power of state-of-the-art machine learning approaches to predictive modeling while leveraging the wealth of domain knowledge that is critically needed for solving such problems. This line of research is beginning to be followed in a number of projects (under predictive modeling) and presents a major opportunity for the ARL to take leadership in this area that will be of great relevance to the computational science community at large.

Given the mission-critical nature of ARL applications, it is important to understand the limitations of the machine learning paradigm. For example, typical machine learning methods do not produce a statistical representation of the sample data that would enable uncertainty quantification, sample generation for validation, and statistical fusion of different samples.

Again, a previous recommendation that ARL make a significant effort to become the data archiving center for the Army is reiterated. Because the Army generates large volumes of data that are currently isolated in local computer systems, the value cannot be extracted. A central repository is fundamental to data-intensive computing. Such a repository will open research avenues and form the basis for cooperation with academia. For example, the data sets from large-scale Army test and evaluation exercises represent a unique resource and opportunity.

The panel recognizes the difficulty of hiring top-notch talent in machine learning. As a mitigating strategy, ARL needs to place a heavier emphasis than usual on university collaboration with machine learning experts and their students. ARL has a very rich and challenging set of real-world problems that provide exciting challenges to machine learning researchers within ARL and elsewhere. To achieve this, ARL needs to develop a specific collaborative process whereby university collaborators actually interact with and transfer expertise to ARL personnel. ARL has a very rich problem space to drive its research—but to execute it ARL needs top-notch researchers and collaborators and ARL needs to invest in its own people and by promoting collaborations.

ARL could design and sponsor a machine learning competition based on Army-collected data under deception scenarios. This introduces a new problem that will shape future research in machine learning and computer science. Also, ARL could, along with its counterpart laboratories, advocate creating a Defense Advanced Research Projects Agency (DARPA) program in the areas of robust methods in AI/ML and submilliwatt software and hardware computing architectures.

PREDICTIVE SCIENCES

Predictive science has long served as a foundational component of the R&D practices of the ARL. From the early computational predictions of ballistic trajectories during World War II to current work on advanced armor to ensure soldier protection, predictive computational techniques have been an integral part of ARL’s execution of its national-security mission.

Current predictive science work at ARL focuses on developing the various “multi-” capabilities that are required for accurate computational analysis, which include multiscale, multidisciplinary, and multifidelity analysis. In each of these regimes, ARL researchers have a stated goal of including relevant verification and validation (V&V) and uncertainty quantification (UQ) capabilities in their computational analyses. A related R&D goal is to leverage results of prior computations toward developing various surrogate reduced-order models that retain the accuracy characteristics of the original models while requiring significantly lower computational time. Memory space, power consumption, and weight/physical size are also other important metrics.

Under the current crosscutting schema of research campaigns and ERAs, two specific ERAs underlie the key predictive science efforts presented at the review. The first ERA is based on application of ML and AI and has shown substantial improvements since the 2015 ARLTAB review. However, overconfidence in this arena is cautioned, as the 2015 review recognized a dearth of relevant R&D in this field. Nonetheless, the improvements shown in this ERA (discussed in more detail in the “Data-Intensive Sciences” section of this chapter) are laudable and demonstrate that in predictive sciences there is a path to effective integration of simulation-based techniques with ML approaches. This leads to a hybrid modeling methodology whose fast-running models (FRMs) can be used by soldiers in the field to obtain results with sufficient accuracy.

The second ERA is motivated by a desire to better understand the physics of failure at a range of scales, including the material, mechanical, and structural regimes. This ERA builds on long-standing R&D programs at ARL for assessing proposed soldier-protection technologies ranging from assessment of personal protection materials to design of advanced armor for vehicles.

Accomplishments and Advancements

There is a general improvement in the quality of predictive science R&D from 2015, and in particular, a decreased variance in quality across the range of efforts presented. The notion of “relevance” was apparent in much of the R&D presented, ranging from “relevant UQ” to “multiscale analysis relevant to ARL’s mission.” This improved focus on mission relevance is laudable, so future panels can evaluate R&D activities through this mission-focused frame of reference. Another area of improvement is in recruitment and retraining of staff into roles with greater mission relevance.

Some specific examples of R&D results presented warrant extra attention, as they demonstrate an appropriate focus on ARL’s mission sensibilities. The research on dynamic surrogate model evaluation in a computational framework for scale bridging with application to multiscale modeling of RDX explosive exhibited many noteworthy aspects of predictive computational science, including a scalable software foundation utilizing the hierarchical multiscale (HMS) technology demonstrated during the 2015 ARLTAB review. The R&D work on improving interscale data communication, achieved by relieving a bottleneck that would limit performance, provided impressive optimization results that demonstrate ARL R&D leadership in advanced computing. This R&D effort also demonstrated the use of surrogate models to improve performance, so a true cross-campaign activity resulted that demonstrated the “multidisciplinary” aspect of ARL’s predictive science roadmap.

The vortex filaments work is a small-scale, exploratory research effort that included several noteworthy features, including an academic collaboration and a focus on a spatial scale of especial relevance to Army soldiers in battlefield conditions. Where much of atmospheric science R&D has concentrated on the mesoscale (i.e., severe storms such as tornados), microscale effects often dominate the practical utility of troop activities (e.g., deploying a drone in a hilly landscape where turbulent vortices are produced by the interaction of wind and terrain). This project fills an R&D gap that is relevant to ARL’s mission via a low-cost, high-impact-potential activity.

The integrated computational materials engineering for polycrystalline materials research effort was an excellent demonstration of the practical value of multiscale analysis as applied to the design of new materials, and it included several noteworthy goals. In particular, the project execution plan provided key data science outcomes, including a curated data repository for both experimental and computational results, and has initiated collaboration with the Georgia Tech faculty group working at the interface of material science and data science. This project’s goals were mentioned as a means to effect the cultural change required to bridge the intellectual gap between experimental and computational subject-matter experts, which has been a long-standing impediment to broad adoption of data science methods.

There is continuing improvement of peer-reviewed publication record, with over 30 journal publications. ARL needs to maintain this trend and work to increase the ratio of publications to staff.

Challenges and Opportunities

Two specific cultural challenges were apparent during the review of the predictive science R&D portfolio, along with one administrative challenge mentioned by several of the project personnel. The first challenge is that the cultural change required for broad acceptance of predictive modeling and simulation may not yet be in place, and it is not yet clear whether there has been a widespread adoption of the V&V and UQ technologies required for computational analyses to be demonstrated as truly “predictive.” Current ARL computational analysis practice does not yet promote standardized UQ processes, and V&V activities were notable not by demonstration, but by more commonly being omitted in discussions of R&D project activities.

This lack of acceptance of emerging methods for demonstrating the predictive effectiveness of computational simulation was specifically called out in the 2015-2016 ARLTAB review. UQ and V&V processes need to become ubiquitous in future computational analyses at ARL.

A second cultural challenge is apparent at the interface of the two ERAs that help define ARL’s Computational Sciences Campaign research foci. While there is considerable potential for effective R&D that fuses predictive science results from ARL supercomputers with ML-based FRMs that would aid soldiers in the field, not much evidence of this cross-ERA fusion was apparent from the presentations and posters. This is a research venue that holds considerable relevance for ARL’s mission. The low-power devices proposed for use in the battlefield will not be capable of performing high-fidelity predictive computational simulations, so simpler models will be needed, and these models can be developed at the interface of these two foundational ERAs.

The third challenge is an administrative one, and a common feature of federal R&D administration. Successful exploratory (6.1) research efforts require a clear path for transition to practice so that exploratory efforts that are not successful (and by definition, exploratory research portfolios need to include high-risk, high-return efforts that may not succeed) can be pruned from the research funding base. Similarly, projects that demonstrate utility for ARL’s missions need to have a well-defined path to 6.2/6.3 reduction-to-practice funding so that they might ultimately lead to successful deployment under

operationally realistic circumstances. This challenge lies less with researchers and more with management at ARL, but overcoming the challenge requires clear lines of communications from both parties.

Note that each of these challenges represents only one side of the larger challenge, with the other representing a concomitant opportunity for improved R&D project relevance. These opportunities could be pursued by ARL. ARL could extend new work in developing and deploying data repositories. In particular, ARL is well positioned to serve as the Army’s data hub. One way to achieve this long-term goal is to support and extend smaller scale efforts at creating curated community data archives of relevance to ARL’s mission. Some of the R&D effort presented utilized prototype data repositories that could be extended to support a transition path to the more ambitious goal of ARL providing the Army’s data hub. ARL could reform a comprehensive gap analysis of ARL predictive science efforts, toward the goal of identifying areas where collaborations are required to achieve successful R&D outcomes. This analysis could support the longer term activity of performing research outreach efforts to academia, industry, and other federal R&D institutions so that research outcomes are not compromised by omission of key components needed for successful deployment. ARL could provide more technical detail on the interfaces between predictive science and other R&D venues that support, or are supported by, this foundational research area. For one example, the interface between the ML/AI ERA and the physics of failure ERA could be better articulated (if it is already well defined) or better developed (if it is not).

OVERALL QUALITY OF THE WORK

As with any broad-based research and development effort, there is tension between the depth (focus) and the breadth (coverage) of exploration, convolved with a balance between basic research and outcome driven activities. Overall, ARL has made substantial progress in addressing its mission objectives since the last review.

Many early-career researchers are exploring advanced computing architectures, publishing in high-quality venues and being recognized through best paper awards. Kudos for trying to look forward a good 30 years and thinking about the consequences of Moore’s law hitting a final plateau, and also for thinking about performance-portable programming models for uncertain future hardware.

The data-intensive sciences overview and the two presentations on neuromorphic processing and cooperative reinforcement learning were excellent. The overall technical quality of the work at the early stages of the research efforts represents a solid first step toward significant research contributions. The data-intensive team has made a strong start in the new research thrust in machine learning. ARL has not only hired new talent but also leveraged its existing researchers. This is important because the ARL researchers bring additional scientific and application expertise that will distinguish ARL’s research in machine learning.

The predictive sciences represent a good combination of machine learning within large simulations to optimize multiscale model computations with the HMS work. This is a promising and important direction that would be important to continue into the future. It is encouraging to see the fruits of ARL’s concerted effort in the past several years to develop a general-purpose framework for HMS, and to see that the research is transitioning toward incorporating into HMS the lessons learned from applying this technique to solve real problems, including HPC optimizations and the use of ML methods. A note of caution is worth mentioning regarding the “past several years” HMS characterization earlier. While the integration of the HMS work into new intellectual venues is commendable, the panel noted that these R&D improvements are incremental additions to the HMS technology, and that new starts for exploratory research are warranted to provide similar support for future ARL R&D.

CONCLUSIONS AND RECOMMENDATIONS

The ARL team gains practical and intellectual advantage via thoughtful collaboration. ARL needs to focus on identifying collaborators and points of leverage. ARL described a tripartite “lead, collaborate, or follow model” for assessing research and development opportunities. ARL needs to create and share white papers on the state of the art and to identify national thought leaders whose research, insights, and ideas can inform and guide research and development.

Conceptual diagrams of future battlefields can inform engineering implementation by mapping activities onto the most critical problems and needs. ARL needs to convert these conceptual views to tangible work plans, emphasizing those focused activities likely to maximize return on investment.

Last, ARL needs to think carefully about metrics for project success and define associated project exit strategies, including transition out of this organization to achieve Army impact. In particular, the panel emphasizes the need for transition funding and engagement for technology application and uptake.

Because the Army generates large volumes of data that are currently isolated in local computer systems, the value cannot be extracted. A central repository is fundamental to data-intensive computing. Such a repository will open research avenues and form the basis for cooperation with academia. For example, the data sets from large-scale Army test and evaluation exercises represent a unique resource and opportunity.

The panel recognizes the difficulty of hiring top-notch talent in machine learning. ARL has a very rich and challenging set of real-world problems that provide exciting challenges to machine learning researchers within ARL and elsewhere.