2

Computational and Information Sciences Directorate and Network Science Enterprise

INTRODUCTION

This report is based on meetings of the Panel on Digitization and Communications Science held at ARL on June 28-30, 2011, and June 6-8, 2012. The 2012 meeting was expanded to include a review of the network science portfolio of activities on which the Computational and Information Sciences Directorate (CISD), Human Research and Engineering Directorate (HRED), and the Sensors and Electron Devices Directorate (SEDD) collaborate. The panel reviewing the network science activities included members selected from the panels that also reviewed SEDD and HRED as well as members of the panel that reviewed CISD.

This chapter includes the report on the Network Science Enterprise. Two divisions in CISD, Information Sciences and Network Sciences, have substantial involvement in the Network Science Enterprise, and the reports on those two divisions are subsumed under that for the Network Science Enterprise. Separate sections are devoted to the Battlefield Environment Division and to the Computational Sciences Division, the remaining two divisions of CISD, which have considerably less involvement in the Network Science Enterprise. Discussions and recommendations on opportunities and challenges, and on the overall technical quality of the work, are presented in the respective sections, including the network science section. The chapter ends with common themes of opportunities and challenges, and observations on the overall quality of the work that cut across the Network Science Enterprise and CISD.

Two major extramural collaborations involve both the Network Science Enterprise and CISD: the International Technology Alliance (ITA), which is an alliance of the United Kingdom (UK) Ministry of Defence, universities, and industry in the UK and United States; and the Network Science Collaborative Technology Alliance (CTA), which has participation from CISD, HRED, and SEDD in ARL as well as a number of universities and industry partners. The ITA and the CTA are representative of “enterprise” collaborations within ARL—a concept that is, quite appropriately and laudably, receiving increasing



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 18
2 Computational and Information Sciences Directorate and Network Science Enterprise INTRODUCTION This report is based on meetings of the Panel on Digitization and Communications Science held at ARL on June 28-30, 2011, and June 6-8, 2012. The 2012 meeting was expanded to include a review of the network science portfolio of activities on which the Computational and Information Sciences Direc- torate (CISD), Human Research and Engineering Directorate (HRED), and the Sensors and Electron Devices Directorate (SEDD) collaborate. The panel reviewing the network science activities included members selected from the panels that also reviewed SEDD and HRED as well as members of the panel that reviewed CISD. This chapter includes the report on the Network Science Enterprise. Two divisions in CISD, Informa- tion Sciences and Network Sciences, have substantial involvement in the Network Science Enterprise, and the reports on those two divisions are subsumed under that for the Network Science Enterprise. Separate sections are devoted to the Battlefield Environment Division and to the Computational Sci- ences Division, the remaining two divisions of CISD, which have considerably less involvement in the Network Science Enterprise. Discussions and recommendations on opportunities and challenges, and on the overall technical quality of the work, are presented in the respective sections, including the network science section. The chapter ends with common themes of opportunities and challenges, and observations on the overall quality of the work that cut across the Network Science Enterprise and CISD. Two major extramural collaborations involve both the Network Science Enterprise and CISD: the International Technology Alliance (ITA), which is an alliance of the United Kingdom (UK) Ministry of Defence, universities, and industry in the UK and United States; and the Network Science Collaborative Technology Alliance (CTA), which has participation from CISD, HRED, and SEDD in ARL as well as a number of universities and industry partners. The ITA and the CTA are representative of “enterprise” collaborations within ARL—a concept that is, quite appropriately and laudably, receiving increasing 18

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 19 attention and momentum within ARL. The ITA was initiated in 2008 and the CTA in 2010; this CTA follows a previous CTA, which also had a network focus. COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE Changes Since the Previous Review During this period, a new director took the helm of CISD, and a new chief took the helm of the Computational Sciences Division. These leadership changes have benefited the respective organizations by injecting continuity and stability and have led to sharper focus of the research vision and activities. Particularly noteworthy in CISD is the strength of the aspiration to increase activities across disciplines, divisions, and other organizations. Also, in the Computational Sciences Division (CSD) there have been definite improvements in direction and focus that may be reasonably credited to these management changes. ARL should take note of the potentially destabilizing and morale-lowering effects of overly rapid turnover of management and of having acting appointees head organizations for extended periods of time. Minimizing such disruptions will greatly benefit ARL. Battlefield Environment Division Introduction The Battlefield Environment Division (BED) presented a compelling picture of the need for basic research in atmospheric science to meet Army needs, which range from aiding special operations to improving the accuracy of conventional artillery. The Army has a unique and pressing requirement for near-Earth atmospheric understanding and characterization beyond what can be provided by other military and civilian entities. The presence of 24 Ph.D.’s out of a total of 56 civilian divisional staff and the ongoing effort to increase staff expertise through the involvement of postdoctoral personnel are impressive developments. The division’s support appears to be a reasonably good mix of 6.1, 6.2, and customer funding, with the first two constituting 55 percent and the last 45 percent. Seventeen percent of customer funding is from the Defense Advanced Research Projects Agency (DARPA). Total program support dropped by approximately 10 percent because of cutbacks in the area of atmospheric sciences applications, which is not surprising given the advances that have been accomplished in previous years. Collaborations in the Mountain Terrain Atmospheric Modeling and Observations (MATERHORN) Program and Partners In Research Transition (PIRT), a joint program with various organizations and universities, illustrate a vigorous extramural activity. There were a number of personnel initiatives dur- ing the past year, including establishing a relationship with a professor at Colorado State University and rotating a BED staff member to the Army Research Office to help in the atmospheric science area. BED’s research focuses on three major areas: atmospheric sensing, atmospheric dynamics, and atmo- spheric modeling applications. Underlying these activities is fundamental science, with the emphasis currently on four areas: near-field phase locking of array laser, Raman spectra of individual particles, turbulence propagation, and atmospheric impact routing. A significant portion of the division’s resources are directed at products that it will provide to the Army. Management has a technical roadmap for the division that extends to 2017 and that deals with major programs currently in-house and others that are anticipated. Overall, the division’s management provided compelling evidence of a grasp of the chal- lenges, especially the balancing of diverse requirements with limited and shrinking resources, and the planning and vision to address the challenges.

OCR for page 18
20 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY Accomplishments and Advancements Raman Spectra of Individual Airborne Particles  Several optics projects in BED are aimed at single- particle detection for biohazard threat applications. BED has made significant contributions in the area of laser-induced fluorescence (LIF) detection methods that have been developed and now deployed, but these LIF methods do not provide adequate differentiation to reliably identify specific pathogens. Defini- tive characterization is more straightforward for chemical detection using mass spectrometry techniques, for example, but pathogen-specific biosensors in atmospheric sensing require techniques such as Raman spectroscopy or infrared (IR) absorption/emission spectroscopy. Both of these techniques suffer from signal-to-noise constraints at the single-particle level, and BED is engaged in basic early-stage research aimed at addressing this challenge. For Raman spectroscopy, current work provides a novel method for trapping individual particles using a new cylindrical (Bessel) beam-focusing geometry that provides a hollow optical ellipsoidal intensity chamber containing particles through photophoretic forces. This pro- vides sufficient residence time (approximately 1 second) yielding the signal-to-noise ratio required for reliable pathogen identification. This technique has already demonstrated the ability to trap a significant range of particle sizes using readily available milliwatt-level beam powers along with practical optics, and it has also demonstrated well-resolved Raman signatures of trapped particles using the same trapping beam. This technique also has the flexibility of allowing for additional specific Raman probe beams. The work is in the very early stages, but it has demonstrated that in airborne particle applications the photophoretic forces are much stronger than the usual optical tweezer radiation pressure techniques that are used routinely in aqueous environments. There is opportunity for obtaining a wealth of data and for data analysis, combined with theoretical modeling, to fully map out the efficacy of the approach. Once this is known, the group has competence in the hardware implementations necessary to demonstrate a combined system that uses preliminary scattering and LIF-based particle sorting, followed by the Raman technique for specific pathogen identification. Turbulence Propagation Theory and Effects  This study has matured very nicely. The principal investigators have published results and have used reviewer feedback to develop a more complete theory for the modulation transfer function (MTF). The present study has two objectives: (1) quantification of optical turbulence effects in passive and active imagers and (2) development of prototype methods for mitigating turbulence effects on imagers. With regard to the first objective, the research team should proceed with publication of the new (more complete) theoretical findings. With regard to the second objective, the researchers have begun to develop concepts for operationalizing the theoretical results. The research team should develop a prototype system to demonstrate that the theory correctly describes what happens in practice, and it should use the results to design and implement a full demonstration program to present to a funding agency within the Army. The most recent research is unique and funda- mental; unlike the adaptive optics approach also developed within BED, this approach does not require a cooperative target to enhance the target image. Climatological Assessment Utilizing Airborne Acoustic Sensors  The problem addressed in this activity is locating a source of acoustic emission on Earth’s surface using detectors on an elevated/ airborne platform. Once the location of the emission is identified, a camera can be directed toward the source for full identification. The path of the propagating acoustic signal has to be corrected for refraction or bending resulting from temperature variations in the atmospheric density and shifts due to horizontal wind. The researchers lucidly defined the problems, and they tested the corrections for a case of heated ground that has a monotonically changing temperature profile concave with respect to Earth’s surface.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 21 This study involves one particular case that may be representative of some situations occurring in the Afghan theater but not to all possible situations, such as those with very stable conditions or an inversion between the Aerostat and the ground, which could happen, especially when the Aerostat is positioned at 1,600 m (cases were presented for 200, 800, and 160 m) altitude. The approach developed a climatology (based on 155 days of data) of near-surface wind and temperature profiles and extracted the needed corrections using the corrections based on published theoretical calculations. The method was tested using data from two field sites, Aberdeen and Yuma Proving Grounds. The data revealed a smooth temperature curve with height but did not show any inversion, a result that might make the method invalid for some situations and may suggest a need for further testing. The developments appear reasonable, perhaps because the requirement for a camera to aim in approximately the correct direction is not very severe, so that even though these early tests show considerable scatter in the corrective terms, the results point in the right direction. Tests using forecast profiles are planned, which is an important step to establishing and applying the method operationally. Attention should still be given to situations with a stable air-layer near the ground and a possible inversion between the Aerostat and the ground. Further studies on the adequacy of climatologically stored data or local forecasts for estimating the corrections of the localization angle and distance are needed and should answer the question regarding the method’s applicability in the field. Weather Research and Forecast Model-Based Nowcasting for Battlefield Operations  BED’s weather research and forecast (WRF)-based numerical modeling is designed to fill an important gap that exists at very fine scales, in order to meet Army needs for timely environmental data on the rapidly evolving battlefield. This need is not currently met well by standard, available weather forecast products designed primarily for larger spatial scales and longer temporal scales. BED’s numerical “nowcasting” effort is designed to provide frequent updates of environmental products on grids of 1 kilometer or less. To be successful, this work has to assimilate, in addition to traditional sources of weather data, a variety of atmospheric observations collected by disparate sensors on the battlefield, and it has to be able to identify and reject unreliable data. BED researchers are investigating ways to exploit satellite data that could provide models with soil-moisture measurements, which would be valuable for improving the accuracy of land-surface flux calculations and atmospheric boundary layer conditions. Traditionally, these high-resolution modeling tasks have required major high-performance computing platforms, but BED is examining means of achieving forward-deployed computing capacity. Currently, the means to accurately represent the effects of turbulence in the atmospheric bound- ary layer at the scales relevant for high-resolution nowcasting is not well understood and is poorly represented by available modeling software. BED has made progress in recognizing this problem and in developing and/or acquiring the necessary expertise to address this issue. A hierarchy of models is under development and is being tested for application in various battlefield conditions. This logical path allows for steady improvement of atmospheric guidance as advances are made, the science matures, and computational capacity increases. BED has made significant strides in addressing the issues unique to weather model verification at the battlefield scale by acquiring and evaluating advanced weather model evaluation software. This effort is already stimulating new ways to extract and evaluate the accuracy of Army-relevant information from the guidance provided by the weather model. BED’s modeling work supports a wide range of products that assist the soldier, including optimal routing algorithms that address impacts of atmospheric condi- tions, improved accuracy of ballistic artillery, and soldier health effects such as heat stress and exposure to airborne toxic aerosols.

OCR for page 18
22 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY Although the numerical modeling effort is making substantial progress toward solving several important problems, it faces a few significant hurdles. WRF applications at sub-kilometer grid scales, especially in unstable conditions, are problematic, because of limitations to the available turbulence physics parameterizations. Data assimilation at very fine scales is handicapped by the paucity of obser- vations at comparable scales. BED is encouraged to pursue this legitimate but challenging research area. Many of the modeling initiatives undertaken during the past year have involved developing new techniques as well as importing of new expertise. It is not surprising, then, that BED has not yet had sufficient opportunity to perform adequate model verification, which is beginning to emerge from these recent developments, and which BED intends to continue. Atmospheric Impacts Routing Tool  This project is a continuation of an effort that has been presented in two prior reviews. This tool has matured and is ready to move beyond the prototype stage into opera- tions. It applies the “A*” algorithm (a computer algorithm that is used in path finding and graph traversal) to the problem of “optimal” air and ground transportation routing. Ground routing involves multiple impact factors, such as Met forecasts, nowcast information, vehicle types, soil conditions, road states, and other factors. The algorithm “optimizes” paths in three-dimensional (3D) or four-dimensional (4D) space after building an architecture to support the ingestion of dynamic 3D data sets. It then makes an informed search for the “best” route, applying a set of criteria for what is best. It can adjust for desired goals, such as wanting higher than normal safety or faster speed to the goal point. Areas of avoidance can be declared as “no-go”/”no-fly” areas. The project has been extended to respond not only to weather conditions but also to other threats or obstacles that could impede or preclude use of a particular route. The technology appears to run very fast, calculating optimized (for example, minimal time) routes while avoiding adverse conditions. It avoids many of the problems associated with pre-defined networks and instead finds a solution for an optimal path through a 3D grid, subject to the constraints imposed by the user or the different threats; it was originally based upon the A* algorithm but has been expanded to incorporate the D* and E* algorithms. Of concern is that only a few methods can verify that the routing system is performing correctly, that is, generating an optimal route given a particular 3D arrangement of threats. One possible solu- tion involves configuring the system to generate all possible routes through a given 3D grid, and then determining how the ensemble varies statistically from the optimal one. Although the results of the A* algorithm are very impressive, the development of a method for testing optimal routing techniques might represent an even more significant improvement. Also, this research has the potential to benefit from cross-disciplinary expertise, especially in optimization and computational methods for large-scale optimization, which is likely to exist in other divisions, such as CSD. Additionally, it would be beneficial if the research considered the value of clearly defined “near-best” and “near-optimal” solutions. Optical Systems  A project in early-stage development involves coherent, phased-array beam combin- ing using single-seed laser sources and arrays of fiber amplifiers with antenna elements controlled both by individual phase modulators and piezo-actuated fiber positioners. This Intelligent Optics Laboratory (IOL) system has already demonstrated record power-combining at the multi-kilowatt level, including active servo control of the transmitter for some degree of atmospheric turbulence correction using ele- ment phase and beam tilt adjustment, with elaborate optimization of servo control to counter transit-time- induced loop delays. The team’s several hardware approaches demonstrated scalability, both in element power and number of elements, which has the potential to take the solution to weapons-level powers. The significant customer pull here (DARPA) provides strong evidence of the leading-edge nature of this work.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 23 Adaptive Optics  The IOL team also was involved in imaging through boundary layer turbulence using locally engineered adaptive optics hardware that does not rely on wavefront sensing and has potential to constitute a low-cost solution. Using a novel, laterally actuated, low-piezo element count, 1-inch diameter adaptive optics, the IOL team has demonstrated performance that is close to that of much more elaborate, and very expensive, commercial solutions. The team also has explored the feedback servo problem, including removing loop delay. Work in this area is highly promising, and further exploration will yield valuable assessment of different feedback signals, such as image spatial frequency content, that provide best image fidelity in a highly dynamic environment. This work provides a good example of how BED is identifying special Army needs, in this case stemming from boundary layer turbulence, that require hardware solutions that cannot simply be adopted from existing astronomy or high-altitude surveillance solutions. These turbulent imaging and beam transmission experiments, as well as the theoretical studies of image turbulence and hardware solutions (see the section “Turbulence Propagation Theory and Effects,” above), are extremely important to fulfilling BED’s mission. Because the problem of imaging and transmission through boundary layer turbulence is somewhat unique to the Army’s mission, it is not clear that adequate resources and attention are being applied across the DoD to solving this problem in order to deliver the advantage that may be possible using constantly improving hardware and low-power graphical processing hardware. This makes BED’s works even more critical. Applied Anomaly Detection  The work on applied anomaly detection is part of the atmospheric sci- ence initiative within BED, and it specifically links the social-cognitive and information genres. It has both 6.2 funding and customer-driven funding sources. This project is noteworthy because it brings together cognitive processing experts, ARL machine learning experts, sensor experts, and a military expert with the goal of giving the soldier the capabil- ity and training to sense a dangerous situation in a field. The objective is to transfer to a machine the military expert’s ability to find anomalies that signal possible danger in a complex or cluttered field. The team has developed techniques that provide an approach for training the machine and thereafter soldiers to perform at the expert level. Image features such as shape, color, contrast, and texture are used to discriminate anomalies. The Applied Anomaly Detection Tool software incorporates the team’s techniques and is the main enabler of dissemination. The tool is designed for training soldiers through computer-based detection exercises. This solid work occupies an important niche that is matched to the Army’s needs, and it blends the strengths of several disciplines. The research should reflect greater awareness and absorption of learning from related work around the world. Given its potential, ARL should encourage this activity and extend it to other applications that can benefit from capturing human, especially soldiers’, expertise. Opportunities and Challenges Artillery Meteorology  The Army and Marine Corps artillery communities are finally actively pursuing significant upgrades to the BED-supported system for artillery meteorology. The next-generation system, Computer Meteorological Data Profiler (CMD-P), which BED claims to be revolutionary, is built around a yet-to-be-tested dynamic microscale model derived from the widely used WRF modeling system. The ability to assimilate a variety of nontraditional observations (e.g., opportunity-met observations from a passing unmanned aerial vehicle) is a key element of the modeling system. The described work is in an early stage. Success in this endeavor will require addressing several challenges, particularly in light of the amount of computing power likely to be available in the field. Using WRF at the desired 0.5 km

OCR for page 18
24 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY spatial resolution will push against the limits of applicability of the physics (e.g., turbulence closure scheme) in the model. Model development should proceed in parallel with a verification process, and the model should be designed to be easily upgraded as advances are made in the WRF model by the wider meteorological community. Some aspects of this work parallel efforts to adapt the WRF to model- ing small-scale effects in the urban boundary layer; the CMD-P2 developers should carefully monitor these parallel efforts. BED developed, maintains, and supports a number of numerical mesoscale and microscale models. It would be advantageous for BED to develop a common modeling system/frame- work rather than a series of unique models. Verification and Validation of Models  Consistent, appropriate verification and validation of meso- scale and microscale models remains a challenge for the wider meteorological numerical modeling community. BED is still at an early stage in arriving at an assessment scheme for its various numerical prediction models. The exploration of the NOAA/NWS RTMA product1 is a reasonable, but only a first, step. BED should reconsider the NCAR DTC MET system2 and explore some of the other assessment tools that have been developed in university modeling programs. This effort is a vital one for the BED modeling team. The credibility of and trust in the performance of BED-developed models needs to be based on a solid, well-accepted verification and validation scheme. Need for New Physics in the Modeling System  BED’s work to apply the WRF model to small spatial scales (for example, 1.0 to 0.5 km) appears to be closely related to that of Artillery Meteorology and other soldier-scale environmental prediction projects (see the section “Weather Research and Forecast Model-Based Nowcasting for Battlefield Operations,” above). Most importantly, this effort is pushing up against the physics of the modeling system, which represents an opportunity for BED to make a fundamental contribution to numerical modeling at the meso- and microscales, given its expertise in the area of empirical observations of turbulence in small regions. A new, well-verified turbulence closure scheme appropriate for small scales for use in the WRF model would be welcomed by the wider model- ing community. Again, this development effort should be supported by a parallel verification effort. The NOAA Meteorological Assimilation Data Ingest System (MADIS) system has latencies that are too long to use with a meso-/microscale modeling system supporting nowcasting. Anisotropic Turbulence  Anisotropy in turbulence near Earth’s surface represents a fundamental problem of boundary layer meteorology that requires characterization for updating our very simplistic isotropic view of turbulence theory. The case of strong anisotropy needs to be parameterized for practical use. Turbulent features in complex terrain and urban environments can be very important for transmis- sion path calculations, unmanned aerial vehicle operations, and wind energy planning. Continuing work toward making this type of information accessible and easy to understand and use is encouraged. A good step in this direction would be to reach outside the community by publishing a fully peer-reviewed paper on this subject. The work is very interesting and has practical application for the Army in the field/ theater and on its bases, as well as for its use of wind turbines for some of its energy needs. The knowl- edge would also have valuable civilian applications. BED mentioned planned use of lidar to measure turbulence structure in the atmosphere. Measurements at elevations on the NOAA Boulder Atmospheric 1Real-Time Mesoscale Analysis (RTMA) products from the National Oceanic and Atmospheric Administration’s (NOAA) National Weather Service (NWS). 2NCAR DTC MET system stands for the Model Evaluation Tools (MET) verification package developed by the National Center for Atmospheric Research (NCAR) Developmental Testbed Center (DTC).

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 25 Observatory tower, higher than provided in the earlier data sets, and the promised tethered turbulence sensors, should allow further characterization in terms of height dependence of these ­ nisotropic struc- a tures. Another interesting avenue of continued work is the study of Fourier transformation use for these turbulent temporal or spatial data series (even though the data are not derived from wave forms), which could have very interesting theoretical ramifications in time. BED might be approaching the time when its high-quality work in adaptive optics can provide atmospheric parameters to the atmospheric modelers and experimenters to improve their ability to gen- erate spatially dense estimates of the near-Earth atmosphere state. BED should direct attention to this possible synthesis of information. As BED refines its strategy to handle opportunities and challenges, it should expand its good working relationships with university groups to include those that are strong in fundamental modeling research. The modeling challenges, especially the ones mentioned above, are substantial, and extending the uni- versity ecosystem to include partners that are strong in fundamental modeling research will be valuable. BED should also welcome industry partnerships. BED should consider the possibility of working with ARO to establish a center of excellence that would bring together appropriate partners from universi- ties and industry, to contribute to the effort in which it is both a leader and a player with a large stake. Overall Technical Quality of the Work BED’s management and staff members are to be congratulated for their progress toward becoming a first-class research organization, especially in times when the kind of research they are conducting is under stress. The presentations delivered, both orally and through posters, were of very high quality and represented a significant concentration of effort on solving fundamental scientific problems. Even the warfighter-focused applications contained a high degree of engineering research leading to noteworthy achievements. Discussions with division personnel revealed significant pride in the division and in its research, as well as very good morale, which are essential to continued success. Computational Sciences Division Changes Since the Previous Review High-performance computing (HPC) is an essential enabling capability that supports many ARL research programs. Many ARL projects presented during the 2011 and 2012 reviews, including those from outside the CSD, involve very large data sets or require extensive computation. CSD’s HPC resources and its ongoing computational research are important to ARL’s research more broadly. HPC’s role in various ARL research projects and Army applications has grown since the 2009-2010 review, and it is expected to continue to grow. Emerging and future advances in computational science will expand the applicability and utility of HPC to ARL and other Army research. Many of the ARL projects involve models of some form. In the 2009-2010 reviews, model verifica- tion and validation were often absent or insufficient. In the 2011-2012 reviews, many of CSD’s projects provided evidence of increased attention to the importance of verification and validation, and to the appropriate methods of performing these. In general, the overall technical quality and focus of CSD’s nascent research program is improving rapidly as it gains maturity and coherence.

OCR for page 18
26 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY Accomplishments and Advancements CSD’s structure includes both a substantial facilities component that serves the HPC and network- ing infrastructure needs of ARL, the Army, and the DoD, and a new and growing research component focused on interdisciplinary computational science. The CSD team has made substantial progress in a very short time in articulating a research vision and realigning activities to support that vision within the research component. CSD’s research component has three major thrusts. Simulation science leverages existing CSD and ARL strengths along strategic areas of opportunity and impact including multiscale modeling (aligned with the computational materials science initiative), simulations of networks (aligned with the network science initiative), and biological HPC. Computational architectures is aimed at the development of computing architectures that support Army missions, including tactical cloudlets, hybrid computing, and power aware computing. Computer science and high-performance networking encompasses com- putational environments, HPC solutions for large data set analytics, leveraging the ARL Enterprise Optical Network (AEON) for networking research, and new networking architectures. In general, CSD is appropriately focused on determining how to effectively deploy and exploit HPC capabilities, rather than on how to develop new HPC and networking hardware. CSD’s multiscale materials modeling program focuses on fundamental materials modeling research that requires intensive computation capabilities. In general, this program is seen as potentially very important because of its breadth of relevance to the Army mission, for example, from electronic and photonic materials and devices to new reactive armors. Care should be taken to preserve this essential connection to Army applications in this program’s individual research projects. The combination of computational resources with important materials problems plays to the strengths of ARL and CSD. This program provides an excellent opportunity to identify and drive research in simulation science, especially in algorithmic innovations needed for scalability while meeting accuracy and stability requirements imposed by complex physics-based models. There was evidence that CSD understands that verification and validation is required not only for the models, but also for the data exchanges between the models and transitions between scales. However, this is a huge field, and focus is necessary to ensure that iden- tifiable progress can be made in some selected areas with the available resources. As this program moves forward, several important issues concerning the basic premise of multiscale modeling itself should be considered; two are mentioned here. The first issue pertains to the semantics of interconnecting models at different scales and the likely challenges that will arise in doing so. Pass- ing output values from one model to the next model to use as input is not the real challenge. Rather, the challenge is to validate that the domains of applicability and assumptions of the interconnected models are compatible over their entire range of parameterization, and that they remain so even as those models are modified and adapted to new purposes. Attempting to do this in general for all applicable models is far too broad an objective for this program and should be avoided; instead, a real and feasible contribu- tion would be to define the information-passing taxonomy. The second issue relates to the belief that modeling at the electron-atom level can in principle be extended to the continuum level by assuming that one can successfully bridge gaps between modeling scales. This presupposes, however, that the physics of the phenomena being modeled are known at all scales, which may not be true. For example, the atomistic-level calculations are based on energy minimization schemes, i.e., essentially the assump- tion of thermodynamic equilibrium. However, at the mesoscale, most materials systems are far from thermodynamic equilibrium and could be described using macroscopic non-equilibrium thermodynam- ics (which could be linear non-equilibrium thermodynamics, nonlinear thermodynamics, and extended thermodynamics).

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 27 One project within the multiscale materials modeling program explored the thermal performance of materials most relevant to the Army (such as the explosive RDX). Its novel simulation approach uses phonon-Boltzmann transport to bridge from the atomic (nano) to the continuum (meso) scale. Initial simulations results reported for 1-dimensional (1D) quartz were validated against experimental mea- surements of the specific heat of RDX. The initial 1D results were promising, with better agreement (a 7 percent discrepancy) than that produced by a full molecular dynamics model. A journal publication based on this work is still in preparation; only conference papers have been published thus far. CSD’s simulation of defects and dislocation dynamics project, which is part of the simulation science research thrust, focuses on computational simulation of defects and dislocations to enable the design of electronic materials of Army relevance. The project leverages an existing code for dislocation dynamics and couples it to an in-house finite element modeling code to provide new capabilities, such as surface detection and tracking of defects in gallium nitride (GaN) films. Model validation consists of comparing computational and analytic results, which is a good initial step. Further validation by comparing experi- mental data from observations with actual material is essential to fully establish model credibility and utility. The research results should be targeted for publications in top venues. An evaluation of alterna- tives to the project’s distributed shared memory (DSM) approach for code coupling should be conducted; DSM could become a bottleneck for the scaling of simulations to model realistic time and length scales. Explosions under vehicle bodies are currently a significant cause of casualties during Army opera- tions. The objective of CSD’s reduced order models for underbody blasts project is to execute blast predictions faster and cheaper, ideally on a cell phone. In general, high-fidelity models are expensive, are highly nonlinear, and have too many parameters for efficient calibration and optimization. This project’s premise is that in many cases a significant portion of the parameter space belongs to a subspace with significantly lower dimension, i.e., fewer relevant parameters. The project uses a high-fidelity model and a design of experiments method to select the most important parameters and then Galerkin projec- tion to reduce model dimension. The preliminary results seem promising, showing a sample reduction of degrees of freedom from 1.4 million to 50. These results were mainly reported in technical reports and presentations. The effectiveness of the method should be evaluated in comparison with that of other dimension reduction approaches when further results are produced. Opportunities and Challenges CSD’s scientific research productivity and progress are substantial, given the relatively small num- ber of personnel (21 full-time equivalents) devoted to research. However, the results of some projects oriented toward fundamental research have not been published in high-quality peer-reviewed venues. Furthermore, no reliable metrics are in place within CSD for evaluating the effectiveness or impact of software artifacts developed as part of, or as the primary goal of, CSD research projects. Management should be sensitive to the possibility of competition within CSD for resources support- ing HPC infrastructure versus performing basic research in computational science. The effort expended by CSD on general purpose software development should be measured and recognized. The decisions on resource allocation will have far-reaching impact. CISD should further invest in CSD with the intention of increasing the number of personnel dedicated to computational science research. However, recruiting may be difficult because of the many opportunities for computational scientists in other areas. Not all CSD projects were well justified. All projects should be evaluated to ensure that the benefit, which should be specifically defined and quantified whenever possible, justify the cost (time and money). Furthermore, a project’s context is important—What impact will it have? Where is it going?—and should be known and articulated.

OCR for page 18
28 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY Awareness of prior extramural work continues to be uneven. Some project presentations demon- strated that the researchers were very familiar with relevant previous research and software; unfortu- nately, others did not. In general, repeating work is a suboptimal use of scarce resources. The division’s real-time radio frequency propagation path loss project focuses on performing general purpose computing on graphics processing units to model path loss in radio frequency propagation, with a goal of modeling 5,000 devices in real time. The project uses Longley-Rice, transmission line matrix (finite difference), and ray tracing modeling methods. Considerable effort has been expended to transfer the FORTRAN code to another language. The models do not account for foliage, which would be very important at higher frequencies (for example, >500 MHz), and there is no incorporation of the signal bandwidth, which may be significant. The existing terrain topography databases may not be adequate for the calculations. The potential benefit of this research does not appear to have been quantified, and sufficient awareness of highly related prior art, for example previous brigade-level modeling in the Force XXI Battle Command Brigade and Below (FBCB2) project, was not evident. However, if the research is successful, then the models could be good tools for understanding system limitations. CSD’s tactical cloudlets program is based on a promising concept with the potential to deliver HPC “on demand” to the soldier, which could have a transformative impact on the Army’s mission. However, there are major research challenges related to the designs of such a system and of a research program to support it. To better support the design of an extensible cloudlets system that can bring new capabili- ties to the Army of the future, CSD is encouraged to develop a research roadmap that includes both near-term (1-year) and long-term (5-10 years) goals, strategies, and milestones. The roadmap should include a list of specific planned or envisioned research tasks that both fit within the program’s concept and support the Army’s mission. As this work continues and new short-term projects are selected along this theme, CSD should emphasize greater selectivity and prioritization of projects, with a focus on projects that can (in the near term) both deliver benefits to the soldier and provide insights into the design space of a broad tactical cloudlet effort. Finally, this effort does not fully conform to the emerging consensus definitions within the larger technical community for the terms “cloud” and “cloudlet.” As currently formulated, this effort might be better described as “battlefield HPC,” rather than as “tactical cloudlet.” Consistency should be sought in describing the effort and in setting its research agenda. Clouds or caches will be deployed all over the battle areas. The big difference from earlier systems is that cloud systems are substantially larger in terms of storage, processing, and the ability to communicate with them, and clients operate much more autonomously. Basically, ARL needs a facility that is operated as a cloud with all that that implies. It would not be an HPC server, although the claim will undoubtedly be that such a private cloud exists. As an initial seed project in a new area for ARL, CSD’s relationship between brain structure and function project has a well-defined and reasonable scope and has made good progress. Its results show that multiple instances of ordinary differential equations previously used separately to simulate brain activity can influence each other in ways that qualitatively resemble linkages between brain regions. However, the presentation’s descriptions of the project results as “validation,” and the assertion that these results show that the model output “can be used as simulated brain signals,” substantially overstate the project’s current status. Validation by comparison to empirical data is required before those claims can be made. Every practicing modeler knows from experience, and it has been formally proven, that composi- tions of separately valid models are not automatically valid and need to be validated as a composition. Further­ ore, the project’s scale may be too small to initiate a new CSD research thread in traumatic m brain injury, and if that is the goal, then a larger-scale effort may be needed. Collaboration with neuro- science or medical researchers is recommended for access to validation data and modeling guidance.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 33 Opportunities and Challenges Validation of the Network Science Vision and Strategy ARL should approach the network science vision and strategy as a series of promising hypotheses, each of which remains to be validated. The following important question has not been clearly answered: Does the ARL research in network science reflect an understanding of the Army’s requirements for the research? To answer the question rigorously, it will be helpful to understand how the results of the technical effort will translate to benefits for the Army and the soldier, and how the metrics will be used to quantify the benefits. Similarly, it will be helpful to have compelling examples of such benefits. However, because this research is in the early stages and there are no results to validate or compelling examples to review, the project should develop imaginary scenarios where the as-yet imaginary results of the research effort will have a beneficial impact. Of course, all of these factors need to be refined and iterated over time in parallel with the research’s evolution. The 2009-2010 assessment report contains a section titled “Challenges in Systems Engineering” in which it is noted that “it remains a significant challenge across CISD to ensure that even in relatively basic research programs a good understanding is formed about how potential systems that might be developed out of research might be deployed and used in real Army scenarios.” The report adds that only a small amount of investment in the early stages of a program is likely to be repaid handsomely later in impact on the Army. These observations remain valid with regard to network science, given its large scale and ambitions, as well as its being in the early stages of the R&D continuum. What Is NOT Network Science? The labeling of certain projects as network science could only be justified by an extraordinary stretch of the definition of network science. Examples of such projects are machine translation and subjective logic using partial observation. There are two points of view on this issue. With one, the “big umbrella” perspective, it does not matter how a work item is labeled as long as it is of high quality and value is generated. In fact, expedi- ency argues for this approach, because network science is currently a hot topic, and it plays well with various clients to show that large resources are dedicated to it. With the other point of view, the lack of rigor in the definition and application of what exactly is meant by network science is likely to have unintended and detrimental consequences. For instance, genuine initiatives in network science may get squeezed out in the competition for resources. Also, the lack of rigor may have the pernicious corrupting effect of diluting standards of technical excellence. ARL management should better define network science and its boundaries, articulate these defini- tions in a document, and apply them rigorously in categorizing work. Reconsideration of the entire computing research structure will also be associated with any redefinition of network science. Cross-Genre Research Although there was evidence of a desire to conduct cross-genre research, there were few examples of such research, suggesting that this aspect of the initiative is still in infancy. The best examples presented tended to have their focus in a single genre or, in exceptional cases, a combination of two genres. As the network science enterprise develops and matures, it will be increasingly important to identify compelling

OCR for page 18
34 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY examples of research in composite networks that makes good on the potential that the network science enterprise strategy has staked out. A good example of cross-genre research is the project on integrated analysis of social and informa- tion links, which aims to combine social-cognitive and information genres and was part of the Network Science CTA in partnership with various universities and ARL. This work combines data-mining tech- niques, estimation theory (maximum-likelihood, expectation-maximization, Cramer-Rao bounds), and optimization over graphs. The estimation and optimization techniques are standard in the repertory of signal processing and information theory in electrical engineering. The project has worked with 1.5 million tweets, which were collected in Egypt during the Arab Spring, to develop methods for recon- structing event timelines from real-time inputs. The inputs are, as might be expected, noisy, incomplete, and conflicting. The project is ambitious. Because noise, correlations, and dependencies abound, new insights might have come from the participation of research statisticians and mathematically trained social scientists, who have considerable experience in these problematic features. The results have been presented primarily in IEEE conferences and journals, with some ACM-sponsored forums as well. A broader discipline base would have been very desirable and would almost certainly have led to a broader array of approaches, methods, and tools to bear on the problem. The above-mentioned project’s connections to the Army’s requirements were not articulated clearly; therefore, it is not apparent whether ARL is playing a sufficiently proactive role in defining and formu- lating the research problems in a manner that addresses the Army’s requirements. Work in the project on socially aware caching in disruption tolerant networks is solid. The results described are primarily from collaboration among university partners within the framework of the network science CTA. This work is positioned as cross-genre, combining social and communication networking. The intention is to use social interest and social relationships, such as roles and positions in the military, to increase the efficiency of caching in tactical mobile networks. However, intentions notwithstanding, the work is primarily mainstream networking, dominated by caching, together with various early-stage attempts to exploit the social dimension. This project would benefit greatly from the active participation and contributions of social scientists working closely with the electrical engineers and computer scientists who are already involved. It would also be valuable to have the Army require- ments and perspective injected more compellingly into the problem formulation and the performance evaluations; only ARL can provide this perspective. Diverse projects in the network science program, such as socially aware replication and caching and smarter middleware, are developing different techniques for getting the right information to the right people at the right time. These projects embody different measurement and modeling approaches to social networks, for example, graphs versus hypergraphs to reflect centrality or in-degree/out-degree. However, there was no presentation of what has been learned from the different approaches. For example, what are the merits and drawbacks of each approach? For what kinds of problems would given approaches be more viable? As these diverse projects progress, the value from syntheses becomes more compelling and the need for it more urgent. ARL should give such higher-level activity more attention. Cross-Organizational Research One of the laudable goals of the Network Science Enterprise is to combine the diverse disciplinary strengths from multiple directorates—in this case, CISD, HRED, and SEDD—to tackle a large problem with potential significant impact. This has yet to happen on any significant scale in the research and technical activities of the enterprise.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 35 To counter any tendencies of staff to apply thinking that is bounded by the constraints of disciplines that are organized by current directorates, ARL should examine the current practices by which the net- work science programs and projects are organized. ARL should consider alternative means of organizing them, to determine the means that most clearly identify and empower technical leaders who will guide teams of staff drawn from across ARL directorates. Trust The presentations on the subject of trust, which is important in examining decision making, provided disappointingly incomplete results. The presentations quickly jumped to metrics and sophisticated math- ematical models and their analyses. For a subject as amorphous and subjective as trust, it is important to work with data, and for data analysis and mathematical analysis to be mutually supportive and inter- active. What data are being used to build the model, and do valid inferences from the data support the model? Examination of trust is certainly a case where real-life scenarios would be helpful at the outset to understand and define the problem before embarking on elaborate models and analyses. Additionally, partnerships with experts in the social-cognitive field were not described. The assump- tions, propositions, and the strategy for validation, all foundational topics, need to be carefully worked out. The research on trust, while important intrinsically and for its impact on multiple network genres, is an early-stage work-in-progress. Premature extrapolation can be harmful. Social-Cognitive Sciences Research The social-cognitive network researchers presented a compelling picture of the context of applica- tion in the tactical environment of untethered warfighter teams. How to provide effective support for distributed decision making in such environments constitutes a unique niche for ARL research. In social-cognitive research more focus should be directed at understanding the variables that deter- mine the right outcomes of effective decision making by untethered teams of soldiers. More attention needs to be directed at identifying the qualities of effective decisions. Identifying and specifying the key independent variables, as well as the dimensions of effective decisions, will reveal answers to some of the research questions being asked. An example of where this would be useful is the study that included 22 potential predictor variables for somewhat unspecified decision outcomes. This is where the model- ing, measurement, and project methods should be directed initially. An effective approach is to ground the research efforts in social/cognitive/behavioral theories of context-embedded cognition and social action. For example, much can be gleaned from decision- theoretical approaches, group/team decision making, and distributed decision making. Such research efforts have much to gain from exploiting the relevant research literature, which is extensive. Addition- ally, examination of the literature on fundamental social and behavioral sciences (theories, methods, and measurement methods) would be beneficial to the projects. The project managers should also seek out collaborators and visiting scientists who have strong backgrounds in experimental design, statistics, and creative approaches to measurements. The planned work on increasing focus on group/social dynamics is appropriate, as are plans to expand attention to “influence” processes and improved outcomes of distributed decision making.

OCR for page 18
36 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY Shaping Behavior The novel project on shaping behavior is part of the Social Cognitive Networks ARC component of the Network Science CTA. At its core, this project uses methods that combine crowd-sourcing with financial incentives—techniques also used by the presenter while leading the team that won the DARPA “red balloon” challenge. The goal of the project is to shape the behavior of a large number of people with small, carefully designed and placed incentives. This is a topical subject with great promise for the Army and society in general. The project is in the very early stages of development. This work represents a rare instantiation of the socio-technical approach espoused by the program. It does so by elevating attention from individuals to social-level variables (the hypotheses are all based on groups and social behavior, as opposed to individuals and person-specific roles). The work relies on large networked unstructured communications, as opposed to pair-wise connections. It is not yet clear how this work may be applied in Army-relevant distributed cognition, decision making, and performance, which is an appropriate future direction warranting exploration. A centerpiece of this presentation was a sketch of the mathematical modeling and optimization for the design of incentives and predicting responses. The mathematical development was fairly standard and at a very early stage. “Big Data” Research in Network Science ARL’s and CISD’s management clearly expressed their cognizance of the problems associated with flooding of data (“drowning in data”) and the opportunity cost of not being able to exploit available data and information to benefit the soldier. Of the top five “big ideas” in the Director’s presentation, two are specifically tied to this topic: “decision science” and the tactical cloudlets element of computational science. However, ARL did not convincingly demonstrate that it has garnered and organized into a compre- hensive understanding all aspects of this challenging problem. The research in academia and industry in this area has now gained considerable momentum. This effort has brought together theory, algorithms, and systems, and it has roots in several disciplines, including computer science, mathematics, statistics, and operations research. Machine learning, together with data mining, knowledge discovery, and social network analysis, which were until quite recently obscure branches of artificial intelligence and statistics, are now flourishing within the framework of “big data.” The solutions are no longer static, and the data populating the databases are derived by and drive sophisticated analytic algorithms. The information is being derived from humans and machines (sensors), as well as from huge stores of information on the Web. The knowledge that is mined will have to be comprehensible to humans, and humans making deci- sions will have to be able to actively question the assumptions behind the knowledge extraction process. ARL should be an important driver in this movement, or at least a judiciously selected subset of the movement. ARL has the advantage of having abundant data, the lack of which is often the bane of researchers. ARL should use this advantage to attract high-quality external researchers to collaborate on problems of interest to it. Additionally, it should partner with other Department of Defense (DoD) agencies that are actively involved. For example, the Air Force Office of Scientific Research (AFOSR) sponsors a high-quality program in network science that is viewed as a leader in fundamental research in large data analysis.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 37 Network Science CTA and Networks and Information Sciences ITA Four university professors who are participating in the Network Sciences CTA and ITA provided presentations describing high-quality research that has been, and continues to be, conducted by teams consisting of academic and ARL researchers. The research presented was driven and conducted by university collaborators in the consortia with contributions from ARL researchers that did not appear to be on par. Although the level of connection was uneven across the consortium projects, the level of connection to the Army’s needs, the understanding of the Army’s needs, and the reflection of constraints from the field in the problem formulations were lower than should be the case. It is important that ARL find solutions to the challenging problems of less than ideal ARL involve- ment in the consortia’s research and less than ideal connectivity of consortia research with Army needs. Any solution will require ARL to be more proactive, even aggressive, with its academic partners in translating and then inserting the perspectives from the field into the problem formulation. The col- laborators from ARL will need to match their scientific and technical abilities, as well as their status in their research communities, with that of their university partners. To be clear, the question is not whether ARL is deriving value from the consortia, for which the answer is definitively in the affirmative, but whether the derived value is providing the impact that ARL desires. There is undoubted value from being aware and being involved, at any level, in high-quality research. However, if this value is to be more than educational, then ARL should perform an in-depth investigation of the means and desired outcomes for extracting value from these collaborations. Such an investigation should consider two models for extracting value from collaborations, discussed below under the section “How to Extract Greater Value from University Collaborations.” Overall Technical Quality of the Work The technical quality of the work was assessed according to several criteria posed by ARL. The first criterion asks whether the scientific quality of the research is of comparable technical quality to that executed in leading federal, university, and/or industrial laboratories both nationally and internationally. The answer is generally affirmative for the network science program and for the two divisions in CISD that are intimately involved in the program, the Information Sciences Division and the Network Sciences Division. Exceptional expertise exists in selected areas, such as machine translation, image processing, anomaly detection, and optical communications and networks. There are extensive collaborations with external research entities, especially universities in the United States and United Kingdom, and leading industrial laboratories. Of concern is the extent and nature of the role of ARL staff members in the con- ception, formulation, and execution of the research in the collaborations; a related concern is whether the research’s impact on the Army will be sufficient. CISD has demonstrated greater awareness, at both the managerial and staff member levels, of the desirability of publishing in peer-reviewed and archival journals. An increase in publications in high- quality journals provides evidence that this awareness is slowly having an effect. However, evidence of staff participation in scientific and professional societies is lacking. Multifarious benefits would accrue from such involvement, such as acquiring peer status in external collaborations and collecting informa- tion on the state-of-the-art in technical developments that would be useful in expanding external collabo- rations. Recognition from professional societies would help recruiting and retention. A basic and quite easy step to take in this direction is organizing and hosting workshops. (This has already been done in varying degrees across organizations.) More effective use of enticements that ARL has at its disposal, for

OCR for page 18
38 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY example, its extensive data sets, should be used to encourage participation by universities and industry in ARL-led meetings and workshops located in the ARL facilities and also in high-quality conferences. The second criterion applied during the assessment is whether the research program reflects a broad understanding of the underlying science and research conducted elsewhere. The answer here varies with individual projects, and generally the projects considered to be exceptional are the ones with the best understanding. It should be pointed out that network science is a fledgling area of scientific and engineering research. It is to ARL’s great credit that it has embraced the subject, and ARL is in many ways a pioneer among major laboratories in doing so on the scale that it has. However, not a great deal of basic science and research is being conducted elsewhere to compare to. Nonetheless, there is room for improvement. With the exception of research on shaping behavior, most of the work on distributed decision making involves study of two-person teams, which does not live up to the aspirations of the network science program. After all, research on two-party interactions has been going on for decades, well before contemporary digitally networked media became available. Research needs to reflect the size of distributed communication and decision-making teams in the compellingly depicted tactical environment of untethered warfighters. Providing effective support for collaborative activities in such environments represents a unique niche for ARL research. There is substantial research in areas adjacent to network science, if not directly within it (depend- ing on its definition), that the ARL program should better understand and embrace. In areas such as “big data,” including data mining and anomaly detection, machine learning has taken on a large, even dominant, role. ARL has no tradition of research in this area; a special effort will have to be made to remedy the situation. The answer to the third assessment criterion, that is, whether facilities and laboratory equipment are state of the art, is largely a solid “Yes.” An important exception is in the cloud computing area. The fourth assessment criterion asks whether the research team’s qualifications match the technical challenges. The match is good, but it could be substantially enhanced by remedial action. The fifth assessment criterion asks whether the research reflects an understanding of the Army’s needs. The answer is that the research generally reflects the Army’s needs. However, in several cases, the research’s potential to successfully address specific Army needs was not clearly defined. The sixth assessment criterion deals with the crafting of programs that employ the appropriate mix of theory, computation, and experimentation, which ARL management has clearly identified as an objective, for example in its network science strategy statement. However, the results are uneven. This criterion is coupled to topics such as feedback, validation, and the desirability of greater systems engineering. Not surprisingly, the less established the research topic, such as the integration of social-cognitive genre with other genres, the greater is the gap between the ideal and the current states of integration of theory, computation, and experimentation. This is not surprising, because time is needed to build up a research area. Experimentation is of such overall importance in the conduct of ARL research that it should be an independent subject of research, for which there is precedence. For example, experimental design and exploratory data analysis are established research areas. Because statistics is the natural home for these topics, the involvement of professional statisticians would be helpful. The seventh assessment criterion asks whether especially promising projects could be transitioned ultimately to the field. Several projects have been identified as promising for ultimate transition. There is more that ARL could do, however, with respect to the selection of research focus in the CTA and ITA. Similarly, systems engineering considerations also can be applied from the inception of projects. It would be important to register, track, and quantify the impact of transitions from research to the field.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 39 ARL remains extraordinarily responsive to the ARLTAB’s recommendations. This responsiveness has not diminished over time. Indeed, it has sharpened. It is gratifying for the ARLTAB to see effective responses to recommendations within a year’s span; this is a characteristic of all the CISD units assessed. COMMON THEMES The following two sections discuss common themes that cut across the Network Science Enterprise and CISD. Opportunities and Challenges Cloud Computing Research The investment in cloud computing research in CISD, and generally in ARL, appears small. ARL should consider the merits of expanding its work in this area. A particular concern is that tactical applica- tions of cloud computing, such as tactical cloudlets, are particularly demanding variants, which should be tackled only after achieving mastery of the science and technology of basic cloud computing. ARL should consider making available Hadoop clusters to jumpstart research from the bottom up. These clus- ters, supported by the necessary software engineering knowledge, should be used as platforms to develop machine learning, data mining, search, social networks analysis, and various other applications. Other dimensions to cloud computing, especially for tactical applications, call out for collaborative research, for instance cyber-security and special communication network protocols and resource scheduling to handle near real-time applications (of obvious importance in tactical applications) within the geographically dispersed context of cloud computing. Extensive use of caching will almost certainly be necessary, and mastery of the enabling algorithms will be prerequisite. From a networking perspective, cloud comput- ing integrates processing, storage, and communications, and the research challenges are many. When realized, cloud computing has the potential to give the soldier a significant technological advantage. Cyber-Security Research Cyber-security science has been positioned as starting from a relatively small base with intentions to be a growth area. For instance, CISD lists cyberdefense as one of its “top 5 future big ideas.” Man- agement of the Network Sciences Division (NSD) has deferred questions of strategy for cyber-security research to the next year, because that strategy has not yet been defined. These decisions are appropriate. A significant amount of thought and effort will be needed to formulate a strategy and an action plan. The topics in cyber-security are expansive, and the competition for high-quality researchers is fierce and growing. ARL should adopt a targeted approach to reach Army-specific goals in order to maximize impact on Army cyber-systems. This can be achieved through careful planning that selects problems that are difficult but can feasibly be addressed by a relatively small group of researchers. Considerable capabilities exist in other DoD agencies in such areas as cyptography and malware analysis. Without extensive knowledge of the work by others in these areas, CISD risks duplication of effort. CISD is working in the areas of intrusion detection systems and alert management. There is a considerable body of prior work and a sophisticated and broad commercial market for technologies in these areas. It does not appear as though there are important research topics to be mined in the areas of intrusion detection systems and alert management.

OCR for page 18
40 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY There are many research questions related to advances in data leak prevention that require new insights and technical developments that are not shackled by the biases of unimaginative approaches to authentication and authorization. ARL may wish to consider and develop this area of research. Scal- able deception is an underdeveloped technology that is directly responsive to data leak prevention and that is fraught with challenging research problems. Successful work in this area would respond to the congressional mandate to prevent data leaks from sensitive DoD sources. Also worthy of further consideration are the topics of “science of security” and “security metrics.” These, together with the topic of insider threat, involve research problems that a small number of high- quality researchers in ARL can address with potentially high impact. Overall Quality of the Work The Advisability of Broadening the Discipline Base and Strengthening Fundamental Research There is great value in developing a broader base in the disciplines at ARL. Certain disciplines, such as computer science, are obviously greatly needed and also fairly well represented. Yet other dis- ciplines are under-represented or not represented. One example is statistics. Several areas are closely connected to statistics, including machine learning and statistical computing. Statisticians characteristi- cally support research across disciplines and organizations. Social scientists, especially when trained in mathematics and quantitative analysis, have the potential to significantly lift the quality of several ARL projects, especially in the social-cognitive area. There is ample scope of projects for mathematicians at ARL. Consider, for example, the value of mathematical formulations of the information network and its interactions with adversaries, mathematical formulations of features in data, and secure multi-party computations. It is not a big jump from mathematical formulations of structures to algorithms that allow fast computations from data. Given the competition for algorithm specialists, a possible path may be to hire mathematicians and train them in algorithms. ARL and CISD would benefit from a bigger commitment to fundamental research. There is consid- erable room for improvement in fundamental understanding of problems and in advancing fundamental scientific knowledge. Especially in theory, this dimension of research seems to have been conceded to the university collaborators. If true, then this would be a mistake. It is essential for ARL to be equal partners in all dimensions of any research collaboration to extract full value. ARL Researchers’ Interactions with the External Technical Community ARL management and staff members have already taken to heart prior ARLTAB recommendations for proactive engagement in the technical community, especially outside the Army and DoD. Continu- ing that progress is recommended. External Recognition ARL should approach external recognition in professional societies in a coordinated, systematic way that is driven by appreciation of the multi-dimensional value that would be generated. Consider the IEEE, which is representative of professional societies. It has three membership levels: member, senior member, and fellow. IEEE requires membership for a minimum number of years at one level to be eligible for membership at the next level. ARL should consider this hierarchy as a pipeline, and it

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 41 should provide incentives for its staff to advance to the next stage and to provide mentoring and other forms of assistance. This process could be replicated across key societies. Professional Activities Professional activities may be conducted on a stand-alone basis (for example, ARL would organize and host meetings of broad interest, inviting high-quality, high-visibility external speakers); in partner- ship (for example, with other federal agencies); or with the support of the professional societies and the organizers of high-quality conferences. Examples of activities include organizing workshops, seminars, and conference sessions on topics of special interest to the Army. Being an organizer will make it easier for ARL to accommodate other ARL researchers in the program. Locating such meetings in ARL facili- ties has additional value. Centers of Excellence The National Science Foundation (NSF), among other agencies, funds many centers of excellence. ARL should consider engaging such centers to help drive its research program. The NSF and the AFOSR, for example, typically conduct a stream of workshops, research programs, and other engagements with specific technical communities. ARL could take advantage of these meetings and workshops to refine the formulation of research problems and to better position itself to press its agenda in work with con- sortium partners. Publications CISD should build on its recent progress with publications by pressing ahead with its program to increase the quality and quantity of publications in peer-reviewed, archival journals. ARL and CISD should take a systematic approach, starting with the composition of an inventory of all journals and conferences that are deemed to be top tier in various respective fields, a tracking system of publications by divisions, and a system of incentives and rewards for staff members. Online Courses Online courses are becoming widespread, and quality online learning opportunities are increasing. ARL and CISD should consider exploiting this nascent practice and technology. Staff members would benefit from completing courses in computer science, artificial intelligence, and programming taught by experts from top schools. Conversely, ARL should consider offering select courses for ARL staff and possibly for staff at the extended DoD community. How to Extract Greater Value from University Collaborations ARL may not be realizing top value from its investments in the university consortia. Quite often the research itself is of high quality, but equally often it is more suited to goals posed by academia and offers less than what might be possible for ARL and its staff, the Army, and for advancing the likeli- hood of transitioning to the field. ARL is encouraged to examine the means for extracting greater value from such collaborative consortia. ARL should consider two models for extracting top value from the university collaborations, and these are not mutually exclusive. In the first model, ARL researchers are

OCR for page 18
42 2011–2012 ASSESSMENT OF THE ARMY RESEARCH LABORATORY peers to their academic partners in the research, so that the typical, frequent, face-to-face interactions that might occur among university collaborators in the same corridor also occur across the university-ARL boundary. For this model to apply the ARL researchers need to possess the same depth and stature in the technical area as the university researcher, and, from what has been observed, this is typically substantial. In the second model, ARL researchers perform the invaluable function of translating Army requirements and educating the academic partners on field constraints and Army needs. In this capacity, ARL helps to transform academic problems into no less challenging problems that are also of interest to the Army; that is, ARL staff act as links in the bidirectional feedback loop between academia and the laboratory. Both models have value for ARL, and both have demanding requirements. For the first, collaborating researchers from ARL need to have as deep a fundamental understanding of the science as the university collaborator. Other characteristics also need to be aligned. For example, the publication records need to be comparable, as do external recognitions. In the second model, for the field constraints to be persuasive and for clever, responsive problem formulations, the intellectual depth also needs to be comparable on both sides. In several respects the second model is more demanding. ARL should decide on a model and an implementation strategy. Staffing Staffing gaps, especially in connection to disciplines, were mentioned above, as have staffing issues in building the embryonic cyber-security research area and in balancing research and HPC infrastruc- ture support. Another noteworthy case is the under-staffing of the Information Sciences Division (ISD) in CISD. Given the pulls on ISD from diverse direction, including machine translation, robotics, and customer-driven activities, the residual resources are insufficient for addressing the problems of data, which should be at the core of ISD’s research agenda. Beyond undercutting a key technical area, the understaffing of ISD has other consequences. The comparative data across divisions on publications in archival journal, after normalization by the number of Ph.D.’s in the divisions, was notably lower for ISD, which may be due to under-staffing. CISD should consider the staffing issue in ISD. Role of Patents to Give Impetus to the Culture of Innovations Patents are an important part of the culture of industrial laboratories and increasingly of universities. The role of patents in ARL’s culture appears to be relatively small, which may be how it should remain. However, when properly managed, patenting in the mind-set of industrial researchers enhances the cul- ture of innovation. Major patents and their inventors are recognized universally, with significant benefits to the home organization. Financial rewards are a major incentive. Having the requisite staff to manage the legalities and processing implies increased overhead. It is good practice for organizations such as ARL to periodically review their policies pertaining to patents, the potential for collaborating and pooling resources across sister government organizations, and the methods of communicating policies to staff. Director’s Strategic Initiative and Director’s Research Initiative ARL is to be complimented for establishing and maintaining its Director’s Strategic Initiative (DSI) and Director’s Research Initiative (DRI), which are nurturing a bottom-up approach to the selection of research problems. The breadth and relevance of the 16 research topics currently supported by DRI are indeed impressive. If it is not already being done, tracking and analyzing the history of these projects may be useful in guiding research management.

OCR for page 18
COMPUTATIONAL AND INFORMATION SCIENCES DIRECTORATE AND NETWORK SCIENCE ENTERPRISE 43 The CISD management and research teams continue to be responsive to ARLTAB’s recommenda- tions. CISD has made significant changes in strategies and research processes that alleviate problems on which the ARLTAB commented in the past. Also noteworthy is the care and attention that CISD’s management, starting with the Director and the division chiefs and extending to staff members, have given to planning the panel meetings and preparing the material for presentations in the panel reviews and posters. The ARLTAB appreciates the year-to-year improvement in the proceedings. All of these developments are evidence to support the belief that ARL is receptive to feedback from the ARLTAB. Such receptivity is a hallmark of a dynamic institution that constantly strives for improvement in the pursuit of technical excellence.