2005-2006 Assessment of the Army Research Laboratory (2007)

This introductory chapter describes the biennial assessment process, identifies important research areas that involve crosscutting collaboration across the Army Research Laboratory (ARL) directorates, summarizes the results of the Board’s review of ARL’s crosscutting work in quantum computing and communications, and notes the linkage between the Army Research Office (ARO) and the ARL directorates.

The charge of the Army Research Laboratory Technical Assessment Board (ARLTAB) is to provide biennial assessments of the scientific and technical quality of ARL. These assessments include the development of findings and recommendations related to the quality of ARL’s research, development, and analysis programs. The Board is charged to review the work only of ARL’s six directorates—and this excludes the review of two key elements of the ARL organization that manage and support basic research: the Army Research Office and the Collaborative Technology Alliances. While the primary role of the Board is to provide peer assessment, it may also offer advice on related matters when requested to do so by the ARL Director; the advice provided focuses on technical rather than programmatic considerations. The Board is assisted by standing National Research Council (NRC) panels that focus on particular portions of the ARL program. The Board’s assessments are commissioned by ARL rather than by one of ARL’s parent organizations.

The Army Research Laboratory Technical Assessment Board currently consists of 11 leading scientists and engineers whose experience collectively spans the major topics within the scope of ARL. Six panels, one for each of ARL’s in-house directorates,¹ report to the Board. Each Board member sits on a

panel, six of them as panel chairs. The panels range in size from 9 to 17 members, whose expertise is tailored to the technical fields covered by the directorate(s) that they review. In total, 83 experts participated, without compensation, in the process that led to this report.

The Board and panels are appointed by the National Research Council with an eye to assembling balanced slates of experts without conflicts of interest and with balanced perspectives. The 83 experts include current and former executives and research staff from industrial research and development (R&D) laboratories, leading academic researchers, and staff from Department of Energy (DOE) national laboratories and federally funded R&D centers. Nineteen of them are members of the National Academy of Engineering, three are members of the National Academy of Sciences, a number have been leaders in relevant professional societies, and several are current or past members of organizations such as the Army Science Board, the Air Force Scientific Advisory Board, the Air Force Weapons Laboratory, Defense Science Board panels, and the Defense Advanced Research Projects Agency (DARPA).

The Board and its panels are supported by National Research Council staff, who interact with ARL on a continuing basis to ensure that the Board and panels receive the information they need to carry out their assessments. Board and panel members serve for finite terms, generally 4 years, staggered so that there is regular turnover and a refreshing of viewpoints.

Biographical information on the Board members appears in Appendix B, along with a chart listing the Board membership and the name of each panel, its membership, and the name of the ARL directorate that it reviews.

The current report is the fifth biennial report of the Board. Its first biennial report was issued in 2000, and annual reviews by the Board were issued in 1996, 1997, and 1998. As with the earlier reviews, this report contains the Board’s judgments about the quality of ARL’s work (Chapters 2 through 7 focus on the individual directorates). The rest of this chapter explains the rich set of interactions that support those judgments.

The amount of information that is funneled to the Board, including the consensus evaluations of the recognized experts who make up the Board’s panels, provides a solid foundation for a thorough peer review. This review is based on a large amount of information received from ARL and on panel interactions with ARL staff. Most of the information exchange occurs during the annual meetings convened by each panel at the appropriate ARL sites. In both scheduled meetings and less formal interactions, ARL evinces a very healthy level of exchange and acceptance of external comments. The assessment panels engaged in many constructive interactions during their annual site visits in 2005 and 2006. In addition, useful collegial exchanges took place between panel members and individual ARL investigators outside meetings as ARL staff members sought additional clarification about panel comments or questions and drew on panel members’ contacts and sources of information.

Panel meetings lasted for 2 or 2½ days, during which time the panel members received a combination of overview briefings by ARL management and technical briefings by ARL staff. Prior to the meetings, some panels received extensive materials for review, including staff publications.

The overview briefings brought the panels up to date on the Army’s long-range planning. This context-building step is needed because the panels are purposely composed mostly of people who—while

experts in the technical fields covered by the directorates(s) they review—are not engaged in work focused on Army matters. Technical briefings for the panels focused on the research and development goals, strategies, methodologies, and results of selected projects at the laboratory. Briefings were targeted toward coverage of a representative sample of each directorate’s work over the 2-year assessment cycle. A growing trend is to include poster sessions that allow direct panelist interaction with other projects that either were not covered in the briefings or were covered in prior years.

Ample time during both overview and technical briefings was devoted to discussion, both to clarify each panel’s understanding and to convey the observations and understandings of individual panel members to ARL’s scientists and engineers (S&Es). The panels also devoted sufficient time to closed-session deliberations, during which they developed consensus findings and identified important issues or gaps in a panel’s understanding. Those issues or gaps were discussed during follow-up sessions with ARL staff so that the panel was confident of the accuracy and completeness of its assessments. Panel members continued to refine their findings, conclusions, and recommendations during written exchanges and teleconferences after the meetings. When necessary, the panels received presentations that were classified at the Department of Defense (DoD) “Secret” level. This report, however, does not contain classified information.

In addition to the insights gained from the panel meetings, Board members received exposure to ARL and its staff at Board meetings each winter. Also, some Board members attended the annual ARL Program Formulation Workshop, at which the ARL directorates discussed their programs with the directorates’ customers. In addition, several Board members attended the 2005 and 2006 symposia that highlighted progress among ARL’s Collaborative Technology Alliances (CTAs).

The Board and panels applied assessment criteria organized by six categories (Appendix C presents the complete set of assessment criteria):

1. Effectiveness of interaction with the scientific and technical community—criteria that indicate cognizance of and contribution to the scientific and technical community whose activities are relevant to the work performed at ARL.

2. Impact on customers—criteria that indicate cognizance of and contribution in response to the needs of the Army customers who fund and benefit from ARL R&D.

3. Formulation of projects’ goals and plans—criteria that indicate the extent to which projects address ARL strategic goals and are planned effectively to achieve stated objectives.

4. R&D methodology—criteria that indicate the appropriateness of the hypotheses that drive the research, of the tools and methods applied to the collection and analysis of data, and of the judgments about future directions of the research.

5. Capabilities and resources—criteria that indicate whether current and projected equipment, facilities, and human resources are appropriate to achieve success of the projects.

6. Responsiveness to the Board’s recommendations—The Board does not consider itself to be an oversight committee. The Board has consistently found ARL to be extremely responsive to its advice, and so the criterion of responsiveness encourages discussion of the variables and contextual factors that affect ARL’s implementation of responses to recommendations, rather than an accounting of responses to the Board’s recommendations.

During the assessment, the Board emphasized the following considerations:

1. How does the work of the ARL compare with similar work being performed elsewhere?

2. Is the work sufficiently unique and appropriate to the ARL niche?

3. Are there especially promising projects that, with application of adequate resources, could produce outstanding results that could be transitioned ultimately to the field?

In July 2006, the Board met for 2 days to share members’ summaries of their panels’ observations and concerns. This report represents the Board’s consensus findings and recommendations. The Board’s aim with this report is to provide guidance to the ARL Director that will help ARL sustain its process of continuous improvement. To that end, the Board examined its extensive and detailed notes from the many Board, panel, and individual interactions with ARL over the 2005-2006 period. From those notes it distilled a short list of the main trends, opportunities, and challenges that merit attention at the level of the ARL Director. The Board used that list as the basis for this report. Specific ARL projects are used to illustrate these points in the following chapters when it is helpful to do so, but the Board did not aim to present the Director with a detailed account of 2 years’ worth of interactions with bench scientists. The draft of this report was subsequently honed and reviewed according to NRC procedures before being released.

The needs and opportunities for developing new approaches to selected areas of crosscutting research (i.e., research involving collaboration across two or more ARL directorates) were identified in the Board’s 2003-2004 assessment report.² At that time, modeling and simulation, information assurance and security, and interdirectorate activities were singled out as notable topics worthy of special attention. Although it is clear to the Board that significant progress has been made by ARL in encouraging and facilitating research in each of these areas, the need for additional crosscutting work has also accelerated during the past 2 years. The Board strongly encourages ARL to continue to support new interdisciplinary initiatives. The following areas of particular opportunity are discussed below: high-performance computing, autonomous system common technologies, information fusion, information security, ad hoc wireless networks, and system prototyping and model verification and validation.

It is clear that ARL views high-performance computing as a critical technology driven by requirements from a variety of applications across multiple directorates, including armor and armaments, atmospheric modeling, aerodynamics, and computational biology. The response to these requirements varies across scientific disciplines and across the directorates, including the use of third-party codes, the development of in-house code capabilities, and small R&D efforts of generally short duration. In addition, the strategic plans of the Computational and Information Sciences Directorate (CISD) include petascale computing as an important strategic direction.

The Board is concerned that the worthy ambitions of ARL may outstrip the resources, especially human, available to the laboratory. Even the application of existing simulation tools requires significant investments to understand the applicability and limitations of the algorithms and software, as well as for the extension of such tools to meet specialized ARL needs. Algorithm and code development originating within ARL must be carefully planned, particularly in an era of limited resources: the development of new simulation capabilities involves substantial expenditures over long periods of time. Finally, petascale computing is an enterprise with large uncertainties and risks: the hardware is an order of magnitude more expensive than current hardware and will require a much larger space and power infrastructure, and the vendors will be supplying much less of the software stack needed to use the machines productively. Moreover, it is important to address the question of readiness of any relevant software for a petascale architecture before such acquisitions are made.

Given these risks, and the importance of simulation to the ARL mission, the Board recommends that ARL sponsor a crosscutting study on modeling, simulation, and high-performance computing. Issues that could be examined in such a study include:

1. ARL applications drivers, both current and emerging;

2. Current ARL capabilities in simulation and modeling;

3. Opportunities for new algorithmic and software technologies to have an impact on ARL work;

4. Implications for high-performance computing requirements at ARL, including hardware, software stack, middleware libraries, and applications codes;

5. A method for verification and validation; and

6. Strategic planning issues, including building core competences, developing team structures, and opportunities for leveraging across applications, domains, and directorates.

Current military operations are employing robotic systems in unprecedented numbers and roles. Many types of unmanned aerial vehicles (UAVs) are used for intelligence, surveillance, and reconnaissance (ISR); targeting and tactical intelligence; and even tactical engagement in certain circumstances. Unmanned ground vehicles (UGVs) are being employed to deal with the improvised explosive device (IED) threat. Unfortunately, most of the systems in use require the attention of at least one operator for each vehicle. The potential combat leverage available via unmanned and robotic systems will never be realized until the ratio of operators to unmanned systems is reduced. Although the use of robotic “swarms” in military operations has been advocated widely, they are unrealistic under current operating protocols. The Army’s Future Combat Systems include, conceptually at least, multiple types of robotic components; their leverage could be increased significantly via one-to-many control. Achieving the one-to-many control paradigm through technological superiority is one key approach to countering the higher leverage obtained by potential enemies employing an asymmetric approach to engaging U.S. forces.

During the last biennial assessment cycle, the Board reviewed, at the request of the ARL Director, robotics work being performed by multiple ARL directorates and their partners in the Robotics Collaborative Technology Alliance. The Board found that the ARL robotics work was of high quality. The issues of unmanned vehicle supervisory control and multiplatform coordination were discussed briefly, but most of the review (and indeed most of the work) focused on robotic perception, mission packages, and supervisory control of single platforms.

Although these areas are extremely important for future Army utilization of autonomous systems, the time is ripe to consider an ARL enterprise development effort dealing with the autonomous coordination

of multiple robotic systems (i.e., exchange of information and autonomous actions taken by the robotic systems without human intervention to further the mission of the federation), and addressing the supervisory control of groups of UAVs by smaller groups of human operators. Building on a strong foundation of research on individual robots, ARL can also bring to bear significant talent from the Human Research and Engineering Directorate (HRED) to address human-system integration, and from the Sensors and Electron Devices Directorate (SEDD) and CISD to address platform-to-platform communications and mutual awareness, and can potentially leverage ad hoc networking based on Blue Radio to address a flexible platform-to-platform and platform-to-base communications architecture.

A key advantage of this approach is that it will position ARL to serve Army needs in a large variety of contexts and concepts of operations, regardless of the specific properties of the individual unmanned systems that are ultimately developed or acquired by the Army. Commercial developers are outpacing Army in-house efforts in individual robotic systems, whereas multiplatform coordination and supervisory control of large numbers of robotic systems represent new frontiers that for now are relevant mostly to military rather than commercial users. Strategic application of resources to these problems can position ARL to enable the Army to excel in the operational use of unmanned and robotic systems.

One of the most important new technology processes to emerge over the past few years is “information fusion,” or “knowledge discovery,” whereby disparate pieces of data are combined to yield higher-level knowledge, or information, that becomes “actionable intelligence” when presented in a sufficiently concise form and at the right time. In the commercial arena such technologies have been the drivers behind Web searching and data mining, for example. It also has been recognized in a variety of recent forums to be of increasing military importance, from a 2005 report of the National Research Council (Getting Up to Speed: The Future of Supercomputing³) to the 2006 Joint U.S. Defense Science Board and U.K. Defense Scientific Advisory Council Task Force on Defense-Critical Technologies.

Within the Army in recent years there have been a variety of efforts to kick-start information fusion efforts in support of the soldier, such as Horizontal Fusion and Warrior’s Edge. In many cases, such efforts have focused too soon on general overall architectures before development of a deep understanding of the problems for which such “information” can make a difference, what sensors are available to provide the raw data out of which knowledge can be fused, and how and where this fusion process can most efficiently take place. To make real progress in the near term, the Board encourages ARL to explore multidivision and multidirectorate efforts to select some manageable set of problems and to develop reasonably robust solutions for those problems that will help define the overall information fusion landscape and thus more general architectures.

The National Information Systems Security Glossary defines information systems security as “the protection of information systems against unauthorized access to or modification of information, whether in storage, processing or transit, and against the denial of service to authorized users or the provision of service to unauthorized users, including those measures necessary to detect, document, and counter such threats.” This is clearly of real concern today in the wired computer network arena, both military

and private, and of growing concern to the military as it moves to ad hoc networks formed from groups of warfighters. Therefore, the Board encourages ARL to consider developing crosscutting efforts in this area, especially in the establishment of test and “bake-off” facilities and organizations that help identify what the real problems faced by the Army are (both common and unique), and recognize when the best of commercially viable technologies provide some at least interim solutions. A similar approach has been taken in the machine translation arena, with extraordinary results for the warfighter.

Ad hoc networks are electronic networks in which the individual nodes attempting to communicate come in and out of contact with one another, can move dynamically (and thus affect which other nodes they may be in contact with), and may encounter environmental constraints (e.g., power, bandwidth, real time, security) not present in traditional networks. Such networks, particularly wireless ones, are beginning to permeate many of ARL’s projects, from sensor networks distributed over the battlefield, to dynamic intelligence networks aboard UAVs, to intra- and intersoldier networks. The Board encourages ARL to consider efforts to bring together these disparate groups so that fertilization of approaches, code, and subsystems can engender progress across the board.

Virtually all hardware systems for environmental measurements, communications, signal processing and analysis, and data display and application have become completely dependent on system-integrated computers during the past 20 or so years, essentially in conjunction with the development of advanced microprocessor technology. One of many fringe benefits of this approach is that it is now possible to predict computationally the performance of most parts of most advanced technology systems (e.g., remote sensors in which advanced inverse mathematical methods are required to extract the critical measurement data or adaptive communications systems that alter their mode of operation as environmental conditions change).

Coincident progress in the development and understanding of small-scale structures and phenomena in Earth, the atmosphere, and the oceans, including even many turbulent processes, is such that the output of a variety of computational fluid dynamics and atmospheric and oceanic models is quite adequate for defining the space-time behavior of signal propagation media, both for electromagnetic and acoustic signals. A continuing challenge for ARL is to ensure that appropriate verification and validation activities are applied to projects whose results rely heavily on models. The Board encourages ARL to carefully explore opportunities to exploit its high-performance computer and model resources for applications such as hardware prototyping, system predictive performance modeling, and verification and validation of multiscale analysis and forecast models. Progress in this area should reduce significantly the costs of system hardware and software development and testing. The Board also encourages ARL to consider ways of capturing the results of many of the field tests it performs every year relative to such phenomena so that the results can be searched at future times for answers to questions not yet asked today. When coupled with next-generation computing systems and with advances in information fusion and data mining as discussed earlier, such test sets may allow the creation of “virtual laboratories” in the future, and the quick response to unforeseen Army needs that may prove vital.

As previously noted, each panel normally reviews the work of a single ARL directorate. In 2005, at the request of the ARL Director, the Board undertook an additional review, forming a team that assessed the quantum computing and communications activities within ARL. The entirety of ARL’s activities in these areas falls within two programs: one in CISD and one in SEDD. Both programs had theoretical and experimental components.

The SEDD program in the security of optical quantum key distribution (QKD) systems is world-class. On the theoretical side, its principal investigator has an established track record in this area, publishes in respected peer-reviewed journals, and is widely esteemed in the quantum information community. This experimental program is rather new, but the support of DARPA, the Advanced Research and Development Activity (ARDA), and other agencies external to ARL is a clear indication of the importance of this work. In addition, given ARL’s extremely large extramural program in quantum information through ARO, it is very important that ARL have in-house researchers in the field who can act as advisers to this program.

QKD technology holds the potential for yielding important operational applications in the intelligence arena, once fundamental research problems are successfully addressed. ARL’s principal investigator in this area is in a position to be a leader in the field in this important research. The continued successes of this ARL program would help maintain Army input into QKD developments, and this would make it much more likely that the Army would benefit from the additional security provided if this promising technology reaches its potential.

The CISD program consisted of a set of theoretical examinations of computational algorithms and a laboratory that is well equipped for experimental work. The theoretical work, which lacked focus and integration, consisted of a set of dated work of unclear relevance addressing algorithm development, numerical solutions to nonlinear equations, and “quantum-inspired” algorithms. There were few peer-reviewed publications representing the program, there were no examples provided to indicate that the work has been implemented in application software, and it was not clear that the patent submissions applied for were timely or well received by the U.S. Patent and Trademark Office. The lack of outside investment and the dearth of peer-reviewed publications suggest that the work in this area may not be aligned with mainstream activity in quantum cryptography. However, the experimental facilities have considerable value, which could be exploited if there were tighter links to other researchers in the quantum computing and communications community. The Board encourages ARL to examine possible joint research with, for example, Multidisciplinary University Research Initiatives (MURIs) sponsored by ARO in such areas as quantum imaging, single proton sources, and detectors.

The Board is not charged to review the work funded by the Army Research Office, which is an organizational entity within ARL. ARO is a significant basic research asset with a significant fraction of the total ARL 6.1 budget. Considering the important role that basic research has played in the development of Army-relevant technologies and the similar high-payoff role that it could play in the future, the Board is interested in learning the ways in which the work portfolio of ARO is integrated into the activities normally reviewed by the Board. To understand more fully the ARL basic research enterprise and context, the Board suggests that in all future reviews by the Board, ARO be asked to summarize and present results of those 6.1 programs that it sponsors which are relevant to the ARL work being presented at the review.