5
Survivability/Lethality Analysis Directorate
The Survivability/Lethality Analysis Directorate (SLAD) was reviewed by the Panel on Survivability and Lethality Analysis during January 25-26, 2011, and August 3-5, 2011, at the White Sands Missile Range in New Mexico, and during May 29-June 1, 2012, at the Aberdeen Proving Ground, Maryland.
SLAD describes itself as follows:
SLAD is the premier source of expertise in survivability, lethality, and vulnerability (SLV) assessments, for senior leaders, developers, and evaluators, helping to ensure that U.S. personnel and equipment survive and function effectively in hostile circumstances. SLAD conducts analytical investigations; modeling and simulations; and laboratory and field experiments to provide its analyses as well as technical advice, and it strives to be a subject-matter expert on survivability and lethality matters to program executive officers (PEOs), program managers (PMs), users, testers, the Army’s independent evaluator, and other customers. SLAD produces products intended to reveal critical survivability and lethality issues for Army milestone decisions. In order to best serve the Army’s analytical needs, SLAD leverages both research and engineering conducted by other Army research, development, and engineering centers as well as other services, and SLAD’s work is leveraged by others.1
SLAD consists of two divisions: the Information and Electronic Protection Division and the Ballistics and Nuclear, Biological, and Chemical Division. The SLAD portfolio consists of many small test and evaluation (T&E) programs directly related to assessment of specific Army components, with several larger programs aimed at developing tools necessary to enhance test and evaluation efforts and broaden the scope of SLAD’s contribution to the life-cycle assessment of Army systems. During its 2-year project review cycle, the panel reviews a representative portfolio of SLAD work at a level sufficient to assess
___________________________
1Army Research Laboratory website, http://www.arl.army.mil/www/default.cfm?page=33, accessed February 1, 2013.
in depth the technical quality of its work. This chapter focuses on the programs for which adequate depth of detail was provided and where strong program continuity of effort was demonstrated. It also discusses a set of new and mature programs, in order to provide the context of the full spectra of work performed within SLAD.
CHANGES SINCE THE PREVIOUS REVIEW
Toward the beginning of this review cycle, the criteria applied to the assessment were amended in recognition of the programmatic emphases within SLAD on applied work such as test and evaluation, analysis, and tool development to enhance analysis capabilities. The previous criterion that emphasized the quality of scientific research was therefore replaced with a criterion that emphasizes the quality of the analytical work and of the efforts to support, extend, verify, and validate analytical tools. Joint Vision 2020 is the Department of Defense (DoD) vision that defines how the various elements of the DoD, including the Army forces, will operate in global conflicts as a single, integrated warfighting entity.2 Coupled with this vision is recognition of the constantly evolving threat that requires U.S. military forces to adapt and respond more rapidly with modified tactics, technologies, and or equipment than traditional DoD doctrinal requirements and acquisition processes provide for. This vision defines the need for the development of analysis tools that focus on quickly getting the best equipment to soldiers in the field. SLAD has demonstrated an exemplary capability for rapid and innovative response to solving problems with deployed or soon-to-be-deployed equipment. SLAD extends this rapid, innovative response to include analyses that address the survivability and lethality of integrated systems of equipment and humans, and it creates the partnerships that will enable it to impact system development in the conceptual planning stages.
SLAD increasingly recognizes the need to take advantage of opportunities to do the following:
• Extend attention to survivability and lethality of electronic weapons; electromagnetic pulse devices; directed energy; nuclear, chemical-biological, biomedical, and information warfare; and other weapon systems;
• Upgrade SLAD capabilities in wide-area security; and
• Proactively establish SLAD’s value at an earlier stage in the development cycle of Army materiel programs.
Following the demise of the Future Combat System (FCS), the emphasis of Army acquisition shifted to ongoing campaigns in Iraq and Afghanistan. Opportunities for heavily armored mounted warfare dwindled, and the challenges confronting infantry and other soldiers mounted in lightly armored vehicles or on foot were accorded the highest priority. While continuing development of its combined arms model, System-of-Systems Survivability Simulation (S4), and other research tools, SLAD also extended its research and development focus beyond the survivability aspects of equipment to that of the soldier. This has resulted in improved knowledge and capabilities. SLAD has enlarged its initiatives in human vulnerability assessment with multiple programs and expanding collaborations. The human availability technique methodology to analyze the combined performance of materiel and humans is a new initiative that examines the degradation of the performance of the whole materiel system while operating under combat conditions. This effort is a good example of increased collaboration between SLAD and the
___________________________
2Department of Defense. Joint Vision 2020. Washington, D.C.: U.S. Government Printing Office. May 3, 2000. Available at http://www.fs.fed.us/fire/doctrine/genesis_and_evolution/source_materials/joint_vision_2020.pdf (accessed October 3, 2012).
Human Research and Engineering Directorate (HRED) to expand the capabilities of SLAD’s assessment tools. A timely new program to develop a metric to predict mild traumatic brain injury represents a change from SLAD’s past focus on materiel. These two programs, combined with many others focused on assessing behind-armor, blunt trauma, and injury due to under-body blast, demonstrate the SLAD commitment to ensuring soldier survivability.
ACCOMPLISHMENTS AND ADVANCEMENTS
Collaborations in all SLAD program areas continue to improve, and the contribution to quality of work and dissemination of results was evident. Collaborations in SLAD’s core competency areas are moving toward excellence, and their continued expansion is encouraged.
The collaborative efforts between SLAD and HRED build on the expertise of each organization to rapidly develop a new anthropomorphic test device (WIAMan) for assessing injury from underbody blast (UBB). The program is well designed and has already provided excellent insights. One challenge is to ensure that the characteristics of the device make it applicable to females as well as males. Collaborations with academic partners for the same UBB injury assessment program area identified a work-around for critical issues involving limited distribution of live-fire test data. SLAD researchers have partnered with academia to get better data by using cadavers instead of anthropomorphic test devices (ATDs). By comparing the relative response between ATDs and cadavers, SLAD now has good data obtained under controlled conditions against which to benchmark WIAMan. This is an excellent example of the use of collaboration with organizations rich in experience in synergistic technology areas, leveraging SLAD expertise to expand their capabilities.
The program to develop a multispectral method for evaluation of an electro-optical/infrared sensor is another example of good collaborations with various government agencies. For example, the U.S. Army Aviation and Missile Research, Development, and Engineering Center (AMRDEC) is responsible for the battlefield integration of the selected sensor, and SLAD provides field measurements and multispectral sensor results. It was found that no single wavelength would provide sufficient detection of all possible threats, and so a multispectral method was developed. Collaborations seem well aimed to take advantage of other groups’ expertise to reduce the size of the multispectral sensor and to collect additional data for developing empirical models. This program demonstrates a good mix of empirical work and analysis.
SLAD has partnered with the Computational and Information Sciences Directorate (CISD) to expand the capabilities of its S4 simulation tool to include the Wireless Emulation Laboratory model developed by CISD to emulate a mobile, ad hoc network. This collaboration could enable S4 to include real-time modeling of mobile network systems. This program is still quite young, and few results are available, but SLAD has the beginnings of a sound project plan and is partnering and collaborating with other services (Naval Research Laboratory, Air Force Research Laboratory, and Air Force Office of Scientific Research).
In several programs, innovation resulted in significant contributions to the advancement of methodology and the development of standards.
Historically, operational data have displayed a wide variability that is believed to result from variability of the experience of the data collectors. Live-fire experiments were designed to provide results that would coincide with operational information, thereby providing a controlled experimental basis for more consistent interpretation of operational data. By reinterpreting the operational data based on live-fire results, a more accurate estimate of vulnerability was obtained. These efforts demonstrate the excellent use of analysis to inform interpretation of empirical data, which can then be used to inform
model analysis. This method should lead to better predictions of vehicle performance after underbody explosions.
An innovative new program takes advantage of available smart phone technology to expedite collection of more accurate field data in a consistent manner, resulting in increased data fidelity for analysis. This project is not likely to increase equipment in the field, because it is just a software addition to existing devices. The challenge lies in addressing the issue of data classification.
A new program to develop a metric to predict mild traumatic brain injury resulted in design of a surrogate sensor system that mimics the effect on the human brain of blast and blunt trauma. The program also proposed instrumentation for measuring the severity of the injury received. A sensor was designed and is now being developed. The next step is finding a means to allow the sensor to be more specific to the level of brain trauma. A process to be used in the field for rapid determination of the level of trauma has been proposed as well. This would allow the start of intervention that should reduce the long-term trauma effects. The approach is novel and thorough, and involves extensive collaboration with relevant forums.
Digital radio frequency monitoring is an innovative approach to expand utilization of a promising technology to analyze waveforms other than radar. The work is highly relevant, and the program has good engagement with the tactical community.
The Integrated Network Vulnerability Assessment Discovery Exploitation (INVA/DE) is an excellent tool for detecting the vulnerability of computer network operations. Further collaborations with CISD to pit SLAD’s INVA/DE against CISD’s Interrogator will test the robustness of each program. This impressive program seems very well tied into the intelligence community.
A core competence of SLAD is that of assessing the lethality and vulnerability of various weapon systems proposed for combat vehicles and the vulnerability of different design variants or system alternatives. The lethality and vulnerability analysis for the Ground Combat Vehicle program is an example of the tasks that SLAD routinely and rapidly accomplish for program managers. Its ballistic analysis of a lightweight vehicle-protection system is another excellent example of the strength of SLAD’s core competence that demonstrates the value of SLAD analysis and testing to support Army acquisitions; SLAD involvement should be inserted early into the acquisition process when requirements are first being determined. The active protection systems (APS) program shows that SLAD has contributed to an area of work from which it had been largely excluded by Army program management and organizational assignments. SLAD personnel provided new results on the vulnerability of light-armored vehicles to the residual threat of rocket-propelled grenades, which can have a tremendous impact on future design and fielding of better APS systems.
SLAD is home to several state-of-the-art test and analysis facilities. At the Electromagnetic Vulnerability Assessment Facility (EMVAF) at White Sands, experiments are conducted to characterize the effects of electromagnetic signatures on vehicles and other equipment performance. SLAD has been working in this area for a long time, and the SLAD researchers are recognized as experts in the field by the technical community. This facility supports a large cross-section of collaborators within the federal government. SLAD recently made a good decision to hire a global positioning system (GPS) expert to develop programs to study GPS jamming. The group working at the EMVAF consistently does good work and proactively pursues appropriate partnerships and collaborations.
SLAD still suffers from an aging workforce and faces challenges in building and maintaining a strong, fresh, knowledgeable workforce. It is imperative that SLAD document its tribal knowledge
before its senior personnel retire. SLAD would greatly benefit from a strong professional development program that is rich in training opportunities and participation in workshops, conferences, and continuing education (e.g., modeling and simulation workshops). SLAD would also benefit from hosting conferences and workshops in its areas of expertise. Although it has demonstrated improved knowledge of the literature and more collaboration, SLAD is still not a leader in most of the relevant technical areas. Active leadership in the greater technical community would serve to enhance SLAD’s reputation both within and outside the Army and its ability to recruit bright scientists and engineers.
Other possibilities for professional development exist within current SLAD programs. For example, the program on developing a metric to predict mild traumatic brain injury represents a new line of endeavor for SLAD; it may not culminate in a fielded metric, but it adds to the scientific knowledge base. The program is impressive in terms of its collaboration and participation in the broader research community.
SLAD traditionally has been successful in shifting emphasis as appropriate to address the changing demands and requirements of technology and DoD strategy. Information operations and cyberwarfare are examples of recent changes that significantly affect SLAD. SLAD has extended its portfolio to include assessment of vulnerability of communications, networks, and information processing on the battlefield. Current and future focus will have to include more modeling and simulation applications, integrated with physical testing and evaluation, to include more predictive capabilities and risk analyses in these areas.
SLAD continues to extend its suite of analytical tools by building on successful collaborations and by capitalizing on the success of customer-driven, short-turn-around-time assessment programs to identify technical gaps that would benefit from enhanced tool development. The overarching umbrella for tool development is SLAD’s system-of-systems analysis (SoSA) tool, S4. Most of the tools being developed within SLAD can be used as stand-alone tools, but most are also slated for use within the model library of the S4 code. The many challenges that surround this plan of action are discussed separately at the end of this chapter. This section discusses the challenges and opportunities associated with enhancing individual programs and tools.
In certain programs, such as the one examining lethality and vulnerability for the ground Combat Vehicle Program, SLAD estimates and assesses the impact of changes against known and projected threats. SLAD’s record in meeting such challenges is good. The challenges include delivering results as quickly as possible. Any improvements in responsiveness would be appreciated by developers and decision makers.
The question of whether to buy or develop software is often faced by SLAD analysts. In assessing the vulnerability of various electronic weapons, the challenge is to come up with a code to convert from commercially supplied solid geometry models (supplied by contractors) directly into SLAD vulnerability models to achieve radio frequency fidelity commensurate with model requirements. SLAD should examine the commercially available software for providing this conversion.
MUVES 3 is the next-generation ballistic vulnerability/lethality model software code developed to replace the well-used MUVES-S-2 code. MUVES-S-2 will eventually be retired, which will require that all users transition to use of the newer MUVES 3 code. The plan for transitioning from MUVES-S-2 to MUVES 3 does not appear to demonstrate involvement of current MUVES-S-2 users outside of SLAD but still within the Army—specifically the Army Materiel Systems Analysis Activity (AMSAA) and the Center for Army Analysis and the Training and Doctrine Command Analysis Center (TRAC). These organizations depend on data produced by MUVES-S-2 and need to be considered in the transition to MUVES 3. This transition program should be expanded to support all organizations that depend on MUVES-S-2 output.
The program to develop a metric to predict mild traumatic brain injury is a new program for SLAD and much remains to be done to establish the value, both in terms of medical and operational feasibility of the proposed brain injury metric. Consideration of cognitive effects should be made a clear part of the research plan. As the range of relevant research grows (for example, automotive research and sports medicine), it will be a challenge for the currently sized research team to keep apace of developments.
Knowledge of the pertinent literature is imperative for the success of any solid research program. SLAD presenters are providing more discussion of the relevant literature, but awareness of the literature remains lacking in several program areas. SLAD is to be commended for moving into the important area of assessing the combined performance of materiel and humans. Army analyses typically consider the performance of a vehicle and that of its operators independently. Such separate treatment often misses critical issues arising from interactions between them, especially if either is in a degraded state or a stressing environment. A broad literature on workload analysis and related work stress could conceivably relate to driving in combat operations. This work dates back to the early 1950s and considers vigilance, eye-hand coordination, physiological measures, and psychological stress levels during various environmental conditions. An extensive review of the previous pertinent work is needed in order to ensure an appropriately scoped niche for SLAD. SLAD should expand interdisciplinary efforts with other organizations to include human performance groups at appropriate universities and laboratories. For example, the design and development of the RAH-66 Comanche were addressed in a manner that integrated attention to humans and equipment, which included understanding of the man-machine interface, reduced pilot workload with single-pilot flight capability, crew cockpit ballistic protection, and crew cockpit crash worthiness protection.
SLAD is conducting a series of UBB projects aimed at refining and validating high-order models that assess UBB vulnerability, in order to develop reduced-order models that are quick-running and focused. Both the approach and SLAD expertise are appropriate. Emphasis should first be placed on completion of the high-resolution model and its validation. Although there is merit in having a methodology for strategically allocating capabilities for homing in on specific areas of concern, there is also risk in losing the ability to detect what could be important interactions. Caution should be applied in the experimental design of analyses using the reduced-order models. The value of being able to participate early in the development cycle (e.g., in requirements determination or concept evaluation, when data packages are high-level and low-resolution) suggests that the directorate might consider a reduced-order version of many of its models to identify unattainable requirements and filter out infeasible or unresponsive alternative concepts. The appropriate degree-of-order reduction will, of course, depend on the application. SLAD recognizes the value of early participation and is developing appropriate methodology.
The good work of SLAD analysts is rarely evinced at early phases in the design process of equipment and components. SLAD presented many examples in which its analyses made substantial contributions late in the acquisition process. The Army would benefit greatly if these significant contributions in equipment evaluation were made much earlier in the acquisition cycle. Examples of these programs include the following:
• Initial experiences in testing and assessing active protection systems for light-armored vehicles. This project’s goal is to determine performance of proposed systems. The project provides evidence of SLAD’s core contributions to the acquisition process, but it would have been even more effective if SLAD had been engaged earlier in the life of the acquisition program. This very good work reflected what SLAD ought to be doing, and it resulted in rapid deployment, which positions SLAD for involvement in the upgrades and design cycles of Army systems.
• Analyzing combat injuries to drive the development of body armor. This effort is an excellent demonstration of the benefit that SLAD provides to the Army and specifically how SLAD can improve the Army acquisition process through earlier involvement.
• Reducing vulnerability from damaged lithium-ion batteries. Results of this effort should be embedded in requirements for any new vehicles proposing the use of similar batteries. SLAD should capture and share data with both industry and other government agencies doing work in this area to expand the knowledge base.
SLAD is developing valuable insights and analytic capability in fields that are critical to modernizing the Army. Making prospective customers, including those outside of the Army, aware of SLAD capabilities will be important to establishing a role for SLAD that is analogous to its preeminent work in kinetic survivability. Making those in the Army acquisition process aware of SLAD capabilities will help to ensure that SLAD is established as an influential partner in equipment design and acquistion.
SLAD has many state-of-the-art test facilities that support its test and analysis capabilities. Expanding community awareness of these facilities will aid in building collaborations inside the Army, with other DoD laboratories, and with academia and industry. The Electromagnetic Vulnerability Assessment Facility (EMVAF) is a state-of-the-art facility in which leading-edge experimentation is being conducted (e.g., GPS jamming). Few Army laboratories currently perform this testing on dismounts, so SLAD should pursue greater visibility and collaboration for this effort. During discussion of the electronic warfare (EW) methodology development project, SLAD identified an anechoic chamber and a reverb chamber. No clear delineation was made between the capabilities of these chambers and other competing chambers in facilities outside the Army. SLAD should benchmark all of its facilities to show how they compare with other facilities for end-to-end testing. SLAD should consider showcasing all of its unique and state-of-the-art test facilities; methods might include hosting booths at DoD and other professional conferences and publishing papers that describe the research that SLAD is conducting in these facilities.
A facility to bench test and satisfy DoD T&E requirements for a system as large as a helicopter is expensive, but the Rotorcraft Survivability Assessment Facility allows the completion of live-fire testing with more precision and less risk than any other facility. The challenge will be to increase utilization and show the overall cost savings.
Many SLAD programs offer the opportunity for collaboration. Although SLAD has shown great improvement in the number of collaborations, there is room for expansion. Examples of areas ripe for collaborative efforts include the following:
• Computer network operations. SLAD needs to secure participation at the war games and or training environments (e.g., the EUCOM war-game in Stuttgart).
• INVA/DE. The team should establish a relationship with NETWARCOM.
• Optical augmentation (OA) and SLAD theoretical OA modeling. This program would benefit greatly from collaboration with a software developer/coder. SLAD could gain information by examining what the Air Force program in this area is doing.
• Lethality analysis for medium-caliber munitions in urban environments. The project demonstrated awareness of previous U.S. work in urban combat but did not uncover available data from work done by our Allies, specifically German efforts in military operations on urban terrain. An opportunity for additional data on wall performance and definitions may be available within the field of seismic research. Urban environments are very complex and can alter expected weapon effects. Because this project addresses a deficiency perceived within the DoD, identification of other ongoing efforts and collaborations within the DoD should be identified.
• Lethality and vulnerability analysis for the Ground Combat Vehicle Program. SLAD should expand collaborations with universities and industrial laboratory research programs that support vehicle manufacturers and crash testing programs. SLAD should capture relevant lessons learned from Iraq, Afghanistan, and other fields of action. In addition, SLAD should understand what JIEDDO and other rapid response organizations are doing to address related issues. This is an opportunity to position SLAD to influence the requirements for new combat ground vehicles.
OVERALL TECHNICAL QUALITY OF THE WORK
SLAD aims to provide sound assessment and evaluation support of the survivability, lethality, and vulnerability (SLV) of Army equipment and soldier systems to ensure that soldiers and systems can survive and function reliably in full-spectrum battlefield environments. Although SLAD is not as well staffed as it should be, its staff is competent, creative, and enthusiastic about SLAD’s mission. SLAD facilities include state-of-the-art laboratories and equipment that enable ground-breaking research and provide a strong basis for collaborations with outside partners. SLAD has many opportunities to capitalize on its test and evaluation base to selectively expand programs, to develop tools and methodologies to broaden its analysis capabilities, and to define and maintain the competitive edge essential to being the Army’s primary source for SLV assessment.
SLAD does an excellent job of assessing and researching tightly constrained instances of vulnerability and survivability. With each step that relaxes those constraints by expanding the organizational, temporal, and spatial envelopes addressed, SLAD’s ability to develop and apply appropriate methodology and analytic capability is increasingly taxed. It may now be overextended and thus less efficient and effective than necessary. SLAD should prioritize its development of modeling and analysis tools to ensure adequate allocation of resources to areas that it can most effectively support.
Physical testing is expensive and time consuming. Therefore, it makes sense for SLAD to construct high-resolution models of certain phenomena and, once they have been validated by physical testing, to use the models in place of subsequent physical tests. This work requires engineers, physicists, and analysts who are comfortable with the fundamental principles of the physics involved. In most cases SLAD demonstrates a solid understanding and sound application of scientific principles in developing engineering solutions to applied problems, but there is some evidence that not all SLAD personnel have a comfortable working knowledge of sound physics principles.
It is not necessary for SLAD to become involved primarily in basic scientific research to be a viable contributor to the Army and scientific communities. SLAD is uniquely positioned to observe emerging Army needs because of its core competence in rapid response programs. SLAD uses information produced by many of these small, customer-driven programs to identify areas ripe for research and tool development. Although gap analyses and niche identification are improving, the priority of these fundamental aspects of program planning should be elevated. Clear identification of the gaps in relevant research is the first step toward ensuring that SLAD programs are well positioned to contribute in areas that take advantage of SLAD strengths and provide the context in which SLAD collaboration would benefit advancement of the state of the art.
One of the largest challenges that SLAD faces as it moves toward greater emphasis on modeling tool development is hiring a sufficient cadre of appropriate personnel. With about 45 percent of its personnel over the age of 50 and only 8 percent of its scientific and engineering workforce possessing Ph.D.’s, SLAD needs to bring on board new personnel and train current employees in new areas. SLAD should consider looking toward postdoctoral, sabbatical, internship, and visiting scholar programs to provide healthy intellectual input and invigoration. In the short term, SLAD should consider expanding
its postdoctoral program to attract personnel trained in needed areas and providing continuing education of SLAD personnel through broader collaborations in academia and other DoD institutions. One avenue for accomplishing these objectives might exist through the National Research Council postdoctoral program, of which ARL is a member. SLAD should encourage and incentivize more personnel to pursue advanced degrees (M.S., Ph.D., and post-degree sabbaticals) in relevant areas. Partnering with the Army Research Office to fund programs, including 6.1 research and multi-university research initiatives (MURIs), in areas of interest to SLAD would help in training and recruiting personnel. Establishing centers of excellence in universities could also help to provide relevant training and experience to personnel already working in SLAD.
To ensure recognition of SLAD as a leader in the research, test, and analysis community, its personnel should document their work in open literature publications. SLAD has begun to publish a few peer-reviewed publications, but publication should become a more standard practice at SLAD. Publication in scientific or technical peer-reviewed journals provides traditional independent evidence of the rigor, efficacy, currency, correctness, and technical quality of the work that would define SLAD as a leader in innovative test and analysis tool development methodology.
SYSTEM-OF-SYSTEMS SURVIVABILITY SIMULATION
During the past 8 to 10 years the the ARLTAB has identified many concerns regarding the development of System-of-Systems Survivability Simulation (S4) software code. The need for analyses of systems of systems is real, and elements of the tools and models that support such analyses are lacking—in particular, analyses of engineering and technology used to accomplish the generation, processing, and exchange of information that produces desired coherent behavior of units and formations. These analyses require varying levels of fidelity and detail; taken together, they need to represent the dynamics of combat between opposing forces, including movement, fires, consumables, command and control, and communications. Current combat models, for example COMBAT XXI or OneSAF, produce detailed traces of the combat process and dynamics on digital terrain. These traces include instantiations of tactics, techniques, and procedures as well as command and control and communications. In general they do not provide high-fidelity representations of physical processes, such as path losses or protocols used in network communication. Higher-fidelity representations are needed to properly carry out systems analyses focused on identifying and responding to deficiencies in connectivity or capacity, dynamic management of bandwidth, or the performance of algorithms that process and act on sensed data. However, that higher fidelity is not needed throughout the complete space for system-of-systems analysis; in fact, there is a tradeoff between fidelity and analytic efficiency.
SLAD can make contributions to system-of-systems analysis. Whereas past designs for survivability focused on improving armor recipes, designs now encompass passive and active armor and countermeasures, managed in real time by onboard information networks. In the near future, designs will include cooperative survivability among groups of vehicles Collection, processing, and exchange of information will be key processes, integrated to perform real-time decision making and employment of effectors. SLAD logically has responsibility for analyzing such systems of systems from the perspective of physics and engineering. In the case of ballistic vulnerability, SLAD was provided with the tactical context in which rounds were launched and impacted on a vehicle. So, too, in the case of emerging system-of-systems survivability suites SLAD should turn to agencies such as TRAC that focus on and excel at tactical and operational analysis and modeling for context and combat dynamics, including tactics, techniques, procedures, and operational employment. To be effective, SLAD should carefully bound its
role and should focus on competently extending SLAD’s technical, engineering, and scientific analyses within appropriate operational and organizational bounds.
Changes and Accomplishments
Changes and accomplishments noted during this review cycle include the following:
• SLAD improved the definition of scope for S4. The focus of S4 modeling applications has been reduced from brigade level to company level and below.
• SLAD recently instituted a formal software development process. This process includes:
— Betterarticulation and definition of the SLAD vision for system-of-systems analysis,
— Formulation of a verification and validation strategy and required documentation of results,
— Participation of the Los Alamos National Laboratory in the evaluation of S4,
— Completion of S4 proof-of-principle efforts for the Army Evaluation Center (AEC) and the Program Executive Office for Integration, and
— Plans by AEC to accredit the S4 model.
• SLAD expanded collaboration efforts. Collaborations include Stanford Research Institute, Sandia National Laboratory, Army Research Office, Air Force Research Laboratory, Air Force Office of Scientific Research, Naval Postgraduate School, and Army organizations both within and external to ARL (i.e., Army Test and Evaluation Command, Army Evaluation Center, CISD).
• SLAD reported that the Army Test and Evaluation Command has concluded that SLAD’s system-of-systems analyses provide insight beyond current modeling and simulation capabilities and address measures not answered in tests or other modeling and simulation applications.
• SLAD has trained software development staff in capability maturity model integration (CMMI) tenets and practices of process improvement. SLAD has also contracted qualified experts in CMMI to overlay its framework on SLAD’s systems and to help to formulate a plan for improving the rigor of its processes as necessary. However, the SLAD scope is limited to self-assessment, without going through formal certifications.
Opportunities and Challenges
SLAD should consider a tactical pause to review and more carefully define S4’s role and mission. This review should include consideration of collaborative efforts with DoD agencies responsible for tactical and operational issues as well as SLAD’s role in providing high-fidelity analysis and tools focused on the physics of new and emerging survivability designs. Although SLAD has made some improvements in S4, the software and SLAD’s application of it have exhibited flaws, and progress has not yet been significant enough to warrant endorsement of the changes as meaningful. As SLAD plans and implements a systematic software development effort, more substantive improvements are expected. In addition, SLAD should carefully reexamine its technical plan for collaborating with and supporting the efforts of other modeling activities within the DoD (such as TRAC), focus its system-of-systems analysis (SoSA) modeling on cases that demonstrate the usefulness of S4 to others, and align its resources with its plan.
SLAD should prepare a detailed flowchart to determine the structure and capabilities of the current instantiation of the model to enable a more structured, well-thought-out, long-term plan for where this program should go to support the SLAD mission and strategic plan. SLAD should modify its model and software development process to become compliant with software development standards employed elsewhere in the acquisition and analysis community, such as the Software Engineering Institute’s
capability maturity model integration and its associated certifications. To ensure that the team includes the necessary modeling skills and experience, SLAD should consider re-competing the contract for S4 model development. For the next review, SLAD should provide a description of a representative low-level function in ultimate scientific detail, including any parameters that should be estimated by subject-matter experts (SMEs) and the meaning of those parameters to the SMEs. A good candidate here would be the function of shooting at equipment or troops with artillery.
Process and Methodology
SLAD should clearly identify both near- and long-term goals and work with its customers to establish a well-defined set of success metrics against which to evaluate progress. If S4 is to become an effective element of SLAD, then needed is a concise plan that includes the rationale for the use of system-of-systems analysis and S4 in SLAD and how S4 will contribute to the SLAD mission. The modeling effort should include clear definition of the survivability questions to be addressed, verification and validation issues, and the degree of confidence in the output. The value added by the modeling effort should be through the integration of the information from the engineering models, testing, and evaluations within SLAD. If S4 is to become a major tool for SLAD, then the goal should be to gain a balanced competence and credibility that is comparable to the other functional groups within the directorate.
SLAD’s plan should include milestones for the development process, implementation, and validation. Equipment, software, and certifications should be established, and collaborating partnerships should be identified. The S4 team should strive to become recognized as an independent analysis and evaluation organization that plays an important role in assessing and evaluating the survivability and vulnerability of military weapon systems. It is also necessary to build partnering and networking relationships with other modeling and simulation organizations, with SLAD’s unique contribution being that of integrating information from other programs within SLAD and from other ARL organizations (e.g., the Sensors and Electron Devices Directorate) into the S4 modeling efforts.
SLAD requires a dynamic technical leader for SoSA and S4 who will assume clear ownership of all aspects of the SoSA and S4 program and will be responsible for guiding and directing its development in a direction that demonstrates benefit to SLAD analysts and the Army in general. SLAD should adopt a professional software and model development process for S4 including a development plan that clearly identifies the following:
• Key customers, critical requirements, and long-term and short-term goals;
• A specific approach for attaining the goals, to include
— Program structure, framework, and description;
— Detailed descriptions of individual models contained within the framework or to be added, and how this will be done; and
— Inputs, uncertainties for all, and their treatment;
• A plan for software quality assurance and model verification and validation, to include performance metrics; and
• Identification of resource requirements and how the program is to be leveraged.
Personnel
Most of SLAD’s core staff displays an impressive domain expertise. That expertise is not present in the S4 developers, who lack experience in system-of-systems analysis, development of large-scale
combat models, and software development. As detailed below, they are weak in the knowledge, skills, and attributes required to ensure success in this program. There is an adequate base of subject-matter expertise to design process models of sufficient detail in the areas of SLAD’s core mission, including the network operations examples described during the review. However, that level of expertise is not present in the S4 model development team, as evidenced by the lack of (1) knowledge of the current state-of-the-art in combat models; (2) a focus on temporal, spatial, and organizational scales sufficient for SLAD’s purpose; and (3) compliance with standard software development processes, including the development and documentation of requirements, the specification of an architecture and its description via flowcharts, the development of a data dictionary, and the production of users’ and programmers’ manuals.
SLAD does not appear to have personnel qualified to design and create software to support analysis of military operations, either maneuver unit warfare or wide area security. SLAD does not have personnel with the training, experience, and expertise to conduct system-of-systems analysis of military operations, to play a role in deciding what models, simulations, and tools are required. A competent staff of modeling and simulation personnel, active in the military operations research and systems analysis community, should be developed and expanded. SLAD should place a military operations researcher (OR) into a position of influence over the military mathematical modeling. Personnel development should focus on obtaining a strong operations research staff with special skills in modeling and simulation. To achieve that, it would be important to present SLAD as an exciting place to be for scientists and engineers, offering good work with a clear and important mission. It would also be important to make SLAD known and respected outside its home fields at Aberdeen and White Sands. Exhibiting SLAD’s work at conferences and other professional venues would also help to entice recent graduates to consider an ARL career.
To leverage software and modeling support, SLAD should build strong relationships with ARL and DoD modeling and simulation groups. SLAD should also expand the source and breadth of SMEs and access to more soldiers with operational experience. To establish a broader institutional system-of-systems awareness, SLAD should encourage and enable its personnel to attend, as observers, advanced warfighting experiments, simulation exercises, and command post exercises, the last of these at all echelons. In this way a foundation of experience could be built up over time and SLAD could become prepared to play its key role in collaborative system-of-systems analysis.
The current approach to S4 development is flawed. All of the S4 coders and modelers reside at the New Mexico State University Physical Sciences Laboratory (PSL); only analysts reside at SLAD. The current methodology for performing S4 operations involves SLAD analysts working with PSL to define a problem. PSL personnel then run the code and provide data to the analysts. To date, PSL has not documented its code. Furthermore, there is no program plan in place, no users’ manual, and no descriptive report with all of the details. Because of this approach, SLAD does not own the code, which is imperative for S4 success. If SLAD is unable to hire coders to assume ownership of the development and maintenance of the code, then the directorate should demand documentation and drive the direction of development through construction and enforcement of a detailed program plan.
Scope and Collaboration
To ensure that S4 will become a useful tool for the system-of-systems analyses that SLAD believes to be a requirement of SLAD’s mission space, it is important that the scope of the model be carefully chosen. The scope of engagements should be at the platoon and squad level, and certainly no larger than the company level, with interactions and dependencies on non-organic elements represented via exogenous events. It might be efficient for SLAD to consider S4 as composed of two components. The
first, for which SLAD would be responsible, would be high-fidelity representations focusing on physical survivability and lethality as well as the sensing and information networking on which those processes depend. The second component would be the use of dynamic traces of combat, generated by models such as those used by TRAC, to drive the SLAD component. It is worth noting that this approach would permit SLAD to focus on appropriate slices in time and space and would thereby produce efficiencies in analysis.
SLAD should collaborate with those developing and using COMBAT 21 and other models (TRAC, AMSAA, Center for Army Analysis, and others) to gain relevant operational knowledge and increased military credibility. SLAD staff need to understand what is done well, how it is done, where deficiencies exist, and what is operationally important. Failure to include these other important stakeholders will result in analysis with no buy-in from customers of the operational community.
One promising way ahead for the Army would be for SLAD to work with TRAC-WSMR to create a joint model for mutual use. SLAD would then be starting from a supported object-oriented model that would presumably have established military credibility and could provide the basis for achieving higher fidelity. This would reduce SLAD’s problems associated with adding modules with the increased fidelity that it believes necessary for its customers. This collaboration would allow TRAC to provide valid tactical information to SLAD and would allow SLAD to provide valid technical information to TRAC.
Other collaborators who should be considered include communities of practice and stakeholder groups (military, academia), of which the following are examples:
• The system-of-systems engineering community (for example, the Undersecretary of Defense for Acquisition, Technology, and Logistics has published the Systems Engineering Guide for Systems of Systems3).
• The modeling and simulation community and standards groups—for example, the Computer Generated Forces (conference), Simulation Interoperability Standards Organization, High Level Architecture, and the Synthetic Environment Data Representation standard group.
Specific Recommendations
• SLAD should consider a tactical pause to review and more carefully define the role and mission of S4.
• SLAD should plan and implement a systematic software development effort.
• SLAD should carefully reexamine its technical plan for collaborating with and supporting the efforts of other modeling activities within the DoD (such as TRAC), focus its system-of-systems analysis modeling on cases that demonstrate the usefulness of S4 to others, and align its resources with its plan.
• The S4 team should investigate integration of the ACQUIRE model to augment the S4 sensor models.
• The S4 team should develop a consolidated set of new communication features for S4 with input from the Army communications community.
• SLAD should have solid engagement with the intelligence community.
• SLAD should leverage the services of AMSAA for validation, verification, and assessment.
___________________________
3Systems Engineering Guide for Systems of Systems, Version 1.0, August 2008, Director, Systems and Software Engineering; Deputy Under Secretary of Defense (Acquisition and Technology); Office of the Under Secretary of Defense (Acquisition, Technology and Logistics). Available at http://www.acq.osd.mil/se/docs/SE-Guide-for-SoS.pdf (accessed October 3, 2012).
• SLAD should focus on physics processes at a high-fidelity level (e.g., packet-level resolution in communications).
• S4 should be structured to accept combat dynamics as an input from other Army combat models.
Other concerns included the approach to survivability analysis with respect to information operations, the understanding and application of agent-based modeling, the need for strong information assurance and uncertainty analysis, and the lack of connectivity with relevant technical communities.
Given the importance of the soldier and experience in theater over the past decade, concentration on the soldier may be the critical research area in the short term. It is clearly an area where SLAD already has expertise.
Technical Merit
Progress has begun toward establishing a formal software development methodology and in bringing the system-of-systems analysis program, including S4, more under SLAD control, but there is still a long way to go to establish SLAD as a credible participant in the system-of-systems analysis environment. Relative to the metrics of quality assessment, the program does not reflect a broad understanding of the underlying science; the qualifications of team members are inadequate for the task; the facilities are state-of-the-art; the analytical work does not reflect a sound understanding of Army requirements; and the mix of theory and computation is appropriate, but the theoretical basis is inadequate.
