Modeling, simulation, and analysis (MS&A) is a crucial tool for military affairs. While DoD’s reliance on MS&A, in some form, dates back to World War I, changes in the enterprise of MS&A have not kept pace with the new demands arising from rapid changes in DoD processes and missions or with the rapid changes in the technology available to meet those demands. This report, commissioned by the Defense Modeling and Simulation Office1 and written by the Committee on Modeling and Simulation for Defense Transformation, identifies these shortcomings and suggests where and how they should be addressed. Although many other studies speak of modeling and simulation as a unit, the development of models without regard for planned subsequent analysis is a flawed process. Consequently, there is a need for examination and recommendations concerning all three: modeling, simulation, and analysis.
The charge to the committee, given in Chapter 1, has four tasks:
Evaluate the current capabilities of MS&A to support defense transformation.
Identify high-leverage opportunities for modeling and simulation (M&S) research.
Recommend approaches for framing existing and future MS&A products to decision makers and giving them the appropriate context and caveats, including evaluation of the risks.
Identify potential organizational and human resource development issues related to these themes.
The committee regards these tasks as a charge to recommend steps DoD must take to improve its entire MS&A infrastructure: the science base of MS&A; the execution of MS&A, including the interface of its practitioners with decision makers who may not be experts in the field; and the education and training of DoD MS&A practitioners. These are the subjects of Chapters 3, 4, and 5. The recommendations in this report are of two types: those pointing to promising areas for investigation and those pointing to procedural changes or suggestions for professional practice. The first type of recommendation involves research, where progress is unpredictable and, to some extent, a function of the resources applied. The second type requires commitment of persons or organizations in authority in DoD, and suggested commitments are noted in the text. The committee believes a holistic approach is needed to improve the MS&A enterprise, addressing all of these aspects of infrastructure simultaneously. In Chapter 6, it identifies the M&S communities at DoD that should be concerned with its separate recommendations.
Chapter 2 examines the changing mission of DoD and the changing environment in which it has been called on to operate. In addition to preparing for traditional warfare, DoD also must prepare for irregular conflicts, police actions, stabilization and reconstruction missions, and other nontraditional activities. This preparation must be done in a strategic environment that lacks the clarity of the cold war. Decision making at DoD increasingly considers the entire range of diplomatic, intelligence, and economic options, as well as military force—the so-called “DIME space.” At the same time, new technology has emerged. For instance, the autonomous operation of separate military units has been replaced by military systems that are increasingly interconnected and interdependent. The growing prevalence of embedded software, which often includes models and simulations, presents new challenges. Networks are now ubiquitous, and their use has become both an essential tool in the development of military strategy and an essential element of the strategy itself. Understanding the behavior of networks as they grow more complex is a difficult scientific and technical challenge. The combined operation of unmanned and manned systems may give rise to unforeseen and unmodeled emergent behavior. Many key aspects of this changing military landscape are
not well understood, and existing knowledge about phenomena such as terrorism or counterinsurgency has not been fully codified into models. Accordingly, models and analysis of networks and of complex adaptive systems, along with the use of embedded M&S, are central drivers for the development of future MS&A tools. The way that the defense MS&A enterprise should respond to these developments— the emergence of networks, adaptive systems, and embedded systems—forms the central theme of this report.
Chapter 3 identifies high-leverage opportunities for MS&A research needed to address the expanded mission space. Such research would expand the science foundation of MS&A. Building on a list of the new capabilities needed for the MS&A enterprise, the chapter discusses promising approaches for obtaining them, emphasizing the mathematical, scientific, and computational advances that have not yet been sufficiently incorporated into modeling technology and new topics where research can have a disproportionately large payoff.
The committee concluded that four main objectives should guide DoD’s MS&A efforts, and it developed the following four general recommendations:
Recommendation 1: DoD should give priority to developing flexible, adaptive, and robust MS&A methods for evaluating military strategies.
Recommendation 2: DoD should ensure that the basic architecture of its MS&A systems reflects modern concepts of network-centric warfare.
Recommendation 3: DoD should give special emphasis to the development of MS&A capabilities that are needed within embedded systems.
Recommendation 4: DoD should establish a comprehensive and systematic approach for developing the MS&A capabilities to represent network-centric operations:
Enhance and sustain collaborations among the various parties developing network-centric MS&A capabilities.
Continue and extend the development of existing approaches to modeling network-centric operation.
Establish a new mathematical basis for models describing network-centric operation, drawing on an array of approaches, particularly complex, adaptive systems research.
The diversity of challenges facing the DoD MS&A community requires a diversity of mathematical and modeling approaches, and the committee recommends that this be embraced as a guiding principle.
Recommendation 5: DoD’s analytical organizations should take a portfolio approach to designing their analyses and supporting research, investing in a range of methods including diverse models, games, field experiments, and other ways to obtain information.
In Chapter 3, the committee goes on to develop four more recommendations related to particular technical directions identified in the chapter:
Recommendation 6: DoD should devote significant research to social behavioral networks and multiagent systems because both are promising approaches to the difficult modeling challenges it faces.
Recommendation 7: DoD should form a research center or consortium focused on game-based training and simulation.
This report highlights the ubiquity of network-centricity for future DoD operations, but there is little scientific foundation for DoD networks or for those that appear elsewhere throughout our society. The committee believes that such foundational research would redound to the benefit of DoD.
Recommendation 8: DoD should support and extend initiatives to cooperate with other agencies funding research on networks.
Recommendation 9: DoD should begin cooperative programs of research into embedded systems with other agencies facing similar demands.
In Chapter 4 the committee presents approaches for improving the interface between MS&A practitioners and decision makers and makes two related recommendations. This interface presents a perennial challenge, not just in DoD, but in every situation where the results of sophisticated and nuanced MS&A must be condensed and conveyed, at appropriate but variable levels of detail, to end users. As DoD plans for its expanded mission space in the face of increased uncertainties, it is critical that the best MS&A be used and used effectively, because novel missions and increased uncertainties lessen the ability of decision makers to rely solely on their collected experience and judgment. Good professional practices are identified for translating a decision problem into an MS&A study that will assist in making that decision and for facilitating appropriate interaction with the decision maker. Emphasis is put on the communication skills needed to frame results accurately, including representation of the many levels of uncertainty inherent in any such solution.
There is a fundamental need for better understanding of the cognitive styles of decision makers and their interaction with different forms of MS&A. For instance, much is unknown about how information is absorbed and what biases might be introduced by alternative means of presentation. Research into the different cognitive styles of decision mak-
ers could help the MS&A community understand these styles and improve the practice of decision making by using MS&A. The committee has developed the following two recommendations to address this concern:
Recommendation 10: DoD should strive to better understand the cognitive styles of decision makers and their interaction with different forms of MS&A. Research into decision-making styles would improve decision making with MS&A by affording intuition and insight into complex problems and enhancing the creativity employed in their solution.
DoD’s expanded mission space includes more and greater uncertainties than conventional warfare. Because of the greater underlying uncertainty of the phenomena we now need to model and analyze, it is increasingly important to aim for MS&A methods that are able to cope with that uncertainty:
Recommendation 11: DoD should seek better methods to characterize, quantify, and manage the uncertainty inherent in all aspects of MS&A—including inputs, modeling assumptions, parameters, and options.
Chapter 5 discusses the training and continuing education of MS&A practitioners. The new defense challenges call for more widespread understanding of new MS&A methods and, in general, a high standard of professionalism across a broader swath of MS&A people. This high level of professionalism will give decision makers more confidence in the ability of MS&A people to contribute meaningfully to very difficult decisions that should not be based on incomplete experience. A core curriculum is proposed, the variety of skills needed by an MS&A team are identified, and ways to ensure that both military and civilian MS&A practitioners remain current are suggested. In addition, the committee points out that the talents required of MS&A practitioners and the roles they play in the process may change over the course of an MS&A study. These discussions led to two recommendations:
Recommendation 12: DoD must give its MS&A practitioners some exposure to all the topics in the core curriculum: the MS&A life cycle; continuous and discrete simulation; probability and statistics; topics in computing; deterministic modeling and optimization; MS&A evaluation; human-simulation interaction; modeling humans; and managing MS&A. Familiarity with these topics is essential for all practitioners, although the depth of knowledge needed will depend on a practitioner’s particular role.
Recommendation 13: DoD should ensure that its educational programs provide experiences in which students integrate the activities of modeling, simulation, analysis, evaluation, and communication to address real-world problems of importance to the consumers of the information.
Finally, in Chapter 6, the committee examines the need for coordinated research in the base of military science to support MS&A:
Recommendation 14: DoD should identify (or create) and charge an organization with responsibility for developing and supporting a program of research and development directed at improving and updating the base of military science for combat and noncombat modeling. That same organization would be responsible for effecting the recommendations on education that are called for in Chapter 5.
The committee discussed the need for research in numerous promising areas but has chosen to recommend only five: social behavioral networks, game-based training and simulation, cognitive decision making, network science, and embedded systems. It further recommends that the last two areas be supported jointly with other agencies that have a common interest in them. The committee regards these five as the most important in the presence of constrained resources. Having recognized the need for an office to coordinate improvements to DoD MS&A, the committee concludes Chapter 6 by suggesting which DoD MS&A communities should be responsible for implementing its recommendations.