much of the outcome space. 1 Of course, there are temporal limitations to this approach—even more so if some of the parameters are stochastic, requiring repeated runs to establish a distribution of results. Even without stochastic effects, problem dimensionality can explode as one acknowledges additional uncertainty. As a result, it becomes necessary to adopt a highly structured approach. For example, the field of statistical design of experiments with tools such as fractional factorial or Latin hypercube designs can substantially reduce the number of trials needed to identify important variables, significant combinations of variables, and the optimal combination of variables. 2

These design techniques are widely used in industrial applications (albeit applications with fewer uncertain variables than often occur in military problems) where the purpose is to determine the best combination of variables to optimize an industrial process. Generally, a fractional factorial design identifies a relatively small number of experiments to be run of a highly structured sort. Once the results from these runs have been obtained, some variables are identified as being important, and a new set of runs is determined. This process continues as long as time and resources permit. At the end, one obtains reliable information on the most significant variables or combinations of variables and their influence on the outcome. As computing power increases, the size and complexity of problems that can be explored in the fashion will also increase. Thus, statistical design of experiments holds the promise of being an approach to cope with uncertainties in complex models. Much progress has been made (Bos, et al., 1978; Davis, 1994), but much more work needs to be done to tailor these methods to problems of military relevance. 3

This approach represents a sharp departure from the long-standing legacy of using allegedly representative “point scenarios” and altogether ignoring major uncertainties (e.g., regarding the fighting capability, for constant equipment, of different nations' forces, or the “true” equation describing the movement rate of a division as a function of various combat variables).

Unfortunately, current M&S has not been designed with uncertainty analysis


RAND has done considerable work on this approach over the last decade, beginning with development of the RAND Strategy Assessment System (RSAS), which evolved into the JICM operational-level model, sponsored by OSD's Director of Net Assessment (see Davis and Winnefeld, 1983, pp. 62-65 for early visions). The original technology, however, was not yet powerful enough for what is becoming feasible now. For a broad and thorough description of exploratory modeling and analysis from a computer science perspective, see Bankes (1993, 1996). For applications to defense planning and adaptive planning involving global warming, see Davis et al. (1996) and Lempert et al. (1996) (a reprint from the journal article in Climatic Change, 33(2), 1996).


For practical discussion of such matters and citations to the literature on experimental design, see Committee on National Statistics (1995).


Alternative approaches or formulations are possible as well. For example, control theorists have focused on consequences of unmodeled dynamics. Dynamical systems focus on chaos as the explanation for apparent random behaviors. These are discussed in Appendix B .

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement