Click for next page ( 78


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 77
5 Modeling and Simulation Research and Development Topics Recent advances in modeling and simulation (M&S) technologies make them increasingly appealing as a means of improving commercial manufacturing and defense acquisition. However, in order for these M&S technologies to support the desired applications in commercial manufacturing and defense acquisition, additional research and development (R&D) is needed. In its statement of task, the committee was asked to investigate emerging M&S technologies, assess ongoing efforts to develop them, and identify gaps that would have to be filled in order to make these emerging technologies a reality. The committee rephrased this task and sought to determine those M&S topics requiring R&D in order for M&S to be effectively used in commercial manufacturing and defense acquisition. The topics requiring R&D were identified by the committee on the basis of the expertise of its members and information obtained from expert briefings. In addition, the committee surveyed literature calling for M&S R&D. The committee grouped these topics into four broad categories: (1) modeling methods, (2) model integration, (3) model correctness, and (4) standards, which are discussed in the sections that follow. 77

OCR for page 77
78 MODELING AND SIMULATIONIN MANUFACTURING MODELING METHODS Lack of adequate modeling methods is one of the most serious shortfalls in using M&S (MORS, 2000~. In order to maximize the potential of M&S technologies for commercial manufacturing and defense acquisition, basic research must be undertaken to improve understanding of modeling methods and characteristics, including scalability, multiresolution modeling, agent-based modeling, semantic consistency, modeling complexity, fundamental limits of modeling and computation, and uncertainty. Scalability Scalability is the attribute of a system's architecture that pertains to the behavior and performance of the system as the size, complexity, and interdependence of its elements or applications increase. Difficulties in dealing with large-scale software systems are well documented (NRC, 2000~. Techniques that work for small systems often fail markedly when the scale is increased significantly. To be upwardly scalable, a system must assure consistency in both the functionality and the quality of the services it provides as the number of its users increases indefinitely. To scale by a million, an application's storage and processing capacity would have to be able to grow by a factor of 1 million just by adding more resources (NRC, 2000~. This implies that as a system expands or as performance demands increase, the underlying architecture must support the ability to reimplement the same functionality with more powerful or capable infrastructure, for example, replacing a single server with a high- performance server farm. Traditional modeling and simulation have focused on microlevel components rather than on macrolevel integration of these components. However, with the advent of large-scale systems such as extended enterprises and distributed mission training, it is necessary to develop approaches for designing scalable M&S system architectures, including process specifications, linguistic support, granularity, and levels of abstraction to support system architecture design. This effort includes modularization, interconnectivity, and integration platforms as well as the standardization of application programs, automatic installation of modules, and verification. Metrics for such designs include robustness, reliability, flexibility, and the ability of the system to adapt dynamically to changing conditions. Several levels of architectural scalability are illustrated in Figure 5-1.

OCR for page 77
RESEARCHAND DEVELOPMENT TOPICS _! am_ - ~] ~~ E~en~ ente~se into . . . . conn~i~,it~~ t ~ Node ~pa~y ~ Enterprise A~hits~re Level 1~ 1 ,, Data dun management ~:~ ~del~ng and Simulation ~Ye' Figure 5-1 Levels of architectural scalability. 79 | Appl0~n Dyer ~ I R*~-t~rm~ event ct,anne! _ - , ........ _ Node capability 13- IT- Infrastru~L`~e Level Current and foreseeable trends are to employ object-oriented technology to enable scalability attributes. In object-oriented terms, the scalability problem can be stated as designing a system with the appropriate interface definitions that allow the implementations behind the interfaces to be upgraded from single objects to multiple coordinated objects or to objects of more capable classes. Abstraction, modularity, and layering are the basis of such interface design concepts (Messerschmitt, 2000~. Scalability designs must live within existing resources in communications bandwidth and computing power available from the underlying computing and network technologies. A practical approach to scalability also requires consideration of interoperability in order to address the problems of data heterogeneity that are due to a lack of accepted standards and the current multiplicity of approaches (IMTI, 2000~. Multiresolution Modeling "Multiresolution modeling" and/or "multiresolution simulation" is defined as the representation of realworld systems at more than one level of resolution in a model or simulation, respectively, with the level of resolution dynamically variable to meet the needs of the situation. R&D into multiresolution modeling has been recommended (NRC, 19971. It is considered especially important for SEA because acquisition programs will

OCR for page 77
so MODELING AND SIMULATION IN MANUFACTURING need to move up and down the resolution hierarchy and use the proper level of models and simulations to support iterative trade-off analyses (Ewen et al., 2000~. In addition, multiresolution simulation has the potential to improve the scalability and flexibility of simulation applications. A related concept is "multiviewpoint simulation." In this case, simulation takes place at a single level of resolution, but the execution events and results are presented at different levels of resolution, or viewpoints, as appropriate to the needs of the user. Significant unresolved issues in implementing multiresolution models, however, account for the need for research in this area. A number of multiresolution simulations have been implemented (Stober et al., 1995; Franceschini and Mukherjee, 1999), but that work has approached the problem largely from an experimental and practical point of view. As yet, no complete and coherent theoretical framework exists for multiresolution models, although some work leading toward such a framework has been completed (Davis, 1993; Franceschini and Mukherjee, 1999~. Some problematic issues arise in multiresolution models, including maintaining consistency between levels of resolution when aggregation and disaggregation operations occur (Davis, 1993; Franceschini and Mukherjee, 1999), dealing with "chain" or "spreading" disaggregation (Petty, 1995), allowing interactions between objects at different levels of resolution, and preserving consistency during reengagements. Some work has been done on each of these issues, but more is required. In addition, multiresolution modeling affects the architecture of the simulations that use it by requiring the ability to dynamically change object and event resolution during run time; those architectural issues are also the subject of ongoing work. One architectural approach that may resolve some of the problematic modeling issues just listed is to develop families of models, rather than single models, at various levels of abstraction (resolution) (Davis, 1995; NRC 1997; Davis and Bigelow, 1998~. Distributed simulation systems are being developed to support interoperation of such model families (Davis, 2001~. Agent-Based Modeling Agent-based modeling is a modeling method based on the simulation of what are called low-level entities, such as individual people or aircraft, that have simple behaviors but that can produce complex and unexpectedly realistic collective, or emergent, behavior (Epstein and Axtell, 1996~. As discussed earlier, such modeling methods are an important area of research for supporting realistic simulation of complex systems-of-systems (NRC, 1997; Ewen et al., 2000~. A sampling of the open research issues in agent- based modeling includes achieving satisfactory run-time performance

OCR for page 77
RESEARCH AND DEVELOPMENT TOPICS 81 when simulating large numbers of agents, determining an adequate level of fidelity for individual agents' behavior, validating agent-based models (Balmann, 2000, Axtel] and Epstein, 1994), and avoiding ad hoc assumptions during model development (Cederman, 1997~. Semantic Consistency Semantic consistency, also known as substantive interoperability, refers to consistent phenomenological representations of real-world systems and processes among interacting distributed simulations. For example, two combat simulations must have consistent models of intervisibility or they will be unable to interoperate meaningfully in a distributed simulation (Dahmann et al., 1998~. Research into semantic consistency and a general mathematical language for expressing models are recommended (NRC, 1997~. Dealing with Complexity and Errors Abstraction is the process of extracting a relatively sparse set of entities and relationships from a complex reality to produce a valid simplification of that reality. Abstraction is a general process; it includes simplification approaches such as aggregation, omission of variables and interactions, linearization, replacing stochastic processes by deterministic ones (and conversely), and changing the formalism in which models are expressed (Zeigler et al., 2000~. The complexity of a model is measured in terms of the time and space required to execute it as a simulation. The more detail included in a model, the greater the resources required of the development team to build it and to execute it as a simulation once it is built. Validity is preserved through appropriate morphism mappings at desired levels of specification. Thus, abstraction methods, such as aggregation, will be framed in terms of their ability to reduce the complexity of a model while retaining its validity relative to the given modeling objectives. inevitable resource constraints require working with models at various levels of abstraction. As noted above, the complexity of a model depends on the level of detail, which in turn depends on the size/resolution product. The size/resolution product reflects the fact that increasing the size, or number of components, and resolution, or number of states per component, leads to increasing complexity (Zeigler et al., 2000~. Since complexity depends on the size/resolution product, complexity can be reduced by reducing the size of the model or its resolution or both. Given fixed resources and a model complexity that exceeds these resources, a

OCR for page 77
82 MODELING AND SIMULA TION IN ~NUFAC TURING trade-off must be made between size and resolution. If some aspects of a system are represented very accurately, only a few components will be representable. Alternatively, a comprehensive view of the entire system can be provided, but only at a low resolution. Several new approaches to modeling complexity are being developed. One of them is the notion of coordinated families of simulations at different levels of resolution, which was mentioned previously. This approach presupposes the existence of effective ways to develop and correlate the underlying abstractions. A second approach, exploratory analysis, attempts to overcome computational complexity by addressing the issue of optimization, or searching through large spaces of alternatives for best solutions to a problem (Davis and Hillestad, 20009. This approach uses low-resolution models with a wide scope intended to capture the main features of an overall system or scenario. The approach seeks to exploit the reduction in the large space of alternatives that low-resolution, or highly abstracted model structures, may provide. A third approach fundamentally reconsiders the issue of optimization as a search for the best among many alternatives. The fast, frugal, and accurate (FFA) perspective on real- world intelligence provides a framework for insight into this issue (Gigerenzer and Todd, 1999; Gigerenzer and Goldstein, 2000~. FFA is taken from the domain of human decision making in which full optimization is associated with unbounded rationality. This perspective recognizes that the real world is a threatening environment in which knowledge is limited, computational resources are bounded, and little time is available for sophisticated reasoning. Simple building blocks that steer attention to informative cues, terminate search processing, and make final decisions can be put together to form classes of heuristics that perform at least as well as more complex, information- hungry algorithms. Moreover, such FFA heuristics are more robust when generalizing to new data, since they require fewer parameters to identify. They are accurate because they exploit the way that information is structured in the particular environments in which they operate. FFAs are a different breed of heuristics. They are not optimization algorithms that have been modified to run under computational resource constraints, such as tree searches that are cut short when time or memory runs out. Typical FFA schemes exploit minimal knowledge, such as object recognition and other one-reason bases for choice making under time pressure, elimination models for categorization, and "satisficing" heuristics for sequential search. In his radical departure from conventional rational- agent formulations, Simon asserted the bounded rationality hypothesis, namely, that an agent's behavior is shaped by the structure of its task environment and its underlying computational abilities (Simon and Newell,

OCR for page 77
RESEARCH AND DEVELOPMENT TOPICS 83 1964~. Fast and frugal heuristics are mechanisms that a mind can execute under limited time and knowledge availability and that could possibly have arisen through evolution. One illustration of Simon's "satisficing" alternative to optimization is the "take the first best" inferencing heuristic, which employs only a fraction of available knowledge and stops immediately when the first, rather than the best, answer is found. "Take the first best" does not attempt to integrate all available information into its decision. It is noncompensatory and nonlinear and can violate transitivity, the canon of rational choice. Fundamental Limits of Modeling and Computation In order to satisfy the needs of M&S for increasingly complex systems and processes, an integration of the statistics-oriented approach to M&S research must be emphasized by the academic community and the computer-science-oriented approach to M&S research must be emphasized by DOD and industry in acquisition and manufacturing. The statistics- oriented approach deals with prediction and management of uncertainty, whereas the computer-science-oriented approach deals with interoperability, reusability, integration, distributed operation, and human/machine interfaces. The computer-science-oriented approach is necessary for the future operational success of defense acquisition and commercial manufacturing, but as processes and systems become increasingly complex, estimation and management of uncertainties will become increasingly important. Some fundamental limitations in computation in dealing with complex systems must be recognized. The performance of any future complex system will be unavoidably stated in probabilistic terms. A suite of software and a collection of databases may be technically interoperable and can be used to calculate system performance under a given set of operating environments, but there is no way that these tools can estimate the percentage of time that the system will perform satisfactorily under different circumstances, what the expected performance will be under uncertainty, or what the confidence level of the estimate is. In order to answer these questions, Monte Carlo experiments must be run on the system. Here, one runs up against fundamental limitations of performance simulation involving uncertainties. "There are fundamental limitations to improve the simulation speed due to fact that confidence interval of

OCR for page 77
84 MODELING AND SIMULA TION IN MANUFACTURING performance estimate decreases at best at the rate of 1/n~/2 where n denotes the length of simulation." This is a heavy computational burden that may become too much for complex systems. In addition, in order to improve the system performance estimate by adjusting or tuning various parameters in different phases of the acquisition process, dimensionality, or combinatorial explosion, must be dealt with. The search space of system design parameters is combinatorially huge. The first fundamental limitation in computation states that each system performance evaluation via simulation is time consuming. The second limitation states that a very large number of such evaluations may be necessary. These difficulties are multiplicative. Finally, there is a third limitation. "No Free Lunch Theorem": Without specific structural assumptions, there exists no optimization or search algorithm that can perform better on the average than blind search in dealing with the first and second limitation. (Ho, 1999, p. 8) These three limitations are fundamental limits on computation in dealing with complex systems. No amount of theoretical, hardware, or software advances can overcome them. Consequently, a strategic redirection is called for in dealing with them. Several emerging trends that directly or indirectly address the problem of system engineering of complex systems are outlined below. One or more of these topics may blossom into proven tools for dealing with the preceding difficulties and enable a more quantitative and optimizing approach. Ordinal Versus Cardinal Optimization Order is much easier to ascertain than value is. If one holds two identical-looking boxes in either hand, it is easy to determine which one is heavier, but much harder to determine how much heavier one is than the other. In many complex decision problems, it is often sufficient to be able to determine which solution is better, or to determine which is in the top 1 percent, rather than which is the absolute best. A theory of ordinal optimization is being developed that may enable quantitative measurements of such assertions via simulation modeling without having ' Y.C. Ho, Ordinal Optimization Teaching Module. Available at . Accessed June 2002.

OCR for page 77
RESEARCH AND DEVELOPMENT TOPICS 85 to confront the first and second fundamental limitations on computation of complex systems (NRC, 1999b). Off cient Search Via Learning Blind search in a large space is inefficient. Therefore, to deal with the large search spaces imposed by the second and third computational limitations discussed above, the structure of specific problems must be learned along the way. A number of automated learning theories currently in vogue in artificial intelligence research, such as knowledge discovery, data mining, Bayesian networks, and Tabu search, may be significant for developing M&S capabilities. Tabu search is a heuristic technique for search in combinatorial optimization problems (Grover, 1990~. Errors in Distributed Simulations Given fixed resources and a model complexity that exceeds these resources, a trade-off must be made between size and resolution. If some aspects of a system are represented very accurately, only a few components will be representable. Alternatively, a comprehensive view of the entire system can be provided, but only at a low resolution. Such resolution may introduce errors that may pose particular problems in distributed simulations. In such complex, networked systems of models, owing to low resolution each model will typically be in error to some degree. Therefore, it is natural to expect that in a complex system of many linked models, even if individual inaccuracies are small, such errors can accumulate, propagate, and reinforce each other, rendering the behavior of the aggregate significantly different from the behavior of the real system. Error propagation in distributed simulations plays an important role in verification, validation, and accreditation, and therefore is an important area of research that needs to be strengthened. In the current state of the art, it is possible to suggest that such error propagation may, or may not be, a significant issue in distributed simulations. On the one hand, modeling errors in complex systems can be like noises that are more or less statistically independent. The cumulative effect of many independent errors behave according to the central limit theorem and decrease with increasing complexity under some reasonable assumptions. A simple case is the law of large numbers, which improves accuracy by averaging many measurements. A second mitigating factor is the theory of ordinal optimization, mentioned above. Research here has shown that for the purpose of comparison (i.e., which is better?), very crude models are quite sufficient. Consider the metaphor of two bags of gold. You are free to choose the heavier bag. Every one of us can unfailingly tell the heavier

OCR for page 77
86 MODELING AND SIMULATION IN MANUFACTURING bag, even with small differences. But most of us will have difficulty if we are asked to estimate accurately the difference in weight between the two bags. "Value" is much harder to estimate than "order." In most cases of simulation optimization, we only need to know the order or be able to locate the top 1 percent of the design. It is not necessary to know the performance "value" accurately. Approximate simulation models are quite adequate for the former purpose. Once the top 1 percent have been located with high probability, we can lavish our attention and computing budget on this much smaller subset. A large volume of literature on the theory and success stories has been built up on this subject during the past decade (Ho and Cassandras, 2001~. On the other hand, it is known from work on numerical analysis, that numerical methods can introduce instabilities that greatly magnify errors even if the underlying models are stable. To obviate error-induced instabilities, criteria that enable choice of time-step size and other controllable factors are well known for nondistributed simulations. However, the major difference between distributed simulations and their nondistributed counterparts is that control and data are encoded in time- stamped messages that travel from one computer to another over a (bandwidth limited) network (Fujimoto, 2000a). Traditional analyses in the design of numerical methods consider trade-offs between accuracy and speed of computation (Isaacson and Keller, 1966~. However, since distributed messaging requires that continuous quantities be coded into discrete packets and sent discontinuously, it is more appropriate to consider discrete event simulation as a natural means to consider accuracy or bandwidth trade-offs. Recent work has shown that significant reductions of message bandwidth demands (number and size of messages) with controllable error and local computation costs are possible2 (Zeigler et al., 1999~. Finally, the issue of numerical stability in complex simulation is related to the problem of sample path continuity with respect to parameter and timing perturbation. Here again, literature exists (Ho and Cassandras, 1 997~. Theory of Complex Systems Complex systems, such as the national electric power-grid and worldwide communications networks, are vulnerable to attacks and catastrophic failures (Amin, 2000~. A theory of complex systems is emerging that may shed light on the fundamental nature of such complex interconnected systems, why and how they fail, and the limits to and 2 The interested reader may wish to consult Chapters 14 and 16 in Zeigler et al. (2000) for an extended discussion of error in modeling and distributed simulation.

OCR for page 77
RESEARCHAND DEVELOPMENT TOPICS 87 disadvantages of complexity (Ho and Pepyne, 2001). This is related to the problem of inferring total system performance from that of components. Any system is assembled or constructed from a set of components and/or suboperations. When broken down to the elemental constituent part, each part or suboperation can be modeled, and its performance measured, even if probabilistically. However, each part's contribution to the overall performance success or failureof the entire system is different. For example, in an unmanned combat air vehicle, the performance of the automatic target detection subsystem is more important than is the successful landing of the returning system. The former directly affects the success of the mission, while the latter may cause the destruction of an expendable system. There is need for an analysis technique for assessing the relative expected importance and contribution of each part or suboperation to the overall goal of a system engineering project as a function of network architecture and hierarchy. Such a tool would enable managers to measure the critical elements of a systems engineering project and direct resources at those parts more systematically and quantitatively. Uncertainty Uncertainty is becoming increasingly important in modeling and simulation. Characterization of uncertainty refers to methods for tracking and quantifying the propagation through a model's calculations of the uncertainty that is inevitably present in the attribute values and interactions of components within a simulation. Decision making under uncertainty refers to models that assist in evaluating uncertainty and risk in situations in which incomplete information is available. Exploratory analysis under uncertainty is a process of searching the space of possible simulation outcomes as a function of the many assumptions in a scenario in order to find and delimit interesting or dangerous outcome regimes (NRC, 1 997a,b). MODEL INTEGRATION The infrastructure for modeling consists of fools and capabilities that support the practice of modeling. This infrastructure must support model integration and interoperability in order for the M&S requirements of acquisition and manufacturing to be met. Important topics associated with model integration are interoperability, composability, integrating heterogeneous processes, and linking engineering with effectiveness simulations.

OCR for page 77
92 MODELING AND SIMULATIONIN MANUFACTURING Moorby, 1991~. It is easier to learn, but lacks constructs to support system- leve] design. Unfortunately, VHDL is limited in the representation of mixed analog and digital processing. A framework for the modeling and simulation of hybrid analog/digital systems has long been needed. Today, the need to design mixed-signal chips (MSCs) to support the growth in wireless devices and next-generation automotive electronics has brought this problem to the foreground. MSCs have been implemented as custom- application-specific integrated circuits (ASlCs), but must now be mass- produced for use in wireless technology. MSCs receive analog signals, process and manipulate them mainly in digital form, and reconvert them back to analog form. The challenge for systems design is the high level of functionality of an MSC. It contains radio frequency components, such as receivers, antennas, filters, and amplifiers; analog components, such as digital-to-analog converters, battery and power supplies, and interfaces to sensors; and digital components, such as digital signal processors, microcontrollers, microprocessor memory, analog-to-digital converters, and interface buses. Hybrid design has traditionally been tackled by mapping the input- output behavior through thresholding and interpolation. The fundamental difficulty is that, driven by the needs of accuracy and efficiency, the resolutions of time in the respective simulations for the analog and discrete subcomponents may be different. This translates into different units of time. While techniques such as lock-step, fixed time step, ping-pony, and Calaveras have been proposed in the literature, they are essentially arbitrary and lack a scientific basis to yield a common notion of time. The difficulty is aggravated when analog and discrete subsystems occur in feedback loops. Current efforts to solve this problem merely extend the previous methodology by standardizing the input-output signals for exchange between the simulations of analog and discrete subsystems. New approaches are needed (Ghosh and Grambasi, 20011. Linking Engineering and Effectiveness Simulations It is useful to distinguish between two broad classes of simulations. The first is product modeling or engineering simulations, which simulate the physics of products or systems being designed with a high degree of detail and physical fidelity. The intent of these simulations is to assist design engineers in understanding the physical performance of the product or system as designed. They often simulate only one system or subsystem at a time and run slower than read time. They can be loosely defined as using M&S to determine how to build a system. The second class of simulations is performance modeling or effectiveness simulations, which

OCR for page 77
RESEARCH AND DEVELOPMENT TOPICS simulate products or systems that are assumed to exist and operate as designed. The intent of these simulations is to determine how effective the systems would be in use, or what performance parameters the systems must have in order to be effective in use. They often simulate scenarios involving many simulated systems and run in real time or faster. They can be loosely defined as using M&S to determine which system to build. The ability to link these two types of simulations is necessary for achieving the goals of defense acquisition. The ability to reuse engineering models and simulations in effectiveness simulations would save time and money. 93 MODEL CORRECTNESS Model correctness is the fundamental requirement of ensuring that the predictions of a simulation model can be relied upon (Zeigler, 1998~. The vision of defense acquisition contained in SBA requires the development of accurate and reliable models of real-world systems. A prerequisite to this is an understanding of the real-world systems and objects to be modeled, their contextual domains, and the phenomenology of their operations and interactions, all at a level of detail sufficient to justify the model. Once the models have been implemented as simulations, their correctness must be rigorously evaluated. Domain Knowledge Improved understanding of the real-world basis for models is needed in the areas of phenomenology of warfare, physics-based modeling, and human behavior modeling. Phenomenology of Warfare The military domain is of special importance because it is the primary focus of SBA and because it is the domain in which human lives are most likely to be risked on the basis of decisions made using M&S. Lack of recent investment is not compensated for by previous investment because of the rapidly changing nature of military technology, doctrine, and operations. For example, models are lacking in such emerging areas as information operations and operations other than war. Effort is needed to develop deeper, more rigorous, and more quantitative understanding of the phenomenology of warfare, especially involving the complex, interconnected, and nonlinear military systems and systems-of-systems

OCR for page 77
94 MODELING AND SIMULA TION IN MANUFACTURING planned for the future. Relatively little recent investment has been made in understanding the phenomenology of military operations at the mission and operational levels (NRC, 1997a,b). Physics-based Modeling Mathematical models in which the equations that constitute the model are those used in physics to describe or define the physical phenomenon being modeled are referred to as physics-based models. For example, physics-based flight dynamics models use aerodynamics equations rather than look-up tables to model the flight characteristics of a simulated aircraft. The physics of failure and assessment of a potential system's durability and operational availability is of special interest. Such assessments would greatly benefit from accurate physical models that support predictions of the modes and times of failure of physical systems. Several studies have concluded the need for improvements in physics- based modeling (Johnson et al., 1998; Hollis and Patenaude, 1999; Starr, 1998~. Physics-based modeling is arguably more important for defense manufacturing and acquisition than for other simulation applications such as training. Human Behavior Modeling Computer-generated forces are often used in training simulations to provide both opposing forces and supplemental friendly forces for human participants in a simulation. They are also often used to generate all of the entities in battlefield simulations being used for nontraining purposes, such as analysis, experimentation, and SEA. Automated or semiautomated entities are created, and their behavior is controlled by the computer system, perhaps assisted by a human operator, rather than by human participants in a simulator (Kerr et al., 1997; Petty, 1995~. These automated behaviors are produced by algorithms based on models of human behavior. The reliability of the results depends on the validity of the behavior-generation methods. While current behavior-generation methods are reasonably effective at producing behavior that is in accordance with straightforward tactical doctrine, they fall far short of producing realistically human behavior with all its unpredictability and sophistication. Several studies have concluded that a need exists for improvement in human behavior modeling (Ewen et al., 2000; NRC, 1998b; Hoagland et al., 2000; Starr, 1998; Johnson et al., 19981.

OCR for page 77
RESEARCHAND DEVELOPMENT TOPICS Verification, Validation, and Accreditation 95 Verification is the process of determining that a model implementation, or simulation, accurately represents the developers' conceptual description and specifications. Validation is the process of determining the degree to which a model and associated data are an accurate representation of the real world, with respect to the model's intended use. Accreditation is the process of official certification that a model or simulation is acceptable for use for a specific purpose. Several studies have identified verification, validation, and accreditation as important topics for research and development (Johnson et al., 1998; Ewen et al., 2000; Hollenbach, 2000; SBATF, 1998~. A crucial step in the acquisition of a defense system is operational testing and evaluation, the final assessment of a system's effectiveness and suitability prior to fielding. Traditionally done using real-world testing of actua] systems, operational testing has seen a gradual increase in the use of M&S to reduce time and costs. This application of M&S requires extremely accurate simulations and consequently requires highly reliable validation methods. As M&S is used more in operational testing, the demands on the validation of the simulation will increase. Severa] advances in statistical methods are relevant to validation of simulations used for defense acquisition and may provide the basis for needed improvements in validation methods (NRC, 1998a). The limits of applicability of M&S to operational testing have been clearly asserted by the commanders of the services' operational testing organizations (Besa] et al., 2001~. Results generated by models and simulations used may be the basis of decisions affecting human safety or expending large sums of money. Validation methods that quantify the bounds of validity and risk of error in a mode] can help to establish the limits of M&S app]icabi]ity in operational test and evaluation. STANDARDS Standards are at the intersection of technical and nontechnical issues. The ways in which standards are developed are complex and often more successful if done from the ground up rather than from the top down. The M&S community has historically been resistant to setting standards. Because many M&S practitioners are self-taught or have had largely on- thejob training, there are many different methods of doing things. The variety of modeling methods is commensurate with the range of systems modeled.

OCR for page 77
96 MODELING AND SIMULA TION IN A1ANUFAC TURING Currently, a state-of-the-art, standardized external model representation is lacking. Moreover, modeling languages do not adequately support the structuring of large, complex models and the process of model evolution in general. The development and application of standards, however, are essential to the achievement of the level of interoperability, integration, and reuse envisioned for commercial manufacturing and defense acquisition. This section discusses existing modeling and simulation standards, general software standards, and higher-layer standards, and needs for their development and integration. Modeling and Simulation Standards Limited interoperability exists among the modeling and simulation environments available today. However, several standards are emerging that are aimed at solving interoperability and model construction problems. High Level Architecture . HLA is a general-purpose architecture for simulation reuse and interoperability. It was developed under the leadership of the Defense Modeling and Simulation Offices (DMSO) for the purpose of supporting reuse and interoperability across the many different types of simulations developed and maintained by DOD. In 1996, HLA was approved as the standard technical architecture for all DOD simulations, and in 2000, it was approved as an open standard by the Institute for Electrical and Electronic Engineers (IEEE). DMSO sponsored the establishment of the Simulation Interoperability Standards Organization (SISO) as the organization responsible for the promulgation of applications of the HLA standard. Currently, HLA addresses technical interoperability, or the standardization of data interchange among model components at run time. However, it does not address substantive interoperability, the ability to assure that data have common meanings among components so that a coherent federation emerges capable of meeting the objectives of its designers. This capability should be developed. Modelica An early attempt at M&S standardization, the Continuous System Simulation Language (CSSL) was first published in 1967. CSSL defined requirements for a standard continuous simulation modeling language, but had limited impact. Mode]ica is the current manifestation of continuous system modeling standardization efforts (Elmqvist, l 9991. The Modelica

OCR for page 77
RESEARCHAND DEVELOPMENT TOPICS 97 Association, a nonprofit, nongovernmental association consisting of the members of the original Modelica Design Group, was established in 2000 to promote the development and application of the Modelica computer language for modeling, simulation, and programming of physical and technical systems and processes. The Modelica effort is based on recent research results. Object- oriented modeling languages have already demonstrated how object- oriented concepts can be successfully used to support hierarchical structuring, reuse, and evolution of large and complex models independent of the application domain. Noncausal modeling demonstrated that the traditional simulation abstraction can be generalized by relaxing the causality constraints, or by not committing ports to an input or output role too early. These results have the potential for enabling both simpler models and more efficient simulation. Discrete Event System Specif cation DEVSis a formal modeling and simulation framework based on generic dynamic systems concepts (Zeigler et al., 2000~.DEVS contains well-defined concepts for the coupling of components; hierarchical, modular model construction; supporting discrete event approximation of continuous systems; and supporting repository reuse with an object- oriented substrate. DEVS contains important abstract concepts underpinning the representation of mixed-signal electronic designs. The concepts of system modularity and component coupling to form composite models are well defined. The closure under coupling property allows coupled models to be treated as components and therefore supports hierarchical model composition. Advantages of the DEVS methodology for model development include well-defined separation of concerns supporting distinct modeling and simulation layers that can be independently verified and reused in later combinations with minimal reverification. The resulting divide-and-conquer approach can greatly simplify and accelerate model development, leading to greater model credibility with less effort. The DEVS methodology has been realized in high-level languages such as C++ and Java and has been extended for parallel and distributed execution. For example, DEVS-C++ models have been executed on parallel machines. Implementation of DEVS-C++ over message-passing interfaces can afford parallel execution of models and thus supports efficient, high-performance simulation of large-scale models. Furthermore, DEVS-C++is the basis for DEVS/CORBA, a distributed modeling and simulation environment formed by mapping the DEVS-C++ system onto the common object request broker architecture (CORBA) middleware.

OCR for page 77
98 MODELING AND SIMULA TION IN MANUFACTURING Models developed in DEVS-C++ or DEVS-JAVA can be directly simulated in parallel and/or distributed environments over any transmission control protocol/internet protocol (TCP/IP), asynchronous transfer mode (ATM), or other network. The DEVS formalism is both a universal and unique representation of discrete event dynamic systems. It has been combined with the differential equation formalism to form a composite formalism with a well-def~ned semantics that is able to express hybrid digital/analog systems. General Software Standards M&S standards fall outside the category of general software standards, because the body of knowledge intrinsic to M&S generates additional requirements that are left open in more general standards. However, it is worth reviewing the state of M&S-related software standards, as M&S standards must eventually mesh with them. Unit ed Modeling Language Software engineering promotes systematic, disciplined, and quantifiable approaches to the development, operation, and maintenance of software-intensive systems. By applying engineering principles to software, it strives to bring together methods, processes, and tools in a unified fashion. While fundamentally different approaches to software engineering have emerged in recent years, the object-oriented approach has become widely accepted and practiced (Booch, 1994, 1997; Pressman, 1996; UML, 2000~. In the object-oriented worldview, the software development process includes conceptualization, analysis, design, and evolution (Booch, 1997) and supports the architecture-driven paradigm based on a hybrid of spiral and concurrent software development processes. Adherents of the object-oriented approach consider it superior to other software development approaches such as functional and procedural. Furthermore, the modular architecture-driven approach can strongly support incremental, stepwise, iterative specification, design, and development of hardware and software components concurrently. Other advantages of the object-oriented approach include support for scalable high-performance execution and model development; dynamic reconfiguration; systematic and incremental verification and testing; and team-oriented development. The adaptation of object orientation to software engineering has become increasingly indispensable for systems exhibiting heterogeneity and demanding flexibility in terms of both software and interoperability with multiple hardware components.

OCR for page 77
RESEARCH AND DEVELOPMENT TOPICS 99 The unified modeling language (UML) has been managed by the vendor-neutral Object Management Group (OMG) since 1997. UML originated as a combination of approaches to software modeling developed by James Rumbaugh, Ivar Jacobson, and Grady Booch, but has now evolved into a public standard. OMG committees are defining ways in which the next version of UML can facilitate activities such as the design of Web applications, enterprise application integration, real-time systems, and distributed platforms. UML attempts to support a higher-level view of design and coding in terms of diagramming. However, the majority of developers still build in source code, working with linguistic rather than spatial intelligence. UML vendors are attempting to educate programmers to pay attention to design views, allowing users to decide which design view they want to see at any given time. UML definition is still in a state of flux. For example, many proponents believe that its features should be reduced to a small core, or kernel. One proposal for such a kernel would include use cases, class diagrams, and interaction diagrams but would exclude state charts and activity graphs that provide some of the richest semantics in UML. UML is aimed at general software development, primarily for business applications, and is not simulation-aware. UML is the union of at least 10 techniques for diagramming notation. However, there is much more to consider than diagramming in the realm of software engineering, and in particular, software development for models and simulations. In addition to the factors relating to all software, which include software design principles, exploiting patterns, and scalable architecture, the M&S developer must understand the particular characteristics of dynamic systems, the error properties of numerical algorithms, and the intricacies of parallel and distributed simulation protocols. Although state diagrams are included in UML, they are not adequate to handle the variety of dynamic systems of interest in M&S. UML does not support model construction from dynamic system components or from reusable model components as required for SBA. Fundamentally, UML should be applied to the development of software to support modeling and simulation, but not to the construction of dynamic system models. Common Object Request Broker Architecture Middleware technology evolved during the 1 990s to provide interoperability in support of the move to client/server architectures. The most widely publicized middleware initiatives are OMG's CORBA, Microsoft's distributed component object model (DCOM), and DOD's HLA run-time infrastructure (RTI) (Dahmann et al., 1998~. Middleware simplifies the integration of heterogeneous systems so that users can share

OCR for page 77
100 MODELING AND SIMULATIONIN MANUFACTURING infonnation more efficiently, more cost-effectively, more flexibly, and more extensively. It will become more critical as the Web matures and systems become even more distributed. Middleware services are sets of distributed software that exist between the application and the operating system and network services on a system node in the network. Middleware services provide a more functional set of application programming interfaces (APIs) than the operating system and network services in order to allow an application to locate transparently across the network, be independent from network services, be reliable and available, and scale up in capacity without losing function. The ability to operate in real time imposes additional stringent requirements on services that are not part of the middleware standard. Operating in real time implies not necessarily speed, but consistency or predictability of response as measured by small jitter, for example. Real- time object-oriented middleware attempts to provide parameterized objects that can be composed to provide quality of service guarantees to application-layer software. The ACE ORB (TAO), which is an extension of CORBA, is being developed to demonstrate the feasibility of using CORBA for real-time applications versus direct socket-level programming (Schmidt et al., 1998~. Real-time middleware being developed includes real-time extensions to message-passing interfaces (MPl/RT's) (Kanevsky et al., 1997) and real-time dependable (RTD) channel. The latter is based on CactusRT (Hiltunen et al., l 999), which was developed at the University of Arizona in an effort to make communication services with enhanced quality of service (QOS) guarantees related to dependability and real time in the context of distributed real-time computing. ARMADA is another set of communication and middleware services that provides support for fault-tolerance and end-to-end guarantees for embedded real- time distributed applications (Abdel Zaher et al., 19991. Higher-Layer Standards M&S is an enabling technology to the larger activities encompassed by systems engineering. Standards are emerging within this larger context as well, and it is important that these standards develop in a manner compatible with M&S. Generalized Enterprise Reference Architecture and Methodology As indicated earlier, GERAM is a developing standard for enterprise engineering, which is broadly concerned with designing and redesigning

OCR for page 77
RESEARCH AND DEVELOPMENT TOPICS 101 systems for industrial, administrative, and service applications (Vernadat, 1996~. Different modeling methods to support enterprise engineering have been proposed for different applications, including integrated computer- aided manufacturing definition methods for functional modeling, entity- relationship techniques for information systems, object-oriented approaches, decision system analysis, and activity-based costing methods for economic evaluation. However, few integrated methods exist to cover all of the aspects of a business entity. CIMOSA provides full coverage of four fundamental aspects of enterprise modeling - 1) function, (2) information, (3) resource, and (4) organization and clearly differentiates and represents the three fundamental types of flow in any enterprise: (1) materials, (2) information/decision, and (3) control flows (Vernadat, 1998~. However, current modeling and simulation tools are unable to support these modeling concepts fully, and standards are needed for such tools to support the growing use of the GERAM methodology. Data Exchange Standards Indust~y-based organizations have undertaken the development of several standards for data exchange that relate to and can advance the interoperability of models and simulations. The family of standards developed by the international Organization for Standardization known as the Standard for the Exchange of Product Model Data (STEP) aids in the exchange of computer-aided design (CAD), computer-aided manufacturing (CAM), and other types of product data. However, this family of standards has been over a decade in development, and there remains some resistance to its adoption in some commercial tools. During the last several years, significant progress has been made on the XML3 for data exchange. XML is applicable to the exchange of virtually any type of data, and a number of business and technical communities have developed associated standards using nomenclature common in those individual communities. CONCLUSIONS The complexity of planned and existing systems-of-systems is growing more rapidly than the power of the computational and modeling methodologies needed to simulate them. For example, multiresolution 3 Further information is available at .

OCR for page 77
102 MODELING AND SIMULA TION IN MANUFACTURING models that can reliably predict the effect of system design changes on the output of systems-of-systems operations do not exist. Achieving the comprehensive SBA vision requires an understanding of the fundamental limitations associated with the simulation and modeling of complex systems that does not currently exist. Those limitations cannot be overcome without advances in hardware and software and may require basic reformulation of the SBA problem. Research is needed to determine the theoretical and practice] limits of modeling and computation with respect to manufacturing and acquisition and to devise methods to work within and around those limits. To support the envisioned use of M&S, research is needed in modeling theory, especially multiresolution/ multiviewpoint modeling, agent-based modeling, and semantic consistency; and in modeling methodologies for dealing with uncertainty. Advances in technology, such as parallel computing, distributed computing, and distributed simulation, have begun to make integration and interoperability of simulation systems practical. However, the breadth of the comprehensive SBA vision, including model integration across all of the SBA viewpoints, is beyond current hardware and software capabilities. Research is needed to expand current model integration and interoperation, including that between engineering and effectiveness simulations. Setting standards for simulation interfaces and interoperability for system design data, including file formats or format descriptors, is timely and appropriate, and will allow improved interoperability and reuse. Standardization of tools may not be appropriate at this time. In order to ensure correctness of the models in use, research is needed in domain knowledge at a level of detail that can serve as the basis for models in domains relevant to manufacturing and acquisition. Research is needed in verification, validation, and accreditation, especially validation; and in human-behavior modeling, including modeling of cognition and belief. Finally, standards for interfaces and operability must be developed and applied to modeling and simulation software, general software, and the frameworks being developed for integrating other software systems.