Appendix C

Simulation-based Acquisition

Richard Ivanetich, Institute for Defense Analyses

INTRODUCTION

The objective of simulation-based acquisition (SBA) is to enable the acquisition process to proceed in a highly integrated and collaborative manner. Any military system represents the interests of numerous parties-e.g., its operators, acquisition authority, designers, producers, and maintainers. The integration envisioned by SBA allows all these parties to interact closely during the development of the system so that the resultant system reflects as well as possible their combined interests, the necessary tradeoffs between their individual interests having been reconciled in an optimal manner from the overall perspective of the system. In this way, the highest-quality system at the least cost should be obtainable. The opposite extreme is a "stovepipe" process, where the interests of the "downstream" communities (e.g., maintainers) are not represented adequately in the "up-front" design of the system, or such interests are later accommodated by expensive modifications to the system.

Integration and collaboration do exist in the acquisition process today, but the intent of SBA is to extend this capability greatly. From a technical perspective, achievement of this capability centers on the concept of a collaborative environment of design, analysis, and simulation tools in which a computer-based representation of the system under consideration (e.g., ship, aircraft, or major component of such)-a so-called virtual prototype-may be built and examined. This virtual prototype will be available to all concerned with the system, and they may examine it using their design, analysis, and simulation tools. The fact that all parties deal with and have ready access to a common representation of the system is the key element enabling these parties to interact in a highly integrated and collaborative manner.

These concepts are explored further in the following sections, beginning first with a more detailed discussion of the use of SBA.

SBA IN THE LIFE-CYCLE PROCESS

The life-cycle phases of a system may be specified progressively as follows:

• Requirements definition,
• Concept exploration-different concepts to meet the requirements are explored at a high level, and one option is chosen,
• Engineering design-the chosen high-level design is converted into a detailed design suitable for production,
• Manufacture,
• Test and evaluation-both developmental and operational, and
• Operation and maintenance-includes training necessary for operation.

System upgrade proceeds through these same phases, too, although the first five might not be as extensive compared to the case for a new system.

SBA relates to these phases in two ways. First, the concerns across the life cycle can be explored early in the life cycle. For example,

• The high-level virtual prototypes developed during concept exploration can be examined by the operators and maintainers to see how well the proposed concepts will meet their needs, with suggestions for improvement being fed back into the concept exploration. This assessment can be accomplished by visual examination of physical configurations depicted by virtual prototypes and by exercising the virtual prototypes in combat simulations.
• The engineering designers can comment on aspects of the high-level design that would be particularly expensive to realize, and discussion initiated with the requirements developers to see if less costly tradeoffs can be made.
• The detailed engineering designs can be examined by the manufacturers for production feasibility and suggested changes in design to simplify production processes.

Second, because of the integration afforded by SBA, design and other products developed during a life-cycle phase can be passed on to the next phase, thereby ensuring greater continuity and allowing cost reduction through reuse. For example:

• Engineering design can begin as a natural extension of the high-level design.
• The detailed engineering design can automatically be used to calibrate manufacturing processes.
• Simulations developed for conceptual exploration can be used for training purposes in the operations phase.

In short, the virtual prototypes and their associated design representations can serve as a record that is passed across the system life cycle.¹

SBA IN THE ENGINEERING PROCESS

Just as SBA promotes integration and collaboration across all life-cycle phases, it also promotes such within each phase. One phase that should be particularly noted in this regard is engineering design because it can involve very large design teams representing many engineering disciplines. Activity both within and across the disciplines must be coordinated. SBA seeks to facilitate the flow of information across the design team and increase the ability of its members to readily access this information. Consequent benefits would be as follows:

• The design time will be shortened because of the increased support in design tools and the much more ready access to design information.
• The decreased design time will allow more detailed design options to be considered, thereby leading to a better and possibly less costly design.
• A more nearly optimal design can be achieved by optimizing the design simultaneously from the perspective of all disciplines involved (e.g., aerodynamics, structures, and materials in aircraft design), rather than by suboptimizations conducted one discipline at a time.

Furthermore, if significant changes in design or tradeoffs not anticipated during concept exploration are introduced, the associated virtual prototype can be fed back to other members of the overall community (e.g., operators, maintainers) so they can assess and comment as necessary upon the effect of the changes.

STATUS OF SBA TECHNICAL CAPABILITIES

Three key components are necessary to provide the overall SBA collaborative environment:

• Product representations-these are the computer-based representations of a system or the components of a system. They should refer to both the design of the system or component and its behavior. The behavior is determined by a model of the system or component. The model could be something relatively straightforward like specifying the weight of a beam used as a structural component in a ship or something more complex like the performance of the overall ship. A product representation capable of describing behavior is said to be an executable product representation.
• Analysis tools-these are any of the large number of tools that would be used, for example, in creating or assessing the designs and associated product representations. Computer-aided design (CAD) tools are a primary example.
• Interface infrastructure-this is the capability that allows the product representations and tools to interact with one another.

The following sections briefly describe the status of capabilities in each of these areas, in terms of both existing capabilities and missing capabilities.²

Product Representations

Existing Capabilities

Sophisticated static (nonexecutable) representations of the geometric aspects of a design exist. This has been demonstrated, for example, in the development of the Boeing 777 aircraft, where all the data describing the physical configuration of the aircraft were generated, manipulated, and retained in digital form accessible to all the designers. Furthermore, sophisticated executable representations of structural components (e.g., beams, bulkhead plates) have been demonstrated and used, for instance, in the development of tankers by Newport News Shipbuilding. In this case, for example, the dimensions of the structural component can be changed in the digital representation by the designer, and the properties of the component (e.g., weight) are automatically recomputed.

Missing Capabilities

Three related missing capabilities have been identified:

• Multi-resolution modeling formalism. Typically, one wants to predict the performance of an overall system from the properties of its components. This has been done manually, but there exists no formalism to facilitate the ready aggregation or disaggregation of product behavior.
• Cross-domain consistency. It is unrealistic to think that there will be just one "object" describing the behavior of a system or component. Members of different disciplines will have their own representations expressing those properties of interest to them (e.g., some will be interested in thermal properties; others will not). A formalism is necessary to help ensure that these different representations are consistent (e.g., mean the same thing by a commonly named variable).
• Propagation of uncertainty. No matter how detailed, there is always some element of uncertainty in the description of a system or component. Individual disciplines (e.g., aerodynamics, structural mechanics) have characterized these uncertainties fairly well. What has not been treated, however, is how uncertainties in the model of one discipline propagate when the model is used in conjunction with the model of another discipline. This is necessary to understand because the overall behavior of a system or component is predicted based on these multiple models.

Analysis Tools

Numerous sophisticated tools (e.g., CAD) are available in modern engineering environments and will not be detailed further here. However, full realization of SBA does require further capabilities, two particularly critical ones being design optimization and cost estimation tools. In particular, multidisciplinary design optimization methods and tools-which allow attempts at design optimization to proceed across all disciplines concurrently-are necessary. Such methods have been illustrated in simple examples (e.g., Advanced Surface Combatant demonstration in the DARPA Simulation-based Design (SBD) program), but development of sophisticated, comprehensive techniques has not yet been achieved. Cost estimates of designs are possible, but there do not exist cost models that easily predict the results of changes in design parameters, especially in cases where new technologies are involved for the system or component under consideration. Such capabilities are necessary to readily effect the cost-performance tradeoff analyses envisioned as an aspect of SBA.

Interface Infrastructure

Existing Capabilities

The general concept of integrating product representations and tools has been demonstrated by Lockheed-Martin and other contractors as part of the SBD program. A key feature here was the ability to put software "wrappers" around existing simulations and tools so that they could interact with one another. In this way, it is possible to use existing components to conduct SBA, and not necessarily require the construction of new components.

Given the wrapped components, the basic interfaces among them were defined by the high-level architecture (HLA) for simulation, although it was also necessary to define domain specific interfaces for the applications considered. The HLA object model templates also provided the basis for defining the nature of the information to be exchanged among the simulations and design tools. Use of HLA was demonstrated in the engineering proto-federation experiment carried out as part of the HLA program, as well as in the DARPA SBD program.

Missing Capabilities

As noted, it was necessary to define domain-specific interfaces to carry out the SBA demonstrations. In general, it will be necessary to define standards for such interfaces so that SBA product representations and tools can be developed independently and shared. No such standards now exist. For instance, important examples relate to geometric modeling representations. No standards now exist to allow the ready coupling of these data to dynamics simulations used for design purposes (e.g., computation fluid dynamic calculations) or to numerically controlled manufacturing devices.

SUMMARY OF SBA TECHNICAL CAPABILITIES

A significant SBA capability now exists, particularly as relates to the use of shared digital representations for depicting physical configurations in place of "paper" representations. For example,^3,4,5

• Errors in design can be detected and corrected much earlier, as evidenced by Boeing's 95 percent reduction in engineering change notices in going from the 757 to the 777 aircraft. Similarly, rework on produced aircraft was reduced from 30 percent on the 747 to 3 percent on the 777.
• The design cycle time can be significantly shortened, as evidenced by the 20 percent reduction in cycle time achieved by Newport News Shipbuilding in developing the Double Eagle tanker.
• Significant cost reductions should also be achievable, as evidenced by the projected 25 to 30 percent cost schedule reduction due to elimination of the physical mockup in development of the NSSN by Electric Boat.

While the above examples refer primarily to the use of geometric modeling and executable product representations for structural components, a demonstration illustrating the more general concept of SBA has been conducted in the SBD program. In that example, design tools, engineering simulations, and combat simulations were integrated, operating on a common product representation.

Still, significant additional technical capability as noted in the subsections above, is required to achieve the full SBA capability. The multi-resolution modeling formalism will allow one to move more readily from component to system representations, as is necessary to exercise engineering-level designs in combat simulations. Multidisciplinary optimization, cross-domain consistency, and the propagation of uncertainty are all necessary to achieve the full degree of collaboration envisioned by SBA for the engineering process. And domain-specific interface standards are required to allow the ready integration of SBA components developed independently by different parties. Some of these needed additional capabilities such as the interface standards might be achievable by disciplined coordination efforts in the SBA community, but most of the additional capabilities are still at the status of difficult research problems today.

CULTURAL FACTORS

SBA is not solely a technical matter. In fact, since it presents new methods for the acquisition process, there are also factors of cultural and managerial acceptance. In some regards, these factors could provide challenges as significant as the technical ones. There appears to be a growing acceptance of SBA in commercial industry-at least from the perspective of geometric modeling and executable representations of structural components. For example, the Boeing and Newport News experiences were noted above, and Lockheed-Martin is also applying SBA in a satellite development program. Within DOD, significant interest has been expressed in OASN(RDA)/ARO and OSD/DOT&E, as well as in the DARPA SBD program. Individual Navy programs (e.g., SC-21, CVX, NSSN, LPD-17) have also expressed some interest. However, no institutional commitment to SBA has been made by the Navy.

One factor relating to the acceptance of SBA is that it requires greater up-front cost, although at the promise of significantly reduced life-cycle cost and possible reuse in other programs. These up-front costs relate to the development of greater design artifacts and simulations. Program managers can be reluctant to incur these costs because their benefits will not be realized during the program manager's tenure. A higher-level institutional commitment could thus be required to promote SBA.

CONCLUSIONS

The capabilities envisioned for SBA offer significant potential for providing a more efficient and effective acquisition process. In part, these capabilities have been realized and their benefit shown in examples such as the Boeing and Newport News ones noted above. Furthermore, the overall concepts have been illustrated in the SBD demonstrations. Still, as noted above, significant technical and cultural challenges remain before the vision of SBA is obtained. What follows indicates steps by which the Department of the Navy, and DOD more generally, can work toward this vision. It should be emphasized that the path to the full vision is long and complex enough that it is not adequate just to postulate this vision. Rather, any planning done in the Department of the Navy and DOD should lay out a logical set of steps to this vision providing increasing capability, and also assess the cost-effectiveness of each of these steps. The following actions would provide some of these steps:

• Pilot projects. As noted, program managers could be reluctant to institute SBA capabilities because of up-front costs to their program, even though there could be significant downstream benefits. Thus, a separate pilot project or projects could be set up to develop SBA capabilities that would feed into a major naval program (e.g., ship or aircraft). Examples of the sort of capabilities developed would be (1) executable product representations (in particular, ones that go beyond the current structural representations), (2) the coupling of representations to combat simulations to assess the utility of the designs being created, (3) the coupling of geometric representation data to dynamic engineering simulations, and (4) means to share information (in both directions) between design engineers and manufacturing producers so as to both enhance the producibility of designs and also let the designs take advantage of new manufacturing concepts. In addition to helping an individual naval program, a pilot project would also promote demonstration, assessment, and transition of SBA capability in the Navy more generally.
• Standards development. While the HLA defines some general interfaces for integrating simulations, more extensive domain-specific interface standards are necessary to allow independently developed design tools, product representations, and simulations to interact with one another. The degree of standardization is not clear a priori. Not all aspects of the interfaces should be standardized, less the standards become too constraining, but some core set should be standardized. Experimentation would be conducted to determine this core set. Such experimentation would be carried out by getting together a set of participants representing members of the life-cycle and engineering communities and letting them work out the standards (including associated semantics) in the context of a demonstration project. This approach would be analogous to the proto-federation experiments carried out under the direction of DMSO in HLA development.
• Research. Even with the items noted in the previous two paragraphs, some significant capabilities requiring basic research will also be necessary to achieve the full SBA vision. Relevant topics include multi-resolution modeling formalism, cross-domain consistency, propagation of uncertainty, multidisciplinary optimization, and advanced cost estimating tools. Research programs are needed to address these problems. They should be conducted in coordination with parties in the other Services and organizations like DARPA and DMSO that would also be interested in these matters. While one cannot predict just when basic research will have fruitful results, one should attempt to guide the researchers by having them apply their results to concrete problems as soon as feasible.

The costs associated with such steps can only be roughly estimated here. A pilot program might cost around $20 million to $50 million per year and run for 2 to 3 years. The experimentation to determine standards might cost approximately $20 million per year and run for 2 years. The research program might cost on the order of $10 million to $20 million per year and run for several years.

Footnotes

1. The virtual prototypes will evolve over the life cycle. The high-level ones developed in concept exploration will become more detailed during engineering design; as hardware components become available, they could for certain purposes replace simulated ones in the virtual prototypes; and the data used in the virtual prototypes will be refined based on information gathered in test and evaluation.

2. Not included here is a discussion of the combat simulations in which the virtual prototypes would be exercised. Such simulations are a key component of SBA, but are a major topic of discussion elsewhere in this report.

3. Visit to Boeing Corporation, March 14, 1997.

4. "Simulation-based Design" briefing at Lockheed-Martin, February 28, 1997.

5. See Patenaude (1996). This report (conducted for the Deputy Director, Test, Systems Engineering and Evaluation, Office of the Secretary of Defense) contains several examples of the use of modeling and simulation in the acquisition process.