Click for next page ( 30


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 29
2 Modeling and Simulation in Defense Acquisition The use of modeling and simulation (M&S) in defense acquisition is not new. Before digital computers and networking technology, analysts mathematically modeled the effect that new or improved defense systems would have on warfighting capability, engineers built physical models of systems, and testers simulated combat in field tests and exercises. In the 1 960s, as computing capabilities increased, the task of modeling and simulating both the design and performance of defense systems moved increasingly toward digital representations and algorithms implemented in computer software. As high-speed digital networking evolved during the 1 980s and 1 990s, the ability to share this digital information both within and across organizations increased rapidly and created opportunities for collaboration in the development of defense systems. In the 21 st century, the long-term needs facing defense acquisition will require an expanding array of M&S technologies to enable rapidly evolving, even revolutionary, defense acquisition. In order to assess the long-term needs for M&S in defense acquisition and the current state of M&S use within DOD, the committee examined DOD's future acquisition vision, current uses of M&S in developing defense systems, existing initiatives for supporting M&S in acquisition, and the results of 10 previous acquisition-related studies of M&S. 29

OCR for page 29
30 MODELING AND SIMULATIONIN MANUFACTURING SIMULATION-BASED ACQUISITION A variety of terms have characterized DOD use of M&S technologies over the past two decades. Within DOD, advances in M&S have occurred primarily in the defense simulation and the product modeling, design, and manufacturing communities (NRAC, 1994~. Within the defense simulation community, work on distributed, linked M&S has generally been termed advanced distributed simulation (ADS). This work evolved from the early simulated networking (SIMNET) effort sponsored by the Defense Advanced Research Projects Agency (DARPA), the development of distributed interactive simulation (DIS) standards, and DOD-sponsored work on high level architecture (HLA). Within the product modeling, design, and manufacturing community, M&S advances have been achieved through commercial efforts, such as the Boeing 777 aircraft design, and through defense efforts, such as the DARPA simulation-based design (SBD) program (NRAC, 1994~. Definition of S;muiation-Based Acquisition In 1994, a Naval Research Advisory Committee (NRAC) study panel characterized the convergence of activities between these two communities the defense simulation and the product modeling, design, and manufacturing communities- as distributed simulation-based acquisition (DSBA) and suggested that this concept had the potential to revolutionize the defense acquisition process (NRAC, 1994~. As the DSBA concept evolved within DOD, particularly within the Office of the Secretary of Defense (OSD) and the Acquisition Functional Area Council ofthe DOD Executive Council on Modeling and Simulation (EXCIMS), the name was shortened to simulation-based acquisition (SBA). In December 1997, the Acquisition Functional Area Council defined the following vision for SEA, which was re-adopted as a concise definition in August 2000: "An acquisition process in which DOD and industry are enabled by robust, collaborative use of simulation technology that is integrated across acquisition phases and programs." (SBATF, 1998; SBAISG, 2000) It is important to note that the term SBA is used not only in reference to current efforts to incorporate M&S into the defense acquisition process, but, more importantly, it is used to designate the desired future DOD

OCR for page 29
SIMULA TION-BASED ACQUISITION 31 acquisition process. The specific goals of the future SBA concept are to use M&S technologies to reduce time, resources, risk, and total ownership costs of military systems in the acquisition process; to increase quality, military worth, and supportability of military systems; and to enable integrated product and process development during acquisition (SBATF, 1998; SBAISG, 2000~. Models of proposed system designs would be constructed and tested in simulated environments, and these virtual prototypes would then be used to refine system requirements and relate trade-off and engineering decisions to the requirements. Computer-based models could be maintained throughout the development, production, and modification phases of the product life cycle (NRC, 1997~. A collaborative acquisition process, making extensive use of advanced M&S technology, exercised in a modified defense culture is the end state desired by DOD. The SBA concept is broad, encompassing not only product development and manufacturing, but also simulations to estimate system performance and mission effectiveness, combat training, the underlying technical information needed to train system operators, the product modeling and manufacturing processes of the commercial enterprises that support defense acquisition, simulations to support maintenance training, technical information used by maintainers, logistics simulations to relate support plans and resources to readiness, and simulations to address system disposal issues. If SBA is thought of as a category of M&S applications (i.e., ways of using M&S), Figure 2-1 illustrates the relationship between SBA and other M&S applications categories. The outermost oval represents all applications of M&S. The two large interior ovals represent two overlapping sub categories of M&S applications: those that support or enhance military power and those that support or enhance commercial success. A few arbitrarily chosen example applications within each All::~hf&S Applications Simulation- Based Acquisition Combo vehicle design I:: :: - it-; ~ ~ -\ ~ We on session . S~urgerysimulatlon Science education Figure 2-1 Relationship between SBA and other categories of M&S applications.

OCR for page 29
32 MODELING AND SIMULA TION IN MANUFACTURING category of application are listed in each portion of the figure. SBA, which, as previously noted, is quite broadly defined, is nevertheless a subset of all military M&S applications. Furthermore, some of the M&S applications within SBA are also valid in the commercial context, as the figure suggests. Each of the military services is developing the SBA concept to fit its specific needs. The Department of the Navy is integrating SBA with other M&S initiatives and standards activities intended to help acquisition program managers (Hollenbach, 20003. The U.S. Anny's SBA vision is called the Simulation and Modeling for Acquisition, Requirements, and Training (SMART) initiative. Among other characteristics, SMART emphasizes collaboration during the acquisition process between the developers, evaluators, and users of combat systems (Purdy, 2001~. In 1998, the Joint Simulation-Based Acquisition Task Force (SBATF) was commissioned to develop a road map for DOD action on SBA (SBATF, 1998~. The SBATF's objectives were to develop representations of architectures needed to establish SBA environments, identify technical challenges, identify primary ownership of each module in the systems architecture, identify investments needed by government and industry, list DOD actions needed to develop the SBA concept, and identify industry actions needed to accelerate the SBA concept. The SBATF made extensive recommendations in the areas of management, architecture, policy and legislation, and education and training. Use of Modeling and Simulation in Defense Acquisition The establishment of DOD's Defense Modeling and Simulation Office (DMSO) in 1991 was an important milestone in recognizing the potential of M&S for defense applications. DMSO is responsible for the development of the DOD M&S master plan; the development of the DOD M&S investment plan; the establishment of the Modeling and Simulation Information Analysis Center (MSIAC); and the development of technology, standards, and tools for M&S. The M&S master plan, combined with the M&S science and technology program, is the main source of information for acquisition program managers. In addition, the MSIAC provides tools and information that can help program managers develop their simulation support plans. The existence of DMSO was an important driver in the development of DOD's HLA and synthetic environment data representation and interchange specification (SEDRIS). HLA provides rules and run-time infrastructure to allow M&S applications to be integrated to meet new or changing requirements; it has already advanced the use of M&S in analysis, acquisition, and training. SEDRIS has enabled the sharing of

OCR for page 29
SIMULATION-BASED ACQUISITION 33 environmental data across M&S applications. Both of these standards have therefore increased reusability of the M&S applications developed by DOD. Examples of Modeling and Simulation in Defense Acquisition M&S is already in use in several defense programs as a means of improving the design of new systems, integrating manufacturing modeling with system simulation, and evaluating the combat effectiveness of new systems. U.S. Navy LPD-I 7 Program The U.S. Navy used M&S in engineering the systems and subsystems ofthe LPD-17 helicopter carrier.' A digital model ofthe LPD-17 was developed using three-dimensional computer-aided design (CAD) modeling and computer-aided manufacturing (CAM). By developing this type of integrated product development environment combined with three- dimensional visualization, the Navy leveraged previous efforts that had demonstrated the value of three-dimensional visualization and modeling. The Sec~wolf; Virginia Class Attack Submarine, and Aegis Class Destroyer programs had shown that use of three-dimensional visualization and modeling to solve engineering problems could reduce reengineering costs once production had begun. A comprehensive plan to use M&S throughout the system acquisition process was developed for the LPD-17 program. This plan included constructive simulation to support concept studies and beyond, virtual simulation to support concept demonstration and beyond, as well as live simulation to support demonstration and validation for the following acquisition milestones: concept demonstration approval, development approval, production approval, and major modification approval (see Figure 2-23. Most of the M&S investment was focused on the three- dimensional product model for the purpose of supporting engineering design. ' Mike Wendel, Coleman Research Corporation. 2000. Presentation to this study committee.

OCR for page 29
34 MODELING AND SIMULATIONINMANUFACTURING ~as" ~ Phil - ~ P6.~ It ~ HI ~ IY On ~ Cow ~ ~~ ~ ~ ~ ~ ~ ~ ~~ ~~~-~ ~ Lion ~ Ex - ~n ~ ,;~. ~~$~ ~ 34.t~ ~ 13~_ ~ a~` 0~. ~~ - :=n - : ( : s'~s O APE :h~' C: C~n R~.~ - We - ~$ ' S~.~s Lam - ~ ~ 6. P~r~ M - Is DO ~ ' ~ ' -'~<'~~~ Ad: ,~ Up, ~ Page.' Balm ~~ - S=~S : ....... .. . p~,,~G~t ^~1: ~ CAT ship CHIP % T~. T,r.a' - ,~ 6 Lit - Id. _ Id - n~:s~ Figure 2-2 Use of M&S in the acquisition process for the LPD-17. Source: Mike Wendel, Coleman Research Corporation. 2000. Presentation to the study committee. Joint Strike Fighter Program Both the government program manager and the system contractor, Lockheed Martin, have used M&S in the Joint Strike Fighter (JSF) program. The U.S.Air Force developed a simulation-assessment-validation environment (SAVE) that focused on the use of M&S within the context of DOD's integrated product and process development framework.2 According to the Air Force, no tool set was available in 1995 for the integration of manufacturing modeling with system simulation. The SAVE initiative sought to bridge this gap and to use M&S in a comprehensive program. The goad was to avoid dater reengineering costs caused by a less robust process that matched design with requirements. Lockheed Martin developed a version of simulation-based acquisition for the JSF program.3 The company used the "V" mode] described by Forsberg, Cotterman, and Mooz (2000) and B]anchard and Fab~ycky (1998) (see Figure 2-3~. This is a new application of the classic systems engineering mode] to describe the integration of M&S with the systems . . engineering process. 2 James Poindexter, Air Force Research Laboratory. 2000. Presentation to this study committee. Matt Landry, Lockheed Martin. 2000. Presentation to this study committee.

OCR for page 29
SIMULA TION-BASED ACQUISITION re~n~ i: ~ ~v ~k~?ha~::~ it:: :~: Aticea~ a. .~ mar ~ :! ~ ~fehide~ . ~ ~ ...,. ~ .,.~., We~ sorry ~n'~ ~ S - Ah - ~ ,Adf 'ERIC IF6gra Wr~ff= ~5~ 6, ver'~m ~ Figure 2-3 Planned applications of M&S in the system design and development phase for the Joint Strike Fighter. Source: Matt Landry, Lockheed Martin, presentation to the study committee, 2000. U.S. Navy SC-21 Program 35 The U.S. Navy's 21st Century Surface Combatant Land Attack Vessel SC-214 program was a pilot project in which computer-generated forces created the simulated battlespace in which the SC-2 1 design concepts were evaluated (Ewen et al., 2000~. These forces included both friend and foe entities. Such programs are important because SBA will require computer-generated forces to analyze the effectiveness of combat systems. A limitation on this type of evaluation is the need to improve human behavior models for M&S applications (NRC, 1 998b). As another step in this field, the U.S. Air Force Research Laboratory has established the combat automation requirement testbed (CART) program to develop models of human performance and behavior. These models will be used to represent the performance of human operators, such as aircrew members, in M&S evaluations of weapons systems during the acquisition process. 4 This program has since been redesignated as DD 21 and is now incorporated into the DD(X) program.

OCR for page 29
36 Future Programs MODELING AND SIMULATIONIN MANUFACTURING The Future Combat Systems (FCS) Program plans a comprehensive approach to the use of M&S in analyzing requirements, supporting design, and supporting test and evaluation (Purdy, 20011. This program, based on the U.S. Army's SMART initiative, has a systems engineering approach similar to that used by Lockheed Martin for the JSF (see Figure 2-4~. and appears to be making very comprehensive use of M&S. The "V" model diagram shown in Figure 2-4 attempts to capture the flow of systems engineering activity as performed in the FCS program and is a framework applicable to other programs. At the upper left, overarching system-level design requirements are developed. Flowing down the left side are increasingly fine details of subsystem design and specification utilizing M&S to assess performance of the subsystems relative to their interface specifications with other subsystems. The right side of the diagram incorporates integration and production of the components and subsystems, using M&S to test whether the larger system requirements are being met. The U.S. Navy's Collaborative Engineering Environment (Crisp, 2002) is being formulated to assist program managers by providing collaborative tools and supporting data focused on the Pre-System Acquisition Phase of the Acquisition Process. The other services would ~__e _ ~ B.. i. ~ ~ ~ ~ ~ ~ i. ~ ~ . ~ ~ . . ~ . ~ . ~ ~ ~ i. ~ ~ ~ .c A ~ ~. ~~ ~~ ~ -. . ~~ a. ~~ ~ s - .::::-., I': ~ ~~ :.:::.\ O - gn \~ ~~ -:~ : ~~ ~~ ~~ ~~ By: Requirements ' Jam -I= ~ ~~ ~ ~ ~ ~ I- ~ ~~: ~~ ~~ ~ ~~ f A- ~ ~ ~ I: ~~:~ ~~ -~ Ace. ~ ~ ~~ ~ ~~ ~ ~~ An.' ~ it,, ~ ~ ~ ~ ~ . ~ f-~ ~ ~~ ~^ ,,,,, ~ ~ ~~ . ., ~ . ~ ~~-~ ~ .~ ~~:~ ~! ,:::.: . ~.~ ~~ ~~ .. . it. ~~, it: .:., , .:_ ~ ~ ~~ ~~ ~~ ~ . ~ ~ ~~ .~ :: ~ :# I: ~~ :~:: ~~ ~ .~ ~ i: ~~:~ i. :. ... ~ air. : ~~ I! f :~ :~ ~~ -: ~ ~ ~~ : : ~ ~~ ~ I: ~! Item-L.~I \ Meson Hi, i: I~-~'rernents \ \ \ .411 Oes3gn FIeq`3i~rnents ~mplee ~ ~~ ~:~:! Con Steno '' .. ~~ ~-~.~ .1 ~ I::, ' i/ trial / ~ portents Figure 2-4 Future Combat System "V" Model. Source: Purdy (2001~.

OCR for page 29
SIMULATION-BASED ACQUISITION 37 benefit by examining this environment as an excellent means to implement SBA concepts at the earliest point in the Acquisition Process." Barriers to More Widespread Use Although DOD is committed to maximizing the benefits of M&S technologies, numerous barriers impede the more widespread use of M&S in defense applications. These barriers include inadequate allocation of resources to support SBA, lack of information for acquisition program managers, obstacles to collaboration between government and industry, the need to protect intellectual property rights, the lack of dissemination of information on SBA to the wider M&S academic community, and the need for standardized education for M&S professionals. Inadequate Allocation of Resources Inadequate managerial authority, as well as insufficient financial and leadership resources, has been allocated to support the achievement of DOD's stated vision for SBA. The development of HLA and SEDRIS are examples of centralized investments that have led to cost savings for individual programs. In contrast, investments in M&S technology, standards, and tools made by individual programs are direct costs and difficult to recoup. To stimulate activities that are of value to the larger community, a variety of approaches should be considered (e.g., programs for centralized investment; or policies to reward an individual acquisition which makes an investment that benefits others). Existing uses of M&S in defense acquisition indicate that efforts to implement SBA have resulted in uneven applications of the approaches and capabilities available to program managers. Investments in the development of M&S technology and standards and investments in the application of M&S to the defense acquisition process appear unbalanced. Although DOD has the infrastructure, procedures, and plans to identify, develop, and maintain needed M&S standards and technologies, the same level of maturity has not been achieved in applying M&S capabilities to the defense acquisition process. Reports from program office principals and from the DOD acquisition and M&S communities indicate that the important ongoing efforts to develop M&S technology and standards are achieving success on a broad front, but that there is an important shortfall in the application of M&S to the defense acquisition life cycle and to the engineering of systems within the defense acquisition life cycle. improvements are also needed in the culture associated with the use of M&S in manufacturing and acquisition.

OCR for page 29
38 MODELING AND SIMULATIONIN MANUFACTURING Lack of Information' for Program Managers Currently, program managers lack comprehensive information on how to integrate M&S with the systems engineering process and information on available tools to support and realize this integration. Although the DOD Acquisition Deskbook5 contains specific information useful to understanding the policy and procedures that support SEA and the use of M&S in the acquisition process, there is no reference to best practices as carried out by DOD program offices in applying M&S to the acquisition process. The Acquisition Deskbook includes a process discussion that explains the role of M&S in system acquisition and explains the integration of M&S into the system acquisition life cycle through descriptions of several acquisition phases in DOD-wide practice. Although this is an excellent beginning, more is needed on how M&S is brought to bear on the systems engineering life-cycle process. DMSO has developed detailed discussions of the role of M&S overall and in each acquisition phase. This discussion must be augmented by a discussion of the role of M&S in each activity of the systems engineering process. Several useful cases exist that would illustrate how specific M&S approaches and tools were applied to an activity and what benefits were derived from doing sofor example, the JSF program's use ofthe U.S. Air Force's Thunder M&S application in order to understand the value of the JSF in a theater context. Thunder is an accepted campaign-level model used throughout both the U.S. Air Force and other DOD units to study force structure and system requirements with a view toward understanding their contribution to operational outcome. Several campaign-level models have wide acceptance in DOD and are applied to assess the value of investment in systems by understanding the return on investment based upon the effect on operational outcome. The combination of campaign- level, mission-level, system-level, and subsystem-level models provides a capability to support system engineering activities at the system, subsystem, and component levels both in new system development life- cycle phases and in system modification and system maintenance life-cycle phases. Needfor an Integrated Systems Engineering Process Currently, there is no broadly accepted definition of an integrated systems engineering process for the development of software-intensive systems. To overcome this barrier, DOD has assigned the Software Engineering Institute and the National Defense Industrial Association's s Available at . Accessed June 2002.

OCR for page 29
SIMULA TION-BASED ACQUISITION 39 System Engineering Committee to define an overarching systems engineering process for such systems. This effort pulls together industry best practices, such as the Electronics Industry Association's 632 Standard, Processes for Engineering a System (EIA, 1999), and the Institute of Electrical and Electronics Engineers' (IEEE's) 1220 Standard, Application and Management of the Systems Engineering Process (IEEE, 1998~. This effort may serve as a model for the development of similar systems engineering models for other DOD domains. Similarly, M&S needs to be better integrated into the overall DOD systems engineering and acquisition process. Methods and standards for DOD M&S (such as HLA and SEDRIS, while beneficial within the M&S discipline, need to be better related and integrated into the overall system acquisition process. For example, it would be beneficial to have tools that assist in ensuring that the software representations developed in accordance with the HLA standard are consistent with the requirements documents for the systems they represent, and that they accurately represent the operational and systems architectures associated with these systems. Although there have been attempts at codifying DOD mission representations (such as the Functional Descriptions of the Mission Space effort, formerly known as Conceptual Models of the Mission Space), these efforts are not sufficient to ensure the degree of integration and consistency that is needed. Obstacles to Collaboration Increased collaboration between government and industry has been recommended by numerous studies as an essential element of the success of SBA. Specific technical requirements recommended for achieving this objective include distributed information repositories with search access, collaboration mechanisms, and security and access control mechanisms for shared data (Ho]]enbach, 20003; co]]aborative environments (SBATF, 1998; Coo]ahan et a]., 2000~; and data standards for integrated data environments (Starr, 1998~. Nontechnica] changes are also needed, including policy and law changes to support the delineation of responsibilities and contractual sharing of data (Ho]]enbach, 2000) and changes in the acquisition process to adapt it to the new methods of SBA. Specific requirements include partnering of combat and weapons system developers earlier in the concept exploration phase, examination of the process by which the government trades simulations with industry during the development process, and examination of the resources required for this effort (Hollis and Patenaude, l 999~. Another nontechnical issue is that security classification of the scenarios behind models and simulations limit

OCR for page 29
40 MODELING AND SIMULATIONIN ~NUFAC TURING their use in academic environments and increase the cost of using them elsewhere. Intellectual Proper Rights Protection of intellectual property rights is also a non technical issue that presents a potential obstacle to use of M&S in acquisition. Specifically, the question of how to protect the proprietary interests of model builders within the SBA process must be addressed. Bidders in highly competitive military procurement programs may be selected on the basis of the results of simulations of proposed weapons systems. The simulations used to provide information for decision making will most likely be provided by the bidders themselves. Competing bidders, military decision makers, and elected officials will all have an interest in examining the simulations for accuracy, while the details of the proposed weapons systems and the proprietary nature of the modeling methodologies will need to be protected. Information Dissemination The progress of M&S use in defense applications may be hindered by the insufficient and uneven dissemination of information on SBA to the broader M&S community. This community is divided into subgroups with diverse M&S interests, such as simulation, interoperability, industrial engineering, physical sciences, and biological sciences. While SBA is well represented at the meetings of some professional organizations, little awareness of it exists at others. Because the SBA vision covers the complete spectrum of phases in system acquisition, many ofthese disciplines have important contributions to make. Although academic researchers may be more interested in the narrower issues relating to their own disciplines, it is important that the SBA vision, requirements, and opportunities for involvement be more widely disseminated. Education and Training Information technology is facing exponential growth as a field, with a corresponding increase in training materials, courses, and certificates in hardware design, network administration, and programming. M&S has developed into a field that interacts with a cross-section of science, business, and engineering applications. M&S professionals, in addition to requiring a basic information technology background, also require an understanding of core concepts and skills specific to M&S. The future

OCR for page 29
SIMULA TION-BASED ACQUISITION development of M&S as a discipline, profession, and industry is strongly reliant on the growth of necessary M&S education and training. Validation and Serif cation 41 If simulation-based acquisition is to be credible, it must employ data and models and simulations that have undergone rigorous validation and verification. Although numerous efforts have been made by DOD and professional organizations (e.g., the Military Operations Research Society) to formulate guidelines that address this issue (Pace and Glasow, 1999), an efficient, rigorous process remains to be formulated and applied systematically. This is particularly challenging for future systems-of- systems in which new doctrine, concepts of operation, and training will convolve as users experiment with and gain experience with the new systems. REVIEW OF ACQUISITION-RELATED STUDIES ON MODELING AND SIMULATION During the 1 990s, many studies were sponsored by U.S. government agencies and industrial organizations in the general area of M&S as it is related to the manufacture and acquisition of systems. On the basis of its experience, the committee selected and focused on 10 studies sponsored by U.S. government agencies or industrial organizations and published since 1994. These studies ranged in focus from design and manufacturing methods, such as collaborative virtual prototyping (CVP), to broad-based M&S strategies, to specific acquisition-related M&S areas such as SBA. After performing its review, the committee found areas of overlap in the recommendations of the ] 0 studies; these overlapping areas can be grouped into the following 5 categories: (1) leadership, (2) processes, (3) technology, (4) motivation, and (5) experimentation. Each study emphasized these areas to a different extent, depending on the original objective of the study, the targeted organization, and the general environment in which the study was conducted. The following subsections elaborate on these overlapping recommendations (see Appendix B for a summary of the objectives and major conclusions and recommendations of each study).

OCR for page 29
42 MODELING AND SIMULATION IN MANUFACTURING Multiple Recommendations on Leadership The majority of the 10 studies concluded that significant leadership attention devoted to advancing the use of M&S in acquisition was extremely important for its success. These studies called for leadership either in the form of a separate office or in the form of an individual who would act as a focal point or champion. This office or individual was recommended to be at the most senior level of the organization to which respective studies were directed. Without exception, the studies recommended investment of funds to advance the use of M&S in acquisition. These investments, applied according to integrated plans, should be directed toward establishing the M&S infrastructure necessary for common use, corporate and enterprise capabilities, and/or demonstration programs. The studies also recognized that investment in M&S would have to be made early in the life cycle of a system, whereas many of the projected benefits would not occur until later. Overall, the leadership recommendations focused on new organizations, roles and responsibilities, and funding and investment. Multiple Recommendations on Processes As noted above, DOD's definition of SEA is directed toward making the acquisition process more effective. Therefore, most of the 10 studies that were focused on DOD and the services addressed the use of M&S as an integral part of such a process. These studies found that early involvement of operational users of systems during the development of systems requirements and design was an important component of increased effectiveness. Studies characterized by strong industry participation stressed the importance of earlier industry involvement. For both government and industry, M&S was seen as an enabler of earlier participation. Other common themes were the need to foster cooperation and collaboration between government and industry and the need to promote information sharing. Specifics included the need to leverage commercial practices for DOD applications, the need to use collaboration technologies effectively, and the need to provide industry with government M&S technologies. Cultural factors and concerns about proprietary data were often cited, however, as obstacles to effective information sharing. There was some recognition that government policy and legislation should be considered as factors in enabling or hindering collaboration. However, there did not appear to be consensus on whether specific new policies and legislation were essential or whether the flexibility in existing policies was

OCR for page 29
SIMULA TION-BASED ACQUISITION sufficient to allow for more collaborative business practices. Overall, the process recommendations focused on acquisition processes, business practices, cooperation and collaboration, information sharing, and policy and legislation. Multiple Recommendations on Technology 43 Rapid technological advances in computers, networks, M&S, and collaboration technologies have been the drivers for the expanded use of M&S in acquisition. Almost all of the 10 studies reviewed included recommendations for the application and further development of these technologies, although several concluded that technological advance is not the critical factor in advancing the use of M&S in acquisition. There was consensus among the studies that the development and application of standards is essential to the effective use of M&S- especially standards that promote interoperabi]ity among models, simulations, and databases. The studies recommended standardization of underlying data schema and protocols for information exchange. Although progress has been made by DOD (for example, HLA) and by industry (for example, the Standard for the Exchange of Product Model Data, or STEP), additional standards are needed, as well as a more general application of existing standards. Several studies recommended the development of architectures or frameworks that would promote collaboration and the reuse of models and simulations. Recommendations ranged from general collaborative environment architectures to specific product data representation structures and model and information repositories. in addition, it was noted that DOD system performance analyses require architectures that include access to common threat and environment representations. Although the studies agreed that many models and simulations already exist, several called for new or improved representations. Areas of emphasis included total- ownership-cost models and models representing new warfare areas such as information warfare and operations other than war. Finally, the studies acknowledged that certain technical problems related to M&S have not been solved and that a need exists for additional basic and applied research. Among the areas recommended for additional research were data security technologies, especially to accommodate multilevel security and to protect proprietary data; multiresolution modeling, including the aggregation and disaggregation operations within such models; and models of human behavior. Overall, the technology recommendations focused on tools, standards, architectures, and protection of classified and proprietary information.

OCR for page 29
44 MODELING AND SIMULA TION IN MANUFACTURING Multiple Recommendations on Motivation Several ofthe 10 studies reviewed suggested that cultural issues related to the acceptance and adoption of new technologies and business practices may be the largest challenge in the adoption of SBA. These studies recommended that incentives be established to motivate government program managers and industry to implement collaboration and the integrated use of M&S and SBA. Few specific suggestions were made regarding the types of incentives to be established, although some specific ideas were mentioned about how best to educate the range of acquisition stakeholders, from small businesses to senior DOD leadership, on the benefits and uses of M&S. These ideas included passive techniques, such as Web pages and information repositories providing lessons learned, as well as more active techniques, such as conferences, workshops, and required training courses for the DOD acquisition workforce. The studies noted that positive measures of expected returns on M&S investments are needed to motivate DOD organizations and program managers to make the required investments. They suggested standard metrics related to improvements in cost, schedule, and performance, as well as some additional quantified benefits. Overall, the recommendations on motivation focused on education and training, incentives, metrics, and return on investment. Multiple Recommendations on Experimentation Several industry and government studies emphasized the need for experimentation to advance M&S and SBA technologies, to develop standards and infrastructure, and to gain experience in using collaborative processes and environments. Characterized as demonstrations, pilot projects, or selected programs, the studies recommended that these experiments be structured carefully with specific objectives related to developing metrics and quantifying benefits. If existing programs were selected, additional funding would be required to support the objectives specific to M&S and SBA. Overall, the recommendations on experimentation focused on demonstrations and the use of pilot and selected programs.

OCR for page 29
SIMULA TION-BASED ACQUISITION TABLE 2-1 Categories of Recommendations from 10 M&S Simulation-Based Acquisition Studies 45 Category of Recommendation Leadership Specific Recommendations Establishment of focal point (senior office or individual) Investment of funds Processes Technology Motivation Experimentation Early involvement of users Collaboration between government and industry Leveraging of commercial practices Development and application of standards Collaborative architectures or frameworks Research in data security Research in multiresolution modeling Research in aggregation/disaggregation Human behavior modeling Incentives to collaboration Education of stakeholders Metrics to measure benefits Pilot projects Selected programs NOTE: See Appendix B in this report for the titles and summaries of the objectives and major recommendations of the I O reports and application of standards. CONCLUSIONS The use of M&S in defense acquisition predates digital computers and network technologies. However, with the rapid advances in these areas, opportunities for collaboration in the development of defense systems have been created. DOD has coined the term "simulation-based acquisition," or SEA, to describe its vision and goals for an enhanced, more collaborative, simulation-aided acquisition process. Several recent and emerging DOD acquisition programs such as the U.S. Navy's LPD- 17 and SC-2 1 /DD-2 1 /DD(X) programs, the Joint Strike Fighter program, and the U.S. Army's Future Combat Systems programhave advanced and promoted the expanded use of M&S during their acquisition, but no program has yet achieved the ultimate stated vision and goals for SBA.

OCR for page 29
46 MODELING AND SIMULA TION IN MANUFACTURING Many barriers remain to more widespread use of M&S in defense systems acquisition. These barriers include inadequate allocation of resources, lack of information for acquisition program managers, lack of an integrated software systems engineering process, issues related to the protection of inte]]ectua] property rights, poor information dissemination on SEA to the broader M&S community, and insufficient education and training for the workforce. This committee's review of lO government-or industry-sponsored studies since 1994 on the subject of M&S use in system acquisition revealed 5 genera] categories of recommendation: leadership, processes, technology, motivation, and experimentation. Table 2- l summarizes the more common specific recommendations in these 5 categories from the lO studies.