Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
B Summary of 10 Acquisition-Related Studies on Modeling and Simulation During the 1 990s, U.S. government agencies and industrial organizations sponsored a large number of studies in the general area of modeling and simulation (M&S) as it related to the manufacture and acquisition of military weapons and other equipment. These studies addressed topics ranging from narrowly focused design and manufacturing methods to more broad-based strategies for simulation-based acquisition (SBA). The Committee on Modeling and Simulation Enhancements for 21St Century Manufacturing and Acquisition selected 10 studies of acquisition-related M&S for in-depth review and discussion. These 10 studies were all formally commissioned and published in 1994 or later. This appendix presents a summary of the objectives and major conclusions and recommendations of each. NAVAL RESEARCH ADVISORY COMMITTEE REPORT In 1994, the Naval Research Advisory Committee (NRAC) performed a study on future uses of M&S for the U.S. Navy (NRAC, 1994~. The study, sponsored by the Assistant Secretary of the Navy for Research, Development and Acquisition, had as its general objective the assessment of the importance of high-fidelity models and advanced 147
148 MODELING AND SIMULA TION IN MANUFACTURING distributed simulation (ADS) technologies to enhance the Nepal lenient of the Navy (DON) test and evaluation and acquisition programs. The five specific goals ofthe study were these: (1) review current utilization of M&S and ADS technologies in the DON; (2) evaluate the strengths and weaknesses of M&S and ADS technologies from the DON perspective; (3) recommend specific research areas related to M&S and ADS technologies that would warrant DON investment; (4) identify key areas that would benefit from an investment in M&S and ADS technologies; (5) and identify candidate demonstration projects to evaluate the utility of M&S and ADS technologies. The study panel was chaired by Dr. Delores Etter, then of the University of Colorado, and included members from both industry and academia. During a three-month period, the panel held a series of meetings and gathered information from industry presentations as well as briefings held at U.S. Navy, U.S. Army, and U.S. Air Force facilities. The panel's report focused on two emerging simulation technologies: ADS and simulation-based design/manufacturing. The panel coined the term "distributed simulation-based acquisition" (DSBA) to describe the capability represented by the merger of tools from these two areas. The study panel envisioned the following conceptualization of DSBA: . A single database to perform simulations to verify product performance, develop design parameters, and address manufacturing concerns Linked simulation tools . . . in all phases of acquisition Inclusion of the operational community . . . early and continually during the acquisition process The DV phase and the EMD phase . . . collapsed into a single phase An integrated modeling and simulation culture and its attendant tool set provide the technical means to pursue concurrent engineering Multi-disciplinary teams will concurrently operate on identical or linked databases performing the following functions: operational concepts; threat definition; requirements tradeoffs, systems definition, training and logistics, and production process development; production; testing; and P3I and upgrades (NRAC, 1994, pp. 45-50) The panel made the following recommendations regarding policy: · Executive agent leadership of the Navy be vested in a position that spans all warfare areas
APPENDIX B technology: The principal focus of the DoN modeling and simulation policy should be to formulate a distributed simulation based acquisition program A technology base investment strategy is required to leverage new developments in those fields through cooperative programs with ARPA, DMSO, Joint Programs, industry, and academia (NRAC, 1994, pp. 59-60) The panel made the following recommendations regarding · Exploit industry developments in simulation based design/manufacturing develop connectivity-ready models, databases, and architectures for Naval unique advanced distributed simulation problems · develop new technology for model reality checking, evaluation and comparison (NRAC, 1994, p. 61) Finally, the panel recommended that the DON evolve DSBA technology through pilot programs, with several candidates named in aircraft, ships, mine countermeasures, sea-based theater ballistic missile defense, and ship self-defense. 149 Although no evidence indicates that the DON implemented any of the specific recommendations made by the NRAC panel, the committee believes that the work of this panel had an impact on later reports. The NRAC panel's conceptualization of DSBA contains most of the technical elements found in later DOD versions of SBA. In addition, the DOD acquisition process approved in 2000 provides for flexibility in collapsing phases of the acquisition process along the lines envisioned by the panel. NAVALAIR SYSTEMS COMMAND STUDY In 1995, the Naval Air Systems Command (NAVAIR) undertook a study focused on collaborative virtual prototyping (CVP) for the common support aircraft (CSA) initiative (NAVA1R, 1995~. The study acknowledged the NRAC (1994) report and was performed in close coordination with the study sponsored by the North American Technology and Industrial Base Organization (NATIBO) discussed below. The NAVAIR study had two specific objectives: to assess the ability and readiness of the aircraft and electronics industrial base to use CVP technologies in the acquisition of CSA, and to identify program management and acquisition actions required in order for government and
150 MODELING AND SIMULATIONIN MANUFACTURING industry to realize the potential productivity gains and cost savings offered by CVP technology. Members of the NAVAIR study team together with members of the NATIBO study team visited 57 organizations between March and September 1995. The NAVAIR study focused on technology assessment, business process reengineering, and demonstrated benefits. The following conclusions and recommendations were made regarding the application of Cvp tec.hn~loov __.,,,_._z,, . There exists a wealth of commercially available products and services to support immediate implementation of a CVP environment for the development of new products The DoD, DoC, NSF, and DoE are developing an infrastructure and a host of collaboration tools There are aircraft specific applications and technologies being matured by the JAST program. These efforts should be leveraged for the development of the common support aircraft Standards are the key element to all distributive enterprise activities The majority of existing models and simulations needed to perform warfare analysis have not been developed to operate in a distributive computing environment · Producibility is a life-cycle cost driver. There are numerous advanced manufacturing programs within the DoC, DoE, and NASA. The DoN should leverage these programs to provide the processing models needed (NAVAIR, 1995, p. 5-1) The following conclusions and recommendations were made regarding business processes: · The commercial sector is rapidly developing tools for distributed computing and virtual prototyping Incorporation of the customer as a member of the IPPD team significantly reduces the development time since non-value- added activities can be minimized. New information and distributed computing technologies have spawned the formation of many small innovative companies. ARPA Electronic Commerce Resource Centers are educating small to medium size firms in the use of electronic commerce. NAVAIR should leverage these programs by working through primes to assist in modernizing critical suppliers. NAVAIR should investigate the benefits of using commercial business practices in revolutionizing the acquisition process. (NAVAIR, 1995, p. 5-2) i] The following conclusions and recommendations specific to the CSA nitiative were made:
APPENDIX B The Naval Aviation Team should develop a strategy and plan for adopting CVP technologies (SBD) and associated business practices. The CSA initiative should leverage the newly established NAVAIR M&S executive committee to survey existing models and simulations that will be applicable to the CSA initiative. The models and simulations should be categorized according to their functional discipline. Developments within the S&T community should be focused to achieve an affordable CSA. CVP technology should be used to facilitate the partnership between government and industry. [D]ARPA and ONR technology efforts should orient their testing/demonstrations to support the CSA initiative. . 151 S&T investments should be made in process technologies that reduce cost. S&T should invest in CSA unique engineering and warfare analysis tools. · CVP technology should be used to facilitate the partnership between government and industry. The CSA IPT [integrated product team] should rapidly adopt and transition the successful technology and business practices from JAST [joint advanced strike technology program]. (NAVAIR, 1995, pp. 5-3 and 5-4) Although focused specifically on the CSA initiative, the NAVAIR (1995) study highlighted issues related to business process reengineering and to partnerships and sharing between government and industry. The themes of partnership and sharing, particularly as they pertain to industry involvement earlier in the acquisition process and to the question of proprietary rights, are reflected in subsequent studies. NORTH AMERICAN TECHNOLOGY AND INDUSTRIAL BASE ORGANIZATION STUDY The North American Technology and Industrial Base Organization commissioned a CVP study (NATIBO, 1996) to assess the maturity, level of use, utility, and viability of CVP technology and its application to the industrial base, including both small and medium-sized companies. The NATIBO report provides an overview and assessment of CVP technology, a discussion of system development and acquisition processes using CVP, a presentation of case studies demonstrating the uses of CVP, and a
152 MODELING AND SIMULA TION IN MANUFACTURING discussion of required investments and expected payoffs. The report also lists technical, financial, procedural, cultural, and policy facilitators and barriers to the use of CVP technology. The NATIBO (1996) study produced the following conclusions with respect to the technical, business, and political environment associated with CVP at that time: · Industly recognizes opportunities offered CVP · CVP technologies exist and are advancing . No true CVP environment currently exists · No metrics are in place for measuring CVP benefits · Proprietary data rights and protection of competitive advantage are key industry concerns · No government guidelines for CVP use have been set Current government acquisition procedures do not promote CVP CVP standards and better integration of tools are needed. Financial investment is considerable for small companies No central repository of CVP information currently exists Model validation process takes too long (NATIBO, 1996, pp. 48-49) Based on the conclusions listed above, the report outlined the following 10 recommendations: establish a central government office for CVP sponsor integration and demonstration projects implement policy to develop standardized metrics for evaluating CVP payoffs in programs; implement request-for-proposal (RFP) language and contracting approaches that encourage CVP use; reevaluate how developers deliver data to the government coordinate CVP requirements with acquisition reform initiatives; address data security and proprietary data concerns and formalize policy regarding these issues; target government investments on CVP integration technologies; streamline the validation process for models; and educate small businesses on less expensive options to acquiring CVP technologies. (NATIBO, 1996, pp. 52-53) Although the NATIBO (1996) study was also focused on CVP technologies, it highlighted many more general SEA issues than the NAVAIR (1995) study had specifically, issues related to industry concerns and steps that the government could take to address these concerns. Issues of proprietary data rights, required investments, and the need for metrics to help support a business case for implementation were
APPENDIX B 153 prominent. This study was also the first to recommend a central government office at the level of the Office of the Secretary of Defense to coordinate policy and to act as a source of information. AMERICAN DEFENSE PREPAREDNESS ASSOCIATION STUDY In 1996, the Undersea Warfare Division of the American Defense Preparedness Association (ADPA) performed a study on the application of M&S to the acquisition of major weapon systems (ADPA, 1996~. This study was sponsored by the U.S. Navy Acquisition Reform Executive and included industry, government, and university participants. The context of the study was the development of a hypothetical Total Ship Integrated Combat System for an Advanced Surface Combatant that might be authorized post-2005, with a focus on the undersea warfare components of such a system. The study assessed the potential of achieving a 50 percent reduction in cycle time from the definition of military needs to the achievement of initial operating capability, and it assessed the potential for making similar reductions in life-cycle cost, considering both the technical and business processes. The study objective was to detennine the merits and benefits of an SBA approach for major weapon systems, addressing the technical merit of proceeding with SBA methods, the business integrity issues associated with such an approach, and the changes necessary in the contacting and procurement system processes necessary to facilitate such an approach. The ADPA (1996) study reached the following conclusions: . Modeling and simulation tools, as well as new processes such as integrated product and process development . . . are already being applied in a range of ongoing acquisition programs. · The challenge for acquisition reform is to provide the catalyst that will expand this growing successful application of M&S tools beyond vertical applications within programs so that cost savings benefits can be realized by sharing data, tools, and techniques between different acquisition programs. The appropriate vehicle for meeting this challenge is simulation- based acquisition The new SBA culture . . . is predicated on mutual trust between government and industry. Program managers already have considerable flexibility. .
154 MODELING AND SIMULA TION IN MANUFACTURING · Decision makers using the SBA process will have the ability to make smarter, faster, more informed decisions which will save time and dollars throughout the life cycle. (ADPA, 1996, pp. 5-3 - 5-7) Based on the above conclusions, the ADPA study team formulated the following recommendations: . . The government should firmly establish SBA as the preferred manner of conducting IPPD-style acquisition and should establish incentives for both government program managers and industry to ensure full and enthusiastic participation; Carefully designed pilot programs, structured as engineering experiments with objectives and metrics, can demonstrate the utility of SBA to the acquisition community and stakeholders and thereby catalyze the cultural change that is required; to do so: Pilot programs should be augmented with necessary additional Finds and should be focused not just on M&S tools, but on the entire SBA process; Metrics should address the building of a program-to-program infrastructure that builds on the ongoing DOD investments in M&S; and The government should provide open access to government infonnation and standard models to the appropriate industry participants in He pilot programs; · The government should re-direct DOD investment in M&S to support and encourage development of an SBA-specific infrastructure, the high level architecture (HLA) and other infrastructure components being developed by DOD are necessary but insufficient for the realization of SBA; in addition, the following should be pursued: . Development of a comprehensive system data schema that provides for common representation and data interchange mechanisms between government and industry standard databases; and Development of key tools that would build on the HLA and act as a jump-start to the pilot efforts; examples include a database repository tool and common data library and M&S analysis tools. (ADPA, 1996, p. 5-14) No evidence indicates that the U.S. Navy Acquisition Reform Executive took specific actions in response to the recommendations of the ADPA (1996) study. However, some ofthe concepts originated in the study (for example, SBA as a combination of technical, process, and cultural elements; and the need for an SBA infrastructure to benefit
APPENDIX B 155 multiple programs) can be found in subsequent industry and government- sponsored studies. DIRECTOR FOR TEST SYSTEMS ENGTNEERING AND EVALUATION STUDY The Director for Test, Systems Engineering and Evaluation (DTSE&E) in the Office ofthe Secretary of Defense (OSD) commissioned a one-year study, also completed in 1996, to assess the effectiveness of the use of M&S in the weapons system acquisition and support processes (DTSE&E, 19961. The study team was asked to investigate metrics being used to evaluate M&S effectiveness; specific tools being used by government and industry to facilitate the design, development, test, manufacture, and support of weapon systems in an IPPD environment; the benefits associated with using M&S in the acquisition environment; and the technical challenges that could preclude the seamless use of M&S in the acquisition process. The DTSE&E study team reviewed previous studies and visited individuals from OSD, government program offices and research and development (R&D) and test and evaluation (T&E) centers, and several industry organizations. The DTSE&E study team produced the following findings: · There is a recognized need for technology to be used by the acquisition community as it reengineers itself into a team-based approach; M&S tools and processes are being used eff~cient]y and effectively in each of the services, but not in an integrated manner across programs or functions within the acquisition process; The words are in place in DOD acquisition documents to support implementation of SBA, though there are some growing pains associated with implementation. (DTSE&E, ] 996, report pages not numbered) . Based on data gathering, the DTSE&E study team formulated the following five recommendations: . Institutionalize the use of M&S and ensure that the community is knowledgeable about the tools available. The services and OSD must provide more responsive guidance relative to the advent of better and more useful simulation tools. Dialogue is needed within the services and between the services and OSD to effect policy on standardization. Program managers must
156 MODELING AND SIMULA TION IN MANUFACTURING . overcome the management and cultural challenges that present barriers to the effective use of available technology. Provide focused information on the availability and capabilities, including success stories, of M&S to weapon system acquisition managers. To meet the challenge of institutionalizing the use of available technology, the services must be committed to providing funds for M&S at the inception of the program. The OSD and services should commit science & technology dollars to upgrade capabilities and facilities that could serve many weapon system acquisitions. Program managers should be encouraged to use these facilities and capabilities instead of contracting to have their own system-specific facilities and tools built. · Develop an information source such as an Internet web page that would list capabilities in design, the tools available, the programs that have used them, and individuals that can be contacted for further information. The same capabilities could be listed for testing. The web page could be used to identify innovative approaches in manufacturing and note those using virtual manufacturing environments. Opportunities to cooperate with industry, such as the DARPA simulation-based design program, should be encouraged and continued. There appears to be great potential in partnerships such as the National Automotive Center, where both the government arid industry benefit from investigating new technology. Incentives to pursue business relationships such as these should be developed to use developing technology more efficiently. (DTSE&E, 1996, report pages not numbered) In addition to providing examples of cost savings and cost avoidance that resulted from the use of M&S in acquisition, the DTSE&E (1996) study reinforced some of the conclusions and recommendations of prior studies. It identified cultural and managerial issues, as distinct from technical challenges, as perhaps the more difficult obstacles to overcome in the institutionalization of M&S and the use of SBA in the DOD. 1997 NATIONAL RESEARCH COUNCIL STUDY The Naval Studies Board of the National Research Council (NRC) performed a study for the U.S. Navy and Marine Corps that resulted in the 1997 publication of a multivolume report. One of the volumes of this report was focused on M&S (NRC, 1 997b). The M&S panel of the NRC study was composed of members from academia, government-sponsored
APPENDIX B 157 centers, and the defense industry. The original goal of the study was to review the overall architecture of M&S within DOD (including the DON, the Joint Chiefs of Staff, and OSD), the ability of models to represent real- world situations, and the merits of models as tools for making technical and force composition decisions. After a preliminary review of existing documents, the M&S panel focused its work more narrowly on the following objectives: clarifying why senior levels of the DON should be concerned about the substantive content and comprehensibility of M&S; assessing what the DON and DOD might need to do in order to benefit fully from the opportunities presented by M&S technology, clarifying what M&S can and cannot be expected to accomplish in aiding decisions on technical, force-composition, and operational planning issues; and establishing priorities for M&S-related research. The following conclusions and recommendations regarding M&S in general were made in Volume 1 (NRC, 1997a): · M&S demands the attention and support of top DON command and management levels because it affects every aspect of military force design, equipment, and operation. A new corporate management approach is required if naval forces are to capitalize fully on the benefits that M&S can offer. This approach is needed to ensure compatibility, consistency, and seamless interfaces between the U.S. Navy, U.S. Marine Corps and joint service approaches to using M&S; coordinate inputs to the joint services M&S projects (e.g., the Joint Simulation System (JSIMS) and the Joint Warfare System (JWARS)~; and ensure that existing simulations are upgraded or replaced. (p. 18) The conceptual foundation of M&S must be brought up to date and include knowledge of how modern warfare is and could be fought. Currently, there is a dearth of theoretical understanding and knowledge of modern post-Cold War types of warfare based on collected and analyzed data. There is also a dearth of model validation that compares the results of models describing warfare with the outcomes of actual conflicts or even of field exercises. Finally, there are no credible methods for model validation. (p. 94) An ongoing M&S research effort is needed that is focused on military science and technique; that includes simulation science and technology applicable to military systems and operation; and that includes the construction and maintenance of databases covering worldwide military forces and environments, organized by warfare area. (p. 95) . . In the M&S volume of the NRC report (NRC, 1 997b), the M&S panel made additional recommendations:
158 MODELING AND SIMULA TION IN MANUFACTURING · The DON should take an active role to ensure that JSIMS and JWARS are produced as evolving systems that incorporate future research results. · The DON, working with the other services and DOD as . appropriate, should establish a robust but focused program in research including both warfare-area research and research on fundamental theory and methods. The DON should establish processes that ensure early scientific review of models emerging from research, a competitive atmosphere in which model users are encouraged and assisted in constantly evolving their M&S technologies to represent the best available knowledge, and a general emphasis on quality, including the ability to represent uncertainty. · The DON should treat SBA as a key enabling technology with extraordinary long-term leverage and should organize and invest consistently with that enterprise-technology view. (NRC, 1997b, pp. 2-3) The NRC M&S panel made these additional recommendations regarding joint models: . · The DON should insist that the DOD and program offices adopt open-architecture attitudes that will promote rather than discourage substitution of improved modules as ideas arise from the research and operations communities. The DON should insist that they build explicit and well-exercised mechanisms to ensure that such substitutions occur. The DON should advocate an approach to joint model development that has a long-haul view and an associated emphasis on flexibility. (NRC, 1997b, pp. 8-9) Regarding research, the NRC M&S panel made the following recommendations: . . The U.S. Navy and U.S. Marine Corps should select a few high- priority warfare areas and create research programs to support them, including: expeditionary warfare and littoral operations; joint task force operations with dispersed forces; long-range precision strike against forces employing countermeasures; theater missile defense, including counterforce and speed-of- light weapon options, against very large ballistic missile and cruise missile threats; and short-notice, early-entry operations with opposition. Research on the following areas of modeling theory should be given priority: multi-resolution modeling, integrated families of models, and aggregation-disaggregation; agent-based modeling and generative analysis; and semantic consistency.
APPENDIX B · Methodological advances are needed in the areas of characterization of uncertainty and in exploratory analysis under uncertainty. · Infrastructure is needed in the areas of: intellectual infrastructure, for example, M&S theory, texts, case studies, software engineering methodologies, "virtual centers," journals, and conferences; object repositories and interface standards to enhance reusability and composability; explanation and traceability capability; and tools, such as automated scenario generation and experimental design, and post-processing and data analysis. (NRC, I 997b, pp. 10-16) 159 Finally, the NRC M&S panel made the following recommendations regarding assimilating and exploiting M&S: . The DON should make a strategic commitment to the success of exploiting M&S. The DON should make an increased investment in the education of M&S practitioners, those who acquire and design M&S tools, and those who rely on them to guide acquisition, training, and operations. (NRC, 1 997b, p. 17) Although the 1997 NRC report was oriented toward M&S in general as it was used by the DON, most of the recommendations also apply to the specific area of M&S for acquisition. Compared with other reports published in the late 1990s, the NRC report placed more emphasis on research investment and emphasized the subject matter content of M&S as opposed to the enabling information technology. JOINT SIMULATION-BASED ACQUISITION TASK FORCE STUDY In 1998, the Acquisition Functional Area Council of the DOD Executive Council for Modeling and Simulation commissioned the Joint Simulation-Based Acquisition Task Force (SBATF) to develop a road map for DOD action on SBA, using the vision for SBA as an acquisition process in which DOD and industry are enabled by robust, collaborative use of simulation technology that is integrated across acquisition phases and programs (SBATF, 1998~. It is worth mentioning that two other SBA studies were completed during approximately the same period. The SBA Industry Steering Group (SBAISG) conducted a short-term study on SBA, reported the results to the Acquisition Functional Area Council in late 1997. In addition, three military research fellows at the Defense Systems
160 MODELING AND SIMULATIONIN MANUFACTURING Management College (DSMC) performed a survey of government and industry SBA activities during the years 1997-1998 (Johnson, et al., 19981. The SBATF's objectives were as follows: Development of notional representations of the operational, systems, and technical architectures needed to establish SBA environments; Identification of technical challenges, rough estimates of time and cost to develop solutions, and opportunities for reuse across programs; Identification of the primary ownership of each module In the systems architecture; Identification of needed rough-order-of-magnitude investments by government and industry, and possible methods to determine return on investment; Development of a road map of near- and long-term DOD actions needed to develop the SBA concept; and Identification of industry actions to accelerate the SBA concept. The SBATF met over a 6-month period, during which the task force performed its own research, obtained input from the stakeholder community, and conducted a t`vo-session structured decision exercise using the quality function deployment (QFD) process to prioritize recommended actions with 45 government and industry participants. The SBATF's final report was reviewed first by the Acquisition Functional Area Council and then by an industry-sponsored conference convened for that purpose in late 1998. The SBATF (1998) report included 24 high-level recommendations, which are reprinted or briefly described below in six major categories: (l) management, (2) architecture, (3) policy, (4) legislation, (5) education, (6) training. The SBATF made the following recommendations regarding management: . . Establish the DOD M&S Acquisition Council as the proponent for SBA integration and utilize the existing AFWG for support. Establishment an SBA AD&S [architecture development and standards] group to coordinate the development of the SBA architecture; Designate the DTSE&E as the OSD champion. DTS&E will plan, coordinate, integrate, and support SBA initiatives and budget requests during the PPBS [planning, programming, and budgeting system] process;
APPENDIX B . . · Designate component lead organization for SBA PPBS. Identify and document resource requirements for accomplishing SBA road map tasks. · Establishme management support for involvement in program manager M&S planning efforts; · Develop and document a flexible verification, validation, and accreditation (VV&A) process for models Develop metrics to track SBA benefits and return on investment across acquisition phases and programs., Metrics should . . . provide indication of cost, schedule, and performance improvements attributable to SBA; and Provide incentives to OSD and service/agency senior leadership, program executive officers, program managers, and industry to implement SBA. (SBATF, 1998, pp. 7-14 - 7-23) The SBATF made the following recommendations regarding architecture: Coordinate the establishment of CEs [collaborative environments] as they are formed both department-wide and within each of the services/agencies. Establish and develop a set of product area collaborative environment prototypes to test key architectural concepts. Establish and develop a mission area collaborative environment prototype to validate the. . . relationships between collaborative environments; Evolve and maintain collaborative environments to achieve the SBA vision, making maximum use of existing DOD, service/agency, and industry resources; · Establish a template for the design and a concept of operations for the use of DPDs [distributed product descriptions]2 throughout the acquisition life cycle; · Establish long-term mechanisms for the evolution and . maintenance of DPDs to support new DOD product acquisitions; Develop an integrated network of resource repositories for the models, simulations, data, and information needed to support commonality and reuse throughout the acquisition community; Identify, review, prioritize, and coordinate science and technology and research and development efforts needed to achieve the SBA vision; 161 ' The SBATF defined a collaborative environment as an enduring collection of subject matter experts supported by interoperable tools and data bases, authoritative information resources, and product/process models that are focused on a common domain or set of problems (SBATF, 1998). 2 The SBATF defined a distributed product description as a distributed collection of product- centric information that is interconnected via Web technology into what appears to the user to be a single, logically unified product representation (SBATF, 1998).
162 MODELING AND SIMULA TION IN MANUFACTURING . Identity and provide access to available environmental representations and information; Develop and maintain authoritative threat models for SBA across programs throughout the acquisition life cycle and identify and provide access to available intelligence information for SBA across programs throughout the acquisition life cycle. Upgrade DOD and commercial TOC-related models and data bases related to function within, and fully support, the acquisition process under the SBA strategy; Foster an integrated digital and virtual life cycle weapons program management system. Enhance interoperable linkages to the industrial base. Promote an aggressive use of electronic commerce; and Develop, coordinate, and sustain the technical architecture needed to achieve the SBA vision. (SBATF, 1998, pp. 7-30 - 7-45) Regarding policy and legislation, the SBATF recommended the "development of a single set of DOD SBA business practices and the implementation of a common SBA policy." The objective of these actions would be to reach the goals of the National Performance Review, Defense Systems Affordability Council, and Under Secretary of Defense for Acquisition and Technology regarding reduced cycle time, reduced total ownership costs, and increased systems performance. The SBATF made the following recommendations regarding education and training: · Continue and improve education and training courses, conferences, workshops, and symposia on a continued basis to keep DOD/industry at all levels up to date on SBA; · Identify appropriate educational forums (example, schools, courses, symposia, etc.) for all SBA stakeholders, including the requirements and training communities, warfghters, depot and maintenance personnel, the S&T community, academia, congressional staff, and international partners. Provide appropriate SBA content to be incorporated into appropriate courses, and · Provide formal SBA training to acquisition workforce personnel. Work with functional boards and work groups to evolve the DAU [defense acquisition university], Defense Systems Management College, and other appropriate curricula to integrate M&S/SBA throughout all appropriate acquisition functional area courses. (SBATF, 1998, pp. 7-50 - 7-53) As a result of the publication of the SBATF final report, the Acquisition Functional Area Council identified several SBA core issues
APPENDIX B 163 that the council wanted addressed in subsequent working-level sessions. An update ofthe SABTF (1998) executive summary was published in the summer of 1999, outlining those issues and how they had been addressed by the council's working group. SABTF (1998) was not formally adopted as guidance by the Acquisition Functional Area Council, although it remains a reference document. No DOD action has resulted from the report, although some of the technical concepts presented in the report (for example, CEs and DPDs) have been used in M&S planning by the Joint Strike Fighter program, and use of DPDs in programs has been adopted as policy by the U.S. Air Force in an acquisition instruction (USAF, 2000~. DEFENSE SCIENCE BOARD TASK FORCE STUDY In 1997, the Under Secretary of Defense for Acquisition and Technology requested that the Defense Science Board (DSB) commission a task force to study the use of advanced M&S in analyzing combat concepts in the 21 st century. The task force focused, therefore, on M&S for the front end of the acquisition process and issued its report in 1999 (DSB, 1999~. The task force, co-chaired by Dr. Ted Gold and General Larry Welch, USAF (retired), was composed of senior technical civilians and retired flag-level officers. The work was completed over a 2-year period, during which the task force visited or received briefings from more than 40 military and industrial organizations. The DSB task force was asked to address the following questions: . How do we want to conduct analysis in the future and what should be the interrelationship between constructive, live, and virtual simulations? What methodological and procedural changes are required to improve DOD analysis? · How do we provide warfighters and force providers with concept prototyping capability at various levels? What tools are needed to perform these analyses? What tools are currently available? What technologies are required to build needed tools? · What DOD organizational and policy changes are needed to research and effectively implement new analysis approaches? The DSB task force generated the following conclusions: There is extensive and successful use of M&S for training, weapon system design trade-offs and evaluation, and engineering simulations; however
164 MODELING AND SIMULA TION IN MANUFACTURING . · Only rudimentary applications of M&S [were found] in the exploration of new warfighting and operational concepts; · Characteristics and attitudes essential to analysis for innovation are not particularly welcome in much of the analysis and simulation community; and · Infonned, involved customers are needed, particularly in the joint community; insight, not advocacy, is needed. · Changes are needed in organizations and processes to make better use of the potential of existing simulation capabilities: · Some organizations [were found] that reflected the attitude and orientation needed, but they were focused on service capabilities ends operating environments, not on joint operations; and · Although there are organizations and centers that have potential for becoming joint centers of excellence for exploring innovative concepts arid capabilities, well-supported charters and expectations are needed. · Basic shortfalls include: · Key elements of Joint Vision 20]0 are not addressed, for example, information warfare, situational awareness, dispersion of forces, maneuvers over strategic distances, dismounted combatants with unprecedented potential, urban operations, and new command and control arrangements; · Lack of clear plans for evolution of models arid systems of models to address these needs; and · M&S and federations of simulations must accept new concepts and doctrines far more readily; for the exploration of new concepts, flexibility becomes a higher-valued M&S feature than fidelity. The customer needs to be the driving force, because: · The current approach is to give resources to developers to build better tools; · General-pulpose models can rarely bear the weight of important decisions or deal with the unfamiliar; and · M&S customers need to have control of significant resources and exercise more direct responsibility for details of defining and overseeing the product they need. (DSB, 1999, pp. 8-10) On the basis of the conclusions listed above, the DSB task force made three recommendations dealing with process improvement and one recommendation dealing with mode] improvement. The recommendations regarding process improvement were as follows: Recommendations for the Joint Chiefs of Staff (JCS) and the Joint Requirements Oversight Council (JROC): · The Chairman, Joint Chiefs of Staff (CJCS) should identify several critical enablers and operational concepts within Joint Vision 20] 0 to be used as the focus for simulation;
APPENDIX B . . The JROC should issue a requirement for joint simulation environments specifically focused on examining innovative concepts and systems and should drive significant M&S resources to this need; The Joint Chiefs of Staff should be a lead, demanding, involved customer for products; and · The CJCS should task the joint and service schools to develop courses for military and senior civilians on how to be effective customers for M&S services. Recommendations for joint focus, i.e. the Joint Chiefs of Staff and the Under Secretary of Defense for Acquisitions and Technology: To focus on analytical and simulation support for joint innovative concepts and systems, the JROC needs to provide consistent support for centers of excellence dedicated to this purpose · Such centers would be part of, or at least directly connected to, a CINC with responsibilities for the joint world similar to the responsibilities the Army's TRADOC has within the Army; There should be a heavy emphasis on . . . experimentation to discover what works and what doesn't before heavily investing; and . A small group, specifically charged to think out-of-the box, is needed to explore key facets of JV2010. Recommendations for organizing joint operational architectures: . . The JROC should continue to support and leverage the Joint Theater Air and Missile Defense Organization experiment to provide a coherent joint operational concept and architecture for air and missile defense; and The lessons learned should be applied to other areas needing joint operational and technical architectures. (DSB, 1999, pp. 35-37) The following recommendation was made by the DSB task force regarding mode] improvement: . 165 The customer community should take a far more active role in defining simulation needs and setting priorities. For example: The Joint Chiefs of Staff and the services should require that all efforts examining new concepts also identify supporting M&S . . . priorities; The Joint Chiefs of Staff and the Director, Defense Research and Engineering (DDR&E) should increase demands that
166 MODELING AND SIMULA TION IN MANUFACTURING simulations address the human element and Joint enablers that drive operational outcomes. · The JCS, DDR&E and PA&E should continue support for JSIMS and JWARS, but require specific plans to more fully incorporate drivers of effectiveness. (DSB, 1999, p. 38) Since the publication ofthe DSB (1999) report, additional emphasis has been placed by the defense simulation community on experimentation and the representation of human behavior. However, there is no evidence that any progress has been made toward implementing the process and model improvements recommended by the task force. 1999 NATIONAL RESEARCH COUNCIL STUDY In 1998, the National Aeronautics and Space Administration (NASA) asked the NRC to undertake a study on advanced engineering environments (AEEs). The study committee, composed of members from industry and academia, was given six tasks, with the objective of developing steps that NASA could take in the short term to enhance the development of AEE technologies. The six tasks were as follows: . . Develop an understanding of NASA's long-term vision of AEE, capabilities, and tools associated with the current state of the art and short-term advances in engineering environments; · Conduct an independent assessment of requirements for, alternative approaches to, and applications of AEEs to aerospace engineering, considering both short- and long-term objectives; · At a high level, explore the potential payoffs of AEEs on a national scale, emphasizing the relationships between aerospace engineering and other elements of the national engineering scene and identifying the necessary conditions for achieving these payoffs; · Evaluate how AEE technologies relate to the development of relevant technical standards and engineering economic assessments; · Identify cultural and technical barriers to collaboration among the government, the aerospace industry, academia, and others for transferring AEE tools and methods from the development stage to public practice; opportunities that may be created by AEEs; and needs for education and training; and Recommend an approach for NASA to enable a state-of-the-art engineering environment capability that is compatible with other government, industry, and university programs and that
APPENDIX B 167 contributes to the overall effort to engender a broadly applicable, technology-based, engineering framework. (NRC, 1999a, p. 41) The committee collected information on AEEs from government, industry, and academic organizations that were involved as developers, providers, or users. The first phase of the study produced a report in 1999 (NRC, 1 999a). Based on the information collected, the committee on AEEs defined the following vision: "AEEs should create an environment that allows organizations to introduce innovation and manage complexity with unprecedented effectiveness In terms of time, cost, and labor throughout the life cycle of products and missions" (NRC, 1 999a, p. 2~. The NRC committee produced 6 findings and 13 recommendations in 4 major categories: (1) a historic opportunity; (2) requirements and benefits, (3) barriers; and (4) organizational roles. The committee found that a historic opportunity exists to develop AEE technologies and systems that could revolutionize engineering processes, but this opportunity is too big for any single organization to realize. The committee made these recommendations: · a government-industry-academic partnership should be formed. This partnership should foster the development of AEE technology and systems and · NASA should draft a plan for creating a broad government- industry-academic partnership. In addition, . . . NASA should charter a joint industry-academic-government advisory panel that focuses on interactions between NASA and external organizations. (NRC, 1999a, p. 34) Regarding requirements and benefits, the NRC committee's findings were as follows: Current AEE R&D is too diffuse and should be focused on: · enabling complex new systems, products, and missions · greatly reducing product development cycle time and costs AEE developers should devise an implementation process that lowers technical, cultural, and educational barriers and that applies AEEs broadly across government, industry, and academia; the top-level goals that NASA has established for the intelligent synthesis environment functional initiative address important AEE requirements. However, given the resources that NASA plans to allocate, the objectives are overly ambitious; and NASA should establish an AEE "center of gravity" that is empowered to select the high-priority analyses and processes that will be developed, integrated, and deployed as a mission
168 MODELING AND SIMULA TION IN MANUFACTURING design system. To ensure success, the location, leadership, and staff should be carefully selected to reflect the differing needs, capabilities, and perspectives of NASA's operational and research centers. (NRC, 1999a, p. 35) With respect to barriers, the NRC committee found that efforts by industry and government to develop and deploy AEEs faced significant barriers in these areas: integration oftools, systems, and data; information management; cultural, management, and economic issues; and education and training. Regarding barriers to the integration of tools, systems, and data, the committee stated: . . A practical approach must be developed for improving the interoperability of new product arid process models, tools, and systems and for linking them with legacy tools, systems, and data. Sponsors of AEE research and development should consider integration of AEE product and process models; and Government agencies and other organizations with a large stake in the successful development of AEEs should interact more effectively with standards groups to facilitate the development of interoperable product and process models, tools, systems, and data, as well as open system architectures. (NRC, 1999a, p. 36) Regarding barriers to information management, the NRC committee found a lack of commonality in product and process descriptions within and among user organizations and between users and suppliers, and that the need for customization greatly reduces the cost-effectiveness of new tools. The committee therefore recommended that corporate and government leaders develop robust and flexible AEE tools for creating, managing, and assessing computer-generated data; for presenting relevant data to operators in a clear and efficient manner; for maintaining configuration management records; and for storing appropriate data on a long-term basis. Regarding cultural, management, and economic barriers, the NRC committee found that, historically, not enough attention has been paid to the organizational, cultural, psychological, and social aspects of the user environment associated with AEE technologies. It recommended that AEEs be integrated into the senior management culture of organizations investing in AEE technologies and systems; that each organization designate a champion with responsibility, authority, resources, and support from a team of senior managers, technical experts, and other critical stakeholders. Similar subordinate teams should be assembled in major organizational elements or facilities involved. Regarding barriers to education and training, the NRC committee found that government agencies have frequently used contract provisions
APPENDIX B 169 to influence contractor business practices and, occasionally, engineering practices. Therefore the committee made the following recommendations: . . . Government agencies involved in the acquisition of complex engineering systems should provide incentives for contractors to implement appropriate AEE technologies and systems and to document lessons learned. These incentives should target both technical and nontechnical . . . aspects of AEE development and implementation; NASA should define an agency-wide plan for the development and implementation of comprehensive, improved engineering processes, practices, and technologies; NASA-wide teams directing the Intelligent Synthesis Environment functional initiative should be consolidated and strengthened; An advisory panel with representatives from industry, universities, the National Science Foundation, NASA centers, and other government agencies and laboratories should be convened by NASA . . . This panel should define incentives for accelerating incorporation of AEE technologies into the engineering curriculum, define the basic elements of a suitable AEE experience for students, and specify resource needs. (NRC, 1999a, pp. 37-38) Finally, regarding organizational roles, the NRC committee made the following recommendations: · AEEs should use commercially available tools as much as possible. In general, the development of application-specif~c tools should be left to industry. If commercial tools are inadequate, government agencies should create incentives for commercial vendors to develop improved, broadly applicable tools. · To maximize the effectiveness of [its own AEE R&D and that undertaken by other organizations], NASA must improve its understanding of the capabilities and requirements of external organizations. (NRC, 1999a) It is too early to assess the degree to which the recommendations of the NRC (1 999a) report have been implemented by NASA. However, it is important to note that the NASA-sponsored ISE initiative, which had objectives similar to those of DOD's SEA initiative, ceased to exist as a separate NASA program in early 2001.
170 MODELING AND SIMULA TION IN MANUFACTURING MILITARY OPERATIONS RESEARCH SOCIETY REPORT In 1997 and 1998, the Military Operations Research Society (MORS) conducted two workshops, known as "SIMTECH 2007," and produced a report on the results in April 2000 (MORS, 2000~. The main goal of the workshops was to promote more effective dialogue between the M&S technology community and M&S users, such as analysts, acquirers, educators, and trainers. Four subordinate workshop objectives were: (1) to review and assess the findings and recommendations from SIMTECH 97, a set of workshops held about a decade earlier; (2) to identify and prioritize military M&S user needs; (3) to assess the probable evolution of M&S technology over the next decade; and (4) to identify opportunities for addressing user needs. Within the workshops, there were several working groups, including one on acquisition, the results of which are described below. The MORS acquisition working group was asked to characterize the acquisition process of the year 2007; to identify major changes that must occur in order to bring about this acquisition process; to identify shortfalls in M&S; and to prepare actionable recommendations to address the shortfalls in investments and incentives, and to address policy and organization. The working group adopted an acquisition vision and goals statement similar to the SBA vision and goals statement promulgated in 1997 by the Acquisition Functional Area Council of the DOD Executive Council for Modeling and Simulation (EXCIMS) and used as a starting point by the SBATF Joint SEA Task Force. The working group characterized the desired end-state as including the following: increased contractor total system responsibility with more efficient government insight to allow trusted partnerships between government and industry; a highly integrated electronic work environment across all life- cycle functions, reduced life-cycle costs and development time consistent with commercial practices; and · DOD commitment to making the most informed acquisition selection and decisions based on life-cycle cost, authoritative data and model sources, collaborative M&S use, and proper JPT use. (MORS, 2000, report has no page numbers) The MORS acquisition working group identified major cultural, management, policy, and technology and environment changes that were
APPENDIX B needed in order to achieve the envisioned end-state. According to the working group, the cultural changes needed were these: . _.~, A. _..~..= A greater reliance on M&S, · Horizontal integration and breakdown of stovepipes, · Government-industry trusted partnerships; . Pervasive sharing of models and data; U-front investment in modern processes; Enabling of international involvement; · The ability to conduct comprehensive life-cycle trades; · Flexibility to accommodate a major force restructuring; and Education for the vision. (MORS, 2000) The management changes needed were as follows: · An alignment of development time to be more consistent with commercial life-cycle times and ready accommodation of technology insertion and turnover; · Business process reengineering of data production; · Making functional IPTs a way of life, for example, leveraging M&S across functions and domains and avoiding redundancy; · Requiring authoritative sources for models and data, for example, having a program manager provide sources for system models and other stakeholders provide sources for environment; and · DOD commitment to a life-cycle-cost basis for acquisition decisions. (MORS, 2000) According to the MORS acquisition working group, the policy changes needed were these: Up-front investment as the norm to reduce life-cycle costs; Making M&S strategy integral to the total acquisition plan; Making M&S critical to formal acquisition decisions, including policy guidance on what the Defense Acquisition Board can expect and guidance to the program manager on what to provide; Incentives for all stakeholders to participate; and DOD policy and guidance on M&S use and sharing of M&S technology between government and industry and across programs. (MORS, 2000) 171 Finally, the technology and environment changes needed according to the MORS acquisition working group were these:
172 MODELING AND SIMULATIONIN MANUFACTURING · Creating a DOD-wide, knowledge-based infrastructure to enable SBA, including program-specific functional integration, appropriate use of commercial-off-the-shelf and government-off- the-shelf products, interoperability and reuse standards; · Creating capability to conduct trades across highly diverse mission and functional areas, Facilitating ease-of-use, otherwise known as cross-platform, plug-and-play, or throw-away; The development of validated data sources, models, and tools; and Investment in a comprehensive modeling capability. (MORS, 2000) The MORS acquisition working group then made a number of recommendations regarding investment and incentives, policy initiatives, organization and focus, and technologies. The working group recommended the following actions for DOD to take regarding investment and incentives: . establish and support sufficient M&S infrastructure investments in the program objective memorandum; provide incentives to all stakeholders accompanied by adequate up-front investments to ensure use of M&S early in and throughout the life-cycle; this would minimize the total cost of ownership, shorten the acquisition cycle time, and improve support for warfighters and decision-makers; and provide incentives for active partnering between acquisition programs and between government and industry. (MORS, 2000) The working group made the following recommendations to DOD regarding policy initiatives: . establish policy and guidance to address M&S use in formal . . . . . acqulslbon c ,eclslons; . direct requirements developers and service acquisition executives to be held accountable to maximize SBA benefits and reduce life-cycle costs; define a policy for using emerging domestic and international commercial products and services in order to maximize SBA potential; and establish DOD policy and a common implementation for sharing M&S and data. (MORS, 2000) Regarding organization and focus, the working group recommended that DOD identify and empower an organization to enable dedicated and enduring pilot and flagship programs and to enable stewardship of SBA,
APPENDIX B 173 and to focus by priority what needs to be done. Finally, regarding technologies, the working group found that modeling methodologies have the most serious shortfalls and therefore require a high priority for funding. The working group made these recommendations: the Defense Modeling and Simulation Office identify model representations as a high priority in the next version of the DOD M&S master plan, the community working on computer-generated forces reprioritize and put effort into new simulation techniques; and that DOD work to resolve level-of-abstraction difficulties and consider the links between computer-aided design, computer- aided manufacturing, and operational effectiveness. (MORS, 2000) Although the MORS report was published in 2000, the final workshop was conducted at approximately the same time as the completion of the Joint SEA Task Force report (SABTF, 1998~. Many of the required changes and recommendations identified by the MORS acquisition working group are consistent with those of the Joint SBA Task Force. There is no evidence yet of substantive, corporate-level DOD action based on these recommendations.
