Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 147
B
Summary of 10 Acquisition-Related Studies
on Modeling and Simulation
During the 1 990s, U.S. government agencies and industrial
organizations sponsored a large number of studies in the general area of
modeling and simulation (M&S) as it related to the manufacture and
acquisition of military weapons and other equipment. These studies
addressed topics ranging from narrowly focused design and manufacturing
methods to more broad-based strategies for simulation-based acquisition
(SBA). The Committee on Modeling and Simulation Enhancements for
21St Century Manufacturing and Acquisition selected 10 studies of
acquisition-related M&S for in-depth review and discussion. These 10
studies were all formally commissioned and published in 1994 or later.
This appendix presents a summary of the objectives and major conclusions
and recommendations of each.
NAVAL RESEARCH
ADVISORY COMMITTEE REPORT
In 1994, the Naval Research Advisory Committee (NRAC)
performed a study on future uses of M&S for the U.S. Navy (NRAC,
1994~. The study, sponsored by the Assistant Secretary of the Navy for
Research, Development and Acquisition, had as its general objective the
assessment of the importance of high-fidelity models and advanced
147
OCR for page 148
148
MODELING AND SIMULA TION IN MANUFACTURING
distributed simulation (ADS) technologies to enhance the Nepal lenient of
the Navy (DON) test and evaluation and acquisition programs.
The five specific goals ofthe study were these: (1) review current
utilization of M&S and ADS technologies in the DON; (2) evaluate the
strengths and weaknesses of M&S and ADS technologies from the DON
perspective; (3) recommend specific research areas related to M&S and
ADS technologies that would warrant DON investment; (4) identify key
areas that would benefit from an investment in M&S and ADS
technologies; (5) and identify candidate demonstration projects to evaluate
the utility of M&S and ADS technologies.
The study panel was chaired by Dr. Delores Etter, then of the
University of Colorado, and included members from both industry and
academia. During a three-month period, the panel held a series of meetings
and gathered information from industry presentations as well as briefings
held at U.S. Navy, U.S. Army, and U.S. Air Force facilities. The panel's
report focused on two emerging simulation technologies: ADS and
simulation-based design/manufacturing. The panel coined the term
"distributed simulation-based acquisition" (DSBA) to describe the
capability represented by the merger of tools from these two areas.
The study panel envisioned the following conceptualization of
DSBA:
.
A single database to perform simulations to verify product
performance, develop design parameters, and address
manufacturing concerns
Linked simulation tools . . . in all phases of acquisition
Inclusion of the operational community . . . early and continually
during the acquisition process
The DV phase and the EMD phase . . . collapsed into a single
phase
An integrated modeling and simulation culture and its attendant
tool set provide the technical means to pursue concurrent
engineering
Multi-disciplinary teams will concurrently operate on identical
or linked databases performing the following functions:
operational concepts; threat definition; requirements tradeoffs,
systems definition, training and logistics, and production process
development; production; testing; and P3I and upgrades (NRAC,
1994, pp. 45-50)
The panel made the following recommendations regarding policy:
· Executive agent leadership of the Navy be vested in a position
that spans all warfare areas
OCR for page 149
APPENDIX B
technology:
The principal focus of the DoN modeling and simulation policy
should be to formulate a distributed simulation based acquisition
program
A technology base investment strategy is required to leverage
new developments in those fields through cooperative programs
with ARPA, DMSO, Joint Programs, industry, and academia
(NRAC, 1994, pp. 59-60)
The panel made the following recommendations regarding
· Exploit industry developments in simulation based
design/manufacturing
develop connectivity-ready models, databases, and architectures
for Naval unique advanced distributed simulation problems
· develop new technology for model reality checking, evaluation
and comparison (NRAC, 1994, p. 61)
Finally, the panel recommended that the DON evolve DSBA
technology through pilot programs, with several candidates named in
aircraft, ships, mine countermeasures, sea-based theater ballistic missile
defense, and ship self-defense.
149
Although no evidence indicates that the DON implemented any of the
specific recommendations made by the NRAC panel, the committee
believes that the work of this panel had an impact on later reports. The
NRAC panel's conceptualization of DSBA contains most of the technical
elements found in later DOD versions of SBA. In addition, the DOD
acquisition process approved in 2000 provides for flexibility in collapsing
phases of the acquisition process along the lines envisioned by the panel.
NAVALAIR SYSTEMS COMMAND STUDY
In 1995, the Naval Air Systems Command (NAVAIR) undertook a
study focused on collaborative virtual prototyping (CVP) for the common
support aircraft (CSA) initiative (NAVA1R, 1995~. The study
acknowledged the NRAC (1994) report and was performed in close
coordination with the study sponsored by the North American Technology
and Industrial Base Organization (NATIBO) discussed below.
The NAVAIR study had two specific objectives: to assess the ability
and readiness of the aircraft and electronics industrial base to use CVP
technologies in the acquisition of CSA, and to identify program
management and acquisition actions required in order for government and
OCR for page 150
150
MODELING AND SIMULATIONIN MANUFACTURING
industry to realize the potential productivity gains and cost savings offered
by CVP technology. Members of the NAVAIR study team together with
members of the NATIBO study team visited 57 organizations between
March and September 1995. The NAVAIR study focused on technology
assessment, business process reengineering, and demonstrated benefits.
The following conclusions and recommendations were made
regarding the application of Cvp tec.hn~loov
__.,,,_._z,, .
There exists a wealth of commercially available products and
services to support immediate implementation of a CVP
environment for the development of new products
The DoD, DoC, NSF, and DoE are developing an infrastructure
and a host of collaboration tools
There are aircraft specific applications and technologies being
matured by the JAST program. These efforts should be
leveraged for the development of the common support aircraft
Standards are the key element to all distributive enterprise
activities
The majority of existing models and simulations needed to
perform warfare analysis have not been developed to operate in a
distributive computing environment
· Producibility is a life-cycle cost driver. There are numerous
advanced manufacturing programs within the DoC, DoE, and
NASA. The DoN should leverage these programs to provide the
processing models needed (NAVAIR, 1995, p. 5-1)
The following conclusions and recommendations were made
regarding business processes:
· The commercial sector is rapidly developing tools for distributed
computing and virtual prototyping
Incorporation of the customer as a member of the IPPD team
significantly reduces the development time since non-value-
added activities can be minimized.
New information and distributed computing technologies have
spawned the formation of many small innovative companies.
ARPA Electronic Commerce Resource Centers are educating
small to medium size firms in the use of electronic commerce.
NAVAIR should leverage these programs by working through
primes to assist in modernizing critical suppliers.
NAVAIR should investigate the benefits of using commercial
business practices in revolutionizing the acquisition process.
(NAVAIR, 1995, p. 5-2)
i]
The following conclusions and recommendations specific to the CSA
nitiative were made:
OCR for page 151
APPENDIX B
The Naval Aviation Team should develop a strategy and plan for
adopting CVP technologies (SBD) and associated business
practices.
The CSA initiative should leverage the newly established
NAVAIR M&S executive committee to survey existing models
and simulations that will be applicable to the CSA initiative.
The models and simulations should be categorized according to
their functional discipline.
Developments within the S&T community should be focused to
achieve an affordable CSA.
CVP technology should be used to facilitate the partnership
between government and industry.
[D]ARPA and ONR technology efforts should orient their
testing/demonstrations to support the CSA initiative.
.
151
S&T investments should be made in process technologies that
reduce cost.
S&T should invest in CSA unique engineering and warfare
analysis tools.
· CVP technology should be used to facilitate the partnership
between government and industry.
The CSA IPT [integrated product team] should rapidly adopt and
transition the successful technology and business practices from
JAST [joint advanced strike technology program]. (NAVAIR,
1995, pp. 5-3 and 5-4)
Although focused specifically on the CSA initiative, the NAVAIR
(1995) study highlighted issues related to business process reengineering
and to partnerships and sharing between government and industry. The
themes of partnership and sharing, particularly as they pertain to industry
involvement earlier in the acquisition process and to the question of
proprietary rights, are reflected in subsequent studies.
NORTH AMERICAN TECHNOLOGY AND
INDUSTRIAL BASE ORGANIZATION STUDY
The North American Technology and Industrial Base Organization
commissioned a CVP study (NATIBO, 1996) to assess the maturity, level
of use, utility, and viability of CVP technology and its application to the
industrial base, including both small and medium-sized companies. The
NATIBO report provides an overview and assessment of CVP technology,
a discussion of system development and acquisition processes using CVP,
a presentation of case studies demonstrating the uses of CVP, and a
OCR for page 152
152
MODELING AND SIMULA TION IN MANUFACTURING
discussion of required investments and expected payoffs. The report also
lists technical, financial, procedural, cultural, and policy facilitators and
barriers to the use of CVP technology.
The NATIBO (1996) study produced the following conclusions with
respect to the technical, business, and political environment associated
with CVP at that time:
· Industly recognizes opportunities offered CVP
· CVP technologies exist and are advancing
. No true CVP environment currently exists
· No metrics are in place for measuring CVP benefits
· Proprietary data rights and protection of competitive advantage
are key industry concerns
· No government guidelines for CVP use have been set
Current government acquisition procedures do not promote CVP
CVP standards and better integration of tools are needed.
Financial investment is considerable for small companies
No central repository of CVP information currently exists
Model validation process takes too long (NATIBO, 1996, pp.
48-49)
Based on the conclusions listed above, the report outlined the
following 10 recommendations:
establish a central government office for CVP
sponsor integration and demonstration projects
implement policy to develop standardized metrics for evaluating
CVP payoffs in programs;
implement request-for-proposal (RFP) language and contracting
approaches that encourage CVP use;
reevaluate how developers deliver data to the government
coordinate CVP requirements with acquisition reform initiatives;
address data security and proprietary data concerns and
formalize policy regarding these issues;
target government investments on CVP integration technologies;
streamline the validation process for models; and
educate small businesses on less expensive options to acquiring
CVP technologies. (NATIBO, 1996, pp. 52-53)
Although the NATIBO (1996) study was also focused on CVP
technologies, it highlighted many more general SEA issues than the
NAVAIR (1995) study had specifically, issues related to industry
concerns and steps that the government could take to address these
concerns. Issues of proprietary data rights, required investments, and the
need for metrics to help support a business case for implementation were
OCR for page 153
APPENDIX B
153
prominent. This study was also the first to recommend a central
government office at the level of the Office of the Secretary of Defense to
coordinate policy and to act as a source of information.
AMERICAN DEFENSE PREPAREDNESS
ASSOCIATION STUDY
In 1996, the Undersea Warfare Division of the American Defense
Preparedness Association (ADPA) performed a study on the application of
M&S to the acquisition of major weapon systems (ADPA, 1996~. This
study was sponsored by the U.S. Navy Acquisition Reform Executive and
included industry, government, and university participants.
The context of the study was the development of a hypothetical Total
Ship Integrated Combat System for an Advanced Surface Combatant that
might be authorized post-2005, with a focus on the undersea warfare
components of such a system. The study assessed the potential of
achieving a 50 percent reduction in cycle time from the definition of
military needs to the achievement of initial operating capability, and it
assessed the potential for making similar reductions in life-cycle cost,
considering both the technical and business processes. The study objective
was to detennine the merits and benefits of an SBA approach for major
weapon systems, addressing the technical merit of proceeding with SBA
methods, the business integrity issues associated with such an approach,
and the changes necessary in the contacting and procurement system
processes necessary to facilitate such an approach.
The ADPA (1996) study reached the following conclusions:
.
Modeling and simulation tools, as well as new processes such as
integrated product and process development . . . are already
being applied in a range of ongoing acquisition programs.
· The challenge for acquisition reform is to provide the catalyst
that will expand this growing successful application of M&S
tools beyond vertical applications within programs so that cost
savings benefits can be realized by sharing data, tools, and
techniques between different acquisition programs.
The appropriate vehicle for meeting this challenge is simulation-
based acquisition
The new SBA culture . . . is predicated on mutual trust between
government and industry.
Program managers already have considerable flexibility.
.
OCR for page 154
154
MODELING AND SIMULA TION IN MANUFACTURING
· Decision makers using the SBA process will have the ability to
make smarter, faster, more informed decisions which will save
time and dollars throughout the life cycle. (ADPA, 1996, pp. 5-3
- 5-7)
Based on the above conclusions, the ADPA study team formulated
the following recommendations:
.
.
The government should firmly establish SBA as the preferred
manner of conducting IPPD-style acquisition and should
establish incentives for both government program managers and
industry to ensure full and enthusiastic participation;
Carefully designed pilot programs, structured as engineering
experiments with objectives and metrics, can demonstrate the
utility of SBA to the acquisition community and stakeholders
and thereby catalyze the cultural change that is required; to do
so:
Pilot programs should be augmented with necessary additional
Finds and should be focused not just on M&S tools, but on the
entire SBA process;
Metrics should address the building of a program-to-program
infrastructure that builds on the ongoing DOD investments in
M&S; and
The government should provide open access to government
infonnation and standard models to the appropriate industry
participants in He pilot programs;
· The government should re-direct DOD investment in M&S to
support and encourage development of an SBA-specific
infrastructure, the high level architecture (HLA) and other
infrastructure components being developed by DOD are
necessary but insufficient for the realization of SBA; in addition,
the following should be pursued:
.
Development of a comprehensive system data schema that
provides for common representation and data interchange
mechanisms between government and industry standard
databases; and
Development of key tools that would build on the HLA and act
as a jump-start to the pilot efforts; examples include a database
repository tool and common data library and M&S analysis
tools. (ADPA, 1996, p. 5-14)
No evidence indicates that the U.S. Navy Acquisition Reform
Executive took specific actions in response to the recommendations of the
ADPA (1996) study. However, some ofthe concepts originated in the
study (for example, SBA as a combination of technical, process, and
cultural elements; and the need for an SBA infrastructure to benefit
OCR for page 155
APPENDIX B
155
multiple programs) can be found in subsequent industry and government-
sponsored studies.
DIRECTOR FOR TEST SYSTEMS ENGTNEERING
AND EVALUATION STUDY
The Director for Test, Systems Engineering and Evaluation
(DTSE&E) in the Office ofthe Secretary of Defense (OSD) commissioned
a one-year study, also completed in 1996, to assess the effectiveness of the
use of M&S in the weapons system acquisition and support processes
(DTSE&E, 19961. The study team was asked to investigate metrics being
used to evaluate M&S effectiveness; specific tools being used by
government and industry to facilitate the design, development, test,
manufacture, and support of weapon systems in an IPPD environment; the
benefits associated with using M&S in the acquisition environment; and
the technical challenges that could preclude the seamless use of M&S in
the acquisition process. The DTSE&E study team reviewed previous
studies and visited individuals from OSD, government program offices and
research and development (R&D) and test and evaluation (T&E) centers,
and several industry organizations.
The DTSE&E study team produced the following findings:
· There is a recognized need for technology to be used by the
acquisition community as it reengineers itself into a team-based
approach;
M&S tools and processes are being used eff~cient]y and
effectively in each of the services, but not in an integrated
manner across programs or functions within the acquisition
process;
The words are in place in DOD acquisition documents to support
implementation of SBA, though there are some growing pains
associated with implementation. (DTSE&E, ] 996, report pages
not numbered)
.
Based on data gathering, the DTSE&E study team formulated the
following five recommendations:
.
Institutionalize the use of M&S and ensure that the community
is knowledgeable about the tools available. The services and
OSD must provide more responsive guidance relative to the
advent of better and more useful simulation tools. Dialogue is
needed within the services and between the services and OSD to
effect policy on standardization. Program managers must
OCR for page 156
156
MODELING AND SIMULA TION IN MANUFACTURING
.
overcome the management and cultural challenges that present
barriers to the effective use of available technology.
Provide focused information on the availability and capabilities,
including success stories, of M&S to weapon system acquisition
managers.
To meet the challenge of institutionalizing the use of available
technology, the services must be committed to providing funds
for M&S at the inception of the program. The OSD and services
should commit science & technology dollars to upgrade
capabilities and facilities that could serve many weapon system
acquisitions. Program managers should be encouraged to use
these facilities and capabilities instead of contracting to have
their own system-specific facilities and tools built.
· Develop an information source such as an Internet web page that
would list capabilities in design, the tools available, the
programs that have used them, and individuals that can be
contacted for further information. The same capabilities could be
listed for testing. The web page could be used to identify
innovative approaches in manufacturing and note those using
virtual manufacturing environments.
Opportunities to cooperate with industry, such as the DARPA
simulation-based design program, should be encouraged and
continued. There appears to be great potential in partnerships
such as the National Automotive Center, where both the
government arid industry benefit from investigating new
technology. Incentives to pursue business relationships such as
these should be developed to use developing technology more
efficiently. (DTSE&E, 1996, report pages not numbered)
In addition to providing examples of cost savings and cost avoidance
that resulted from the use of M&S in acquisition, the DTSE&E (1996)
study reinforced some of the conclusions and recommendations of prior
studies. It identified cultural and managerial issues, as distinct from
technical challenges, as perhaps the more difficult obstacles to overcome in
the institutionalization of M&S and the use of SBA in the DOD.
1997 NATIONAL RESEARCH COUNCIL STUDY
The Naval Studies Board of the National Research Council (NRC)
performed a study for the U.S. Navy and Marine Corps that resulted in the
1997 publication of a multivolume report. One of the volumes of this
report was focused on M&S (NRC, 1 997b). The M&S panel of the NRC
study was composed of members from academia, government-sponsored
OCR for page 157
APPENDIX B
157
centers, and the defense industry. The original goal of the study was to
review the overall architecture of M&S within DOD (including the DON,
the Joint Chiefs of Staff, and OSD), the ability of models to represent real-
world situations, and the merits of models as tools for making technical
and force composition decisions. After a preliminary review of existing
documents, the M&S panel focused its work more narrowly on the
following objectives: clarifying why senior levels of the DON should be
concerned about the substantive content and comprehensibility of M&S;
assessing what the DON and DOD might need to do in order to benefit
fully from the opportunities presented by M&S technology, clarifying what
M&S can and cannot be expected to accomplish in aiding decisions on
technical, force-composition, and operational planning issues; and
establishing priorities for M&S-related research.
The following conclusions and recommendations regarding M&S in
general were made in Volume 1 (NRC, 1997a):
· M&S demands the attention and support of top DON command
and management levels because it affects every aspect of
military force design, equipment, and operation. A new
corporate management approach is required if naval forces are to
capitalize fully on the benefits that M&S can offer. This
approach is needed to ensure compatibility, consistency, and
seamless interfaces between the U.S. Navy, U.S. Marine Corps
and joint service approaches to using M&S; coordinate inputs to
the joint services M&S projects (e.g., the Joint Simulation
System (JSIMS) and the Joint Warfare System (JWARS)~; and
ensure that existing simulations are upgraded or replaced. (p. 18)
The conceptual foundation of M&S must be brought up to date
and include knowledge of how modern warfare is and could be
fought. Currently, there is a dearth of theoretical understanding
and knowledge of modern post-Cold War types of warfare based
on collected and analyzed data. There is also a dearth of model
validation that compares the results of models describing
warfare with the outcomes of actual conflicts or even of field
exercises. Finally, there are no credible methods for model
validation. (p. 94)
An ongoing M&S research effort is needed that is focused on
military science and technique; that includes simulation science
and technology applicable to military systems and operation; and
that includes the construction and maintenance of databases
covering worldwide military forces and environments, organized
by warfare area. (p. 95)
.
.
In the M&S volume of the NRC report (NRC, 1 997b), the M&S
panel made additional recommendations:
OCR for page 164
164
MODELING AND SIMULA TION IN MANUFACTURING
.
· Only rudimentary applications of M&S [were found] in the
exploration of new warfighting and operational concepts;
· Characteristics and attitudes essential to analysis for innovation
are not particularly welcome in much of the analysis and
simulation community; and
· Infonned, involved customers are needed, particularly in the
joint community; insight, not advocacy, is needed.
· Changes are needed in organizations and processes to make
better use of the potential of existing simulation capabilities:
· Some organizations [were found] that reflected the attitude and
orientation needed, but they were focused on service capabilities
ends operating environments, not on joint operations; and
· Although there are organizations and centers that have potential
for becoming joint centers of excellence for exploring innovative
concepts arid capabilities, well-supported charters and
expectations are needed.
· Basic shortfalls include:
· Key elements of Joint Vision 20]0 are not addressed, for
example, information warfare, situational awareness, dispersion
of forces, maneuvers over strategic distances, dismounted
combatants with unprecedented potential, urban operations, and
new command and control arrangements;
· Lack of clear plans for evolution of models arid systems of
models to address these needs; and
· M&S and federations of simulations must accept new concepts
and doctrines far more readily; for the exploration of new
concepts, flexibility becomes a higher-valued M&S feature than
fidelity.
The customer needs to be the driving force, because:
· The current approach is to give resources to developers to build
better tools;
· General-pulpose models can rarely bear the weight of important
decisions or deal with the unfamiliar; and
· M&S customers need to have control of significant resources
and exercise more direct responsibility for details of defining
and overseeing the product they need. (DSB, 1999, pp. 8-10)
On the basis of the conclusions listed above, the DSB task force made
three recommendations dealing with process improvement and one
recommendation dealing with mode] improvement. The recommendations
regarding process improvement were as follows:
Recommendations for the Joint Chiefs of Staff (JCS) and the Joint
Requirements Oversight Council (JROC):
· The Chairman, Joint Chiefs of Staff (CJCS) should identify
several critical enablers and operational concepts within Joint
Vision 20] 0 to be used as the focus for simulation;
OCR for page 165
APPENDIX B
.
.
The JROC should issue a requirement for joint simulation
environments specifically focused on examining innovative
concepts and systems and should drive significant M&S
resources to this need;
The Joint Chiefs of Staff should be a lead, demanding, involved
customer for products; and
· The CJCS should task the joint and service schools to develop
courses for military and senior civilians on how to be effective
customers for M&S services.
Recommendations for joint focus, i.e. the Joint Chiefs of Staff and
the Under Secretary of Defense for Acquisitions and Technology:
To focus on analytical and simulation support for joint
innovative concepts and systems, the JROC needs to provide
consistent support for centers of excellence dedicated to this
purpose
· Such centers would be part of, or at least directly connected to, a
CINC with responsibilities for the joint world similar to the
responsibilities the Army's TRADOC has within the Army;
There should be a heavy emphasis on . . . experimentation to
discover what works and what doesn't before heavily investing;
and
.
A small group, specifically charged to think out-of-the box, is
needed to explore key facets of JV2010.
Recommendations for organizing joint operational architectures:
.
.
The JROC should continue to support and leverage the Joint
Theater Air and Missile Defense Organization experiment to
provide a coherent joint operational concept and architecture for
air and missile defense; and
The lessons learned should be applied to other areas needing
joint operational and technical architectures. (DSB, 1999, pp.
35-37)
The following recommendation was made by the DSB task force
regarding mode] improvement:
.
165
The customer community should take a far more active role in
defining simulation needs and setting priorities. For example:
The Joint Chiefs of Staff and the services should require that all
efforts examining new concepts also identify supporting M&S
. . .
priorities;
The Joint Chiefs of Staff and the Director, Defense Research and
Engineering (DDR&E) should increase demands that
OCR for page 166
166
MODELING AND SIMULA TION IN MANUFACTURING
simulations address the human element and Joint enablers that
drive operational outcomes.
· The JCS, DDR&E and PA&E should continue support for
JSIMS and JWARS, but require specific plans to more fully
incorporate drivers of effectiveness. (DSB, 1999, p. 38)
Since the publication ofthe DSB (1999) report, additional emphasis
has been placed by the defense simulation community on experimentation
and the representation of human behavior. However, there is no evidence
that any progress has been made toward implementing the process and
model improvements recommended by the task force.
1999 NATIONAL RESEARCH COUNCIL STUDY
In 1998, the National Aeronautics and Space Administration (NASA)
asked the NRC to undertake a study on advanced engineering
environments (AEEs). The study committee, composed of members from
industry and academia, was given six tasks, with the objective of
developing steps that NASA could take in the short term to enhance the
development of AEE technologies. The six tasks were as follows:
.
.
Develop an understanding of NASA's long-term vision of AEE,
capabilities, and tools associated with the current state of the art
and short-term advances in engineering environments;
· Conduct an independent assessment of requirements for,
alternative approaches to, and applications of AEEs to aerospace
engineering, considering both short- and long-term objectives;
· At a high level, explore the potential payoffs of AEEs on a
national scale, emphasizing the relationships between aerospace
engineering and other elements of the national engineering scene
and identifying the necessary conditions for achieving these
payoffs;
· Evaluate how AEE technologies relate to the development of
relevant technical standards and engineering economic
assessments;
· Identify cultural and technical barriers to collaboration among
the government, the aerospace industry, academia, and others for
transferring AEE tools and methods from the development stage
to public practice; opportunities that may be created by AEEs;
and needs for education and training; and
Recommend an approach for NASA to enable a state-of-the-art
engineering environment capability that is compatible with other
government, industry, and university programs and that
OCR for page 167
APPENDIX B
167
contributes to the overall effort to engender a broadly applicable,
technology-based, engineering framework. (NRC, 1999a, p. 41)
The committee collected information on AEEs from government,
industry, and academic organizations that were involved as developers,
providers, or users. The first phase of the study produced a report in 1999
(NRC, 1 999a). Based on the information collected, the committee on AEEs
defined the following vision: "AEEs should create an environment that
allows organizations to introduce innovation and manage complexity with
unprecedented effectiveness In terms of time, cost, and labor throughout
the life cycle of products and missions" (NRC, 1 999a, p. 2~.
The NRC committee produced 6 findings and 13 recommendations in
4 major categories: (1) a historic opportunity; (2) requirements and
benefits, (3) barriers; and (4) organizational roles. The committee found
that a historic opportunity exists to develop AEE technologies and systems
that could revolutionize engineering processes, but this opportunity is too
big for any single organization to realize. The committee made these
recommendations:
· a government-industry-academic partnership should be formed.
This partnership should foster the development of AEE
technology and systems and
· NASA should draft a plan for creating a broad government-
industry-academic partnership. In addition, . . . NASA should
charter a joint industry-academic-government advisory panel
that focuses on interactions between NASA and external
organizations. (NRC, 1999a, p. 34)
Regarding requirements and benefits, the NRC committee's findings
were as follows:
Current AEE R&D is too diffuse and should be focused on:
· enabling complex new systems, products, and missions
· greatly reducing product development cycle time and costs
AEE developers should devise an implementation process that
lowers technical, cultural, and educational barriers and that
applies AEEs broadly across government, industry, and
academia;
the top-level goals that NASA has established for the intelligent
synthesis environment functional initiative address important
AEE requirements. However, given the resources that NASA
plans to allocate, the objectives are overly ambitious; and
NASA should establish an AEE "center of gravity" that is
empowered to select the high-priority analyses and processes
that will be developed, integrated, and deployed as a mission
OCR for page 168
168
MODELING AND SIMULA TION IN MANUFACTURING
design system. To ensure success, the location, leadership, and
staff should be carefully selected to reflect the differing needs,
capabilities, and perspectives of NASA's operational and
research centers. (NRC, 1999a, p. 35)
With respect to barriers, the NRC committee found that efforts by
industry and government to develop and deploy AEEs faced significant
barriers in these areas: integration oftools, systems, and data; information
management; cultural, management, and economic issues; and education
and training. Regarding barriers to the integration of tools, systems, and
data, the committee stated:
.
.
A practical approach must be developed for improving the
interoperability of new product arid process models, tools, and
systems and for linking them with legacy tools, systems, and
data. Sponsors of AEE research and development should
consider integration of AEE product and process models; and
Government agencies and other organizations with a large stake
in the successful development of AEEs should interact more
effectively with standards groups to facilitate the development of
interoperable product and process models, tools, systems, and
data, as well as open system architectures. (NRC, 1999a, p. 36)
Regarding barriers to information management, the NRC committee
found a lack of commonality in product and process descriptions within
and among user organizations and between users and suppliers, and that
the need for customization greatly reduces the cost-effectiveness of new
tools. The committee therefore recommended that corporate and
government leaders develop robust and flexible AEE tools for creating,
managing, and assessing computer-generated data; for presenting relevant
data to operators in a clear and efficient manner; for maintaining
configuration management records; and for storing appropriate data on a
long-term basis.
Regarding cultural, management, and economic barriers, the NRC
committee found that, historically, not enough attention has been paid to
the organizational, cultural, psychological, and social aspects of the user
environment associated with AEE technologies. It recommended that
AEEs be integrated into the senior management culture of organizations
investing in AEE technologies and systems; that each organization
designate a champion with responsibility, authority, resources, and support
from a team of senior managers, technical experts, and other critical
stakeholders. Similar subordinate teams should be assembled in major
organizational elements or facilities involved.
Regarding barriers to education and training, the NRC committee
found that government agencies have frequently used contract provisions
OCR for page 169
APPENDIX B
169
to influence contractor business practices and, occasionally, engineering
practices. Therefore the committee made the following recommendations:
.
.
.
Government agencies involved in the acquisition of complex
engineering systems should provide incentives for contractors to
implement appropriate AEE technologies and systems and to
document lessons learned. These incentives should target both
technical and nontechnical . . . aspects of AEE development and
implementation;
NASA should define an agency-wide plan for the development
and implementation of comprehensive, improved engineering
processes, practices, and technologies; NASA-wide teams
directing the Intelligent Synthesis Environment functional
initiative should be consolidated and strengthened;
An advisory panel with representatives from industry,
universities, the National Science Foundation, NASA centers,
and other government agencies and laboratories should be
convened by NASA . . . This panel should define incentives for
accelerating incorporation of AEE technologies into the
engineering curriculum, define the basic elements of a suitable
AEE experience for students, and specify resource needs. (NRC,
1999a, pp. 37-38)
Finally, regarding organizational roles, the NRC committee made the
following recommendations:
· AEEs should use commercially available tools as much as
possible. In general, the development of application-specif~c
tools should be left to industry. If commercial tools are
inadequate, government agencies should create incentives for
commercial vendors to develop improved, broadly applicable
tools.
· To maximize the effectiveness of [its own AEE R&D and that
undertaken by other organizations], NASA must improve its
understanding of the capabilities and requirements of external
organizations. (NRC, 1999a)
It is too early to assess the degree to which the recommendations of
the NRC (1 999a) report have been implemented by NASA. However, it is
important to note that the NASA-sponsored ISE initiative, which had
objectives similar to those of DOD's SEA initiative, ceased to exist as a
separate NASA program in early 2001.
OCR for page 170
170
MODELING AND SIMULA TION IN MANUFACTURING
MILITARY OPERATIONS RESEARCH
SOCIETY REPORT
In 1997 and 1998, the Military Operations Research Society (MORS)
conducted two workshops, known as "SIMTECH 2007," and produced a
report on the results in April 2000 (MORS, 2000~. The main goal of the
workshops was to promote more effective dialogue between the M&S
technology community and M&S users, such as analysts, acquirers,
educators, and trainers. Four subordinate workshop objectives were: (1) to
review and assess the findings and recommendations from SIMTECH 97, a
set of workshops held about a decade earlier; (2) to identify and prioritize
military M&S user needs; (3) to assess the probable evolution of M&S
technology over the next decade; and (4) to identify opportunities for
addressing user needs. Within the workshops, there were several working
groups, including one on acquisition, the results of which are described
below.
The MORS acquisition working group was asked to characterize the
acquisition process of the year 2007; to identify major changes that must
occur in order to bring about this acquisition process; to identify shortfalls
in M&S; and to prepare actionable recommendations to address the
shortfalls in investments and incentives, and to address policy and
organization. The working group adopted an acquisition vision and goals
statement similar to the SBA vision and goals statement promulgated in
1997 by the Acquisition Functional Area Council of the DOD Executive
Council for Modeling and Simulation (EXCIMS) and used as a starting
point by the SBATF Joint SEA Task Force. The working group
characterized the desired end-state as including the following:
increased contractor total system responsibility with more
efficient government insight to allow trusted partnerships
between government and industry;
a highly integrated electronic work environment across all life-
cycle functions,
reduced life-cycle costs and development time consistent with
commercial practices; and
· DOD commitment to making the most informed acquisition
selection and decisions based on life-cycle cost, authoritative
data and model sources, collaborative M&S use, and proper JPT
use. (MORS, 2000, report has no page numbers)
The MORS acquisition working group identified major cultural,
management, policy, and technology and environment changes that were
OCR for page 171
APPENDIX B
needed in order to achieve the envisioned end-state. According to the
working group, the cultural changes needed were these:
.
_.~, A. _..~..=
A greater reliance on M&S,
· Horizontal integration and breakdown of stovepipes,
· Government-industry trusted partnerships;
.
Pervasive sharing of models and data;
U-front investment in modern processes;
Enabling of international involvement;
· The ability to conduct comprehensive life-cycle trades;
· Flexibility to accommodate a major force restructuring; and
Education for the vision. (MORS, 2000)
The management changes needed were as follows:
· An alignment of development time to be more consistent with
commercial life-cycle times and ready accommodation of
technology insertion and turnover;
· Business process reengineering of data production;
· Making functional IPTs a way of life, for example, leveraging
M&S across functions and domains and avoiding redundancy;
· Requiring authoritative sources for models and data, for
example, having a program manager provide sources for system
models and other stakeholders provide sources for environment;
and
· DOD commitment to a life-cycle-cost basis for acquisition
decisions. (MORS, 2000)
According to the MORS acquisition working group, the policy
changes needed were these:
Up-front investment as the norm to reduce life-cycle costs;
Making M&S strategy integral to the total acquisition plan;
Making M&S critical to formal acquisition decisions, including
policy guidance on what the Defense Acquisition Board can
expect and guidance to the program manager on what to provide;
Incentives for all stakeholders to participate; and
DOD policy and guidance on M&S use and sharing of M&S
technology between government and industry and across
programs. (MORS, 2000)
171
Finally, the technology and environment changes needed according to
the MORS acquisition working group were these:
OCR for page 172
172
MODELING AND SIMULATIONIN MANUFACTURING
· Creating a DOD-wide, knowledge-based infrastructure to enable
SBA, including program-specific functional integration,
appropriate use of commercial-off-the-shelf and government-off-
the-shelf products, interoperability and reuse standards;
· Creating capability to conduct trades across highly diverse
mission and functional areas,
Facilitating ease-of-use, otherwise known as cross-platform,
plug-and-play, or throw-away;
The development of validated data sources, models, and tools;
and
Investment in a comprehensive modeling capability. (MORS,
2000)
The MORS acquisition working group then made a number of
recommendations regarding investment and incentives, policy initiatives,
organization and focus, and technologies. The working group
recommended the following actions for DOD to take regarding investment
and incentives:
.
establish and support sufficient M&S infrastructure investments
in the program objective memorandum;
provide incentives to all stakeholders accompanied by adequate
up-front investments to ensure use of M&S early in and
throughout the life-cycle; this would minimize the total cost of
ownership, shorten the acquisition cycle time, and improve
support for warfighters and decision-makers; and
provide incentives for active partnering between acquisition
programs and between government and industry. (MORS, 2000)
The working group made the following recommendations to DOD
regarding policy initiatives:
. establish policy and guidance to address M&S use in formal
. . . . .
acqulslbon c ,eclslons;
.
direct requirements developers and service acquisition
executives to be held accountable to maximize SBA benefits and
reduce life-cycle costs;
define a policy for using emerging domestic and international
commercial products and services in order to maximize SBA
potential; and
establish DOD policy and a common implementation for sharing
M&S and data. (MORS, 2000)
Regarding organization and focus, the working group recommended
that DOD identify and empower an organization to enable dedicated and
enduring pilot and flagship programs and to enable stewardship of SBA,
OCR for page 173
APPENDIX B
173
and to focus by priority what needs to be done. Finally, regarding
technologies, the working group found that modeling methodologies have
the most serious shortfalls and therefore require a high priority for funding.
The working group made these recommendations:
the Defense Modeling and Simulation Office identify model
representations as a high priority in the next version of the DOD
M&S master plan,
the community working on computer-generated forces
reprioritize and put effort into new simulation techniques; and
that DOD work to resolve level-of-abstraction difficulties and
consider the links between computer-aided design, computer-
aided manufacturing, and operational effectiveness. (MORS,
2000)
Although the MORS report was published in 2000, the final
workshop was conducted at approximately the same time as the completion
of the Joint SEA Task Force report (SABTF, 1998~. Many of the required
changes and recommendations identified by the MORS acquisition
working group are consistent with those of the Joint SBA Task Force.
There is no evidence yet of substantive, corporate-level DOD action based
on these recommendations.
OCR for page 174
Representative terms from entire chapter:
following recommendations
