Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 29
2
Modeling and Simulation in Defense Acquisition
The use of modeling and simulation (M&S) in defense acquisition is
not new. Before digital computers and networking technology, analysts
mathematically modeled the effect that new or improved defense systems
would have on warfighting capability, engineers built physical models of
systems, and testers simulated combat in field tests and exercises. In the
1 960s, as computing capabilities increased, the task of modeling and
simulating both the design and performance of defense systems moved
increasingly toward digital representations and algorithms implemented in
computer software. As high-speed digital networking evolved during the
1 980s and 1 990s, the ability to share this digital information both within
and across organizations increased rapidly and created opportunities for
collaboration in the development of defense systems. In the 21 st century,
the long-term needs facing defense acquisition will require an expanding
array of M&S technologies to enable rapidly evolving, even revolutionary,
defense acquisition.
In order to assess the long-term needs for M&S in defense acquisition
and the current state of M&S use within DOD, the committee examined
DOD's future acquisition vision, current uses of M&S in developing
defense systems, existing initiatives for supporting M&S in acquisition,
and the results of 10 previous acquisition-related studies of M&S.
29
OCR for page 30
30
MODELING AND SIMULATIONIN MANUFACTURING
SIMULATION-BASED ACQUISITION
A variety of terms have characterized DOD use of M&S technologies
over the past two decades. Within DOD, advances in M&S have occurred
primarily in the defense simulation and the product modeling, design, and
manufacturing communities (NRAC, 1994~. Within the defense simulation
community, work on distributed, linked M&S has generally been termed
advanced distributed simulation (ADS). This work evolved from the early
simulated networking (SIMNET) effort sponsored by the Defense
Advanced Research Projects Agency (DARPA), the development of
distributed interactive simulation (DIS) standards, and DOD-sponsored
work on high level architecture (HLA). Within the product modeling,
design, and manufacturing community, M&S advances have been achieved
through commercial efforts, such as the Boeing 777 aircraft design, and
through defense efforts, such as the DARPA simulation-based design
(SBD) program (NRAC, 1994~.
Definition of S;muiation-Based Acquisition
In 1994, a Naval Research Advisory Committee (NRAC) study panel
characterized the convergence of activities between these two
communities the defense simulation and the product modeling, design,
and manufacturing communities- as distributed simulation-based
acquisition (DSBA) and suggested that this concept had the potential to
revolutionize the defense acquisition process (NRAC, 1994~. As the DSBA
concept evolved within DOD, particularly within the Office of the
Secretary of Defense (OSD) and the Acquisition Functional Area Council
ofthe DOD Executive Council on Modeling and Simulation (EXCIMS),
the name was shortened to simulation-based acquisition (SBA). In
December 1997, the Acquisition Functional Area Council defined the
following vision for SEA, which was re-adopted as a concise definition in
August 2000:
"An acquisition process in which DOD and industry are enabled
by robust, collaborative use of simulation technology that is
integrated across acquisition phases and programs." (SBATF,
1998; SBAISG, 2000)
It is important to note that the term SBA is used not only in reference
to current efforts to incorporate M&S into the defense acquisition process,
but, more importantly, it is used to designate the desired future DOD
OCR for page 31
SIMULA TION-BASED ACQUISITION
31
acquisition process. The specific goals of the future SBA concept are to
use M&S technologies to reduce time, resources, risk, and total ownership
costs of military systems in the acquisition process; to increase quality,
military worth, and supportability of military systems; and to enable
integrated product and process development during acquisition (SBATF,
1998; SBAISG, 2000~. Models of proposed system designs would be
constructed and tested in simulated environments, and these virtual
prototypes would then be used to refine system requirements and relate
trade-off and engineering decisions to the requirements. Computer-based
models could be maintained throughout the development, production, and
modification phases of the product life cycle (NRC, 1997~. A collaborative
acquisition process, making extensive use of advanced M&S technology,
exercised in a modified defense culture is the end state desired by DOD.
The SBA concept is broad, encompassing not only product
development and manufacturing, but also simulations to estimate system
performance and mission effectiveness, combat training, the underlying
technical information needed to train system operators, the product
modeling and manufacturing processes of the commercial enterprises that
support defense acquisition, simulations to support maintenance training,
technical information used by maintainers, logistics simulations to relate
support plans and resources to readiness, and simulations to address system
disposal issues.
If SBA is thought of as a category of M&S applications (i.e., ways of
using M&S), Figure 2-1 illustrates the relationship between SBA and other
M&S applications categories. The outermost oval represents all
applications of M&S. The two large interior ovals represent two
overlapping sub categories of M&S applications: those that support or
enhance military power and those that support or enhance commercial
success. A few arbitrarily chosen example applications within each
All::~hf&S Applications
Simulation-
Based Acquisition
Combo vehicle design
I:: :: - it-; ~ ~ -\ ~ We on session .
S~urgerysimulatlon
Science education
Figure 2-1 Relationship between SBA and other categories of M&S applications.
OCR for page 32
32
MODELING AND SIMULA TION IN MANUFACTURING
category of application are listed in each portion of the figure. SBA,
which, as previously noted, is quite broadly defined, is nevertheless a
subset of all military M&S applications. Furthermore, some of the M&S
applications within SBA are also valid in the commercial context, as the
figure suggests.
Each of the military services is developing the SBA concept to fit its
specific needs. The Department of the Navy is integrating SBA with other
M&S initiatives and standards activities intended to help acquisition
program managers (Hollenbach, 20003. The U.S. Anny's SBA vision is
called the Simulation and Modeling for Acquisition, Requirements, and
Training (SMART) initiative. Among other characteristics, SMART
emphasizes collaboration during the acquisition process between the
developers, evaluators, and users of combat systems (Purdy, 2001~.
In 1998, the Joint Simulation-Based Acquisition Task Force
(SBATF) was commissioned to develop a road map for DOD action on
SBA (SBATF, 1998~. The SBATF's objectives were to develop
representations of architectures needed to establish SBA environments,
identify technical challenges, identify primary ownership of each module
in the systems architecture, identify investments needed by government
and industry, list DOD actions needed to develop the SBA concept, and
identify industry actions needed to accelerate the SBA concept. The
SBATF made extensive recommendations in the areas of management,
architecture, policy and legislation, and education and training.
Use of Modeling and Simulation in Defense Acquisition
The establishment of DOD's Defense Modeling and Simulation
Office (DMSO) in 1991 was an important milestone in recognizing the
potential of M&S for defense applications. DMSO is responsible for the
development of the DOD M&S master plan; the development of the DOD
M&S investment plan; the establishment of the Modeling and Simulation
Information Analysis Center (MSIAC); and the development of
technology, standards, and tools for M&S. The M&S master plan,
combined with the M&S science and technology program, is the main
source of information for acquisition program managers. In addition, the
MSIAC provides tools and information that can help program managers
develop their simulation support plans.
The existence of DMSO was an important driver in the development
of DOD's HLA and synthetic environment data representation and
interchange specification (SEDRIS). HLA provides rules and run-time
infrastructure to allow M&S applications to be integrated to meet new or
changing requirements; it has already advanced the use of M&S in
analysis, acquisition, and training. SEDRIS has enabled the sharing of
OCR for page 33
SIMULATION-BASED ACQUISITION
33
environmental data across M&S applications. Both of these standards have
therefore increased reusability of the M&S applications developed by
DOD.
Examples of Modeling and Simulation in Defense
Acquisition
M&S is already in use in several defense programs as a means of
improving the design of new systems, integrating manufacturing modeling
with system simulation, and evaluating the combat effectiveness of new
systems.
U.S. Navy LPD-I 7 Program
The U.S. Navy used M&S in engineering the systems and subsystems
ofthe LPD-17 helicopter carrier.' A digital model ofthe LPD-17 was
developed using three-dimensional computer-aided design (CAD)
modeling and computer-aided manufacturing (CAM). By developing this
type of integrated product development environment combined with three-
dimensional visualization, the Navy leveraged previous efforts that had
demonstrated the value of three-dimensional visualization and modeling.
The Sec~wolf; Virginia Class Attack Submarine, and Aegis Class Destroyer
programs had shown that use of three-dimensional visualization and
modeling to solve engineering problems could reduce reengineering costs
once production had begun.
A comprehensive plan to use M&S throughout the system acquisition
process was developed for the LPD-17 program. This plan included
constructive simulation to support concept studies and beyond, virtual
simulation to support concept demonstration and beyond, as well as live
simulation to support demonstration and validation for the following
acquisition milestones: concept demonstration approval, development
approval, production approval, and major modification approval (see
Figure 2-23. Most of the M&S investment was focused on the three-
dimensional product model for the purpose of supporting engineering
design.
' Mike Wendel, Coleman Research Corporation. 2000. Presentation to this study committee.
OCR for page 34
34
MODELING AND SIMULATIONINMANUFACTURING
~as" ~ Phil - ~ P6.~ It ~ HI ~ IY
On ~ Cow ~ ~~ ~ £~ ~ ~ ~ ~ ~~ ~~~-~ ~
Lion ~ Ex - ~n ~ ,;~. ~~$~ ~ 34.t~ ~ 13~_ ~ a~`
0~. ~~ -
:=n - : (
: s'~s O
APE :h~'
C:
C~n R~.~ -
We - ~$ '
S~.~s
Lam - ~ ~ 6. P~r~ M - Is
DO ~ ' ~ ' -'~<'~~~
Ad:
·,~ Up, ~—
Page.'
Balm
~~ - S=~S : ....... .. .
p~,,~G~t
^~1: ~—
CAT ship
CHIP
%
T~.
T,r.a' - ,~ 6
Lit - Id. _
Id -
n~:s~
Figure 2-2 Use of M&S in the acquisition process for the LPD-17. Source: Mike
Wendel, Coleman Research Corporation. 2000. Presentation to the study
committee.
Joint Strike Fighter Program
Both the government program manager and the system contractor,
Lockheed Martin, have used M&S in the Joint Strike Fighter (JSF)
program. The U.S.Air Force developed a simulation-assessment-validation
environment (SAVE) that focused on the use of M&S within the context of
DOD's integrated product and process development framework.2
According to the Air Force, no tool set was available in 1995 for the
integration of manufacturing modeling with system simulation. The SAVE
initiative sought to bridge this gap and to use M&S in a comprehensive
program. The goad was to avoid dater reengineering costs caused by a less
robust process that matched design with requirements.
Lockheed Martin developed a version of simulation-based acquisition
for the JSF program.3 The company used the "V" mode] described by
Forsberg, Cotterman, and Mooz (2000) and B]anchard and Fab~ycky
(1998) (see Figure 2-3~. This is a new application of the classic systems
engineering mode] to describe the integration of M&S with the systems
. .
engineering process.
2 James Poindexter, Air Force Research Laboratory. 2000. Presentation to this study
committee.
Matt Landry, Lockheed Martin. 2000. Presentation to this study committee.
OCR for page 35
SIMULA TION-BASED ACQUISITION
re~n~ i: ~ ~v ~k~?ha~::~ it:: :~:
Aticea~ a.
.~
mar ~ :!
~ ~fehide~
. ~
~ ...,.
~ .,.~.,
We~ sorry
~n'~ ~
S - Ah - ~
,Adf 'ERIC
IF¢6gra
Wr~ff=
~5~ 6, ver'~m ~
Figure 2-3 Planned applications of M&S in the system design and development
phase for the Joint Strike Fighter. Source: Matt Landry, Lockheed Martin,
presentation to the study committee, 2000.
U.S. Navy SC-21 Program
35
The U.S. Navy's 21st Century Surface Combatant Land Attack
Vessel SC-214 program was a pilot project in which computer-generated
forces created the simulated battlespace in which the SC-2 1 design
concepts were evaluated (Ewen et al., 2000~. These forces included both
friend and foe entities. Such programs are important because SBA will
require computer-generated forces to analyze the effectiveness of combat
systems.
A limitation on this type of evaluation is the need to improve human
behavior models for M&S applications (NRC, 1 998b). As another step in
this field, the U.S. Air Force Research Laboratory has established the
combat automation requirement testbed (CART) program to develop
models of human performance and behavior. These models will be used to
represent the performance of human operators, such as aircrew members,
in M&S evaluations of weapons systems during the acquisition process.
4 This program has since been redesignated as DD 21 and is now incorporated into the DD(X)
program.
OCR for page 36
36
Future Programs
MODELING AND SIMULATIONIN MANUFACTURING
The Future Combat Systems (FCS) Program plans a comprehensive
approach to the use of M&S in analyzing requirements, supporting design,
and supporting test and evaluation (Purdy, 20011. This program, based on
the U.S. Army's SMART initiative, has a systems engineering approach
similar to that used by Lockheed Martin for the JSF (see Figure 2-4~. and
appears to be making very comprehensive use of M&S.
The "V" model diagram shown in Figure 2-4 attempts to capture the
flow of systems engineering activity as performed in the FCS program and
is a framework applicable to other programs. At the upper left,
overarching system-level design requirements are developed. Flowing
down the left side are increasingly fine details of subsystem design and
specification utilizing M&S to assess performance of the subsystems
relative to their interface specifications with other subsystems. The right
side of the diagram incorporates integration and production of the
components and subsystems, using M&S to test whether the larger system
requirements are being met.
The U.S. Navy's Collaborative Engineering Environment (Crisp,
2002) is being formulated to assist program managers by providing
collaborative tools and supporting data focused on the Pre-System
Acquisition Phase of the Acquisition Process. The other services would
~__e _ ~
B.. i. ~ ~ ~ ~ ~ ~ i. ~ ~ . ~ ~ . . ~ . ~ . ~ ~ ~ i. ~ ~
~ .c
A ~ ~. ~~ ~~ ~ -. . ~~ a. ~~ ~
s - .::::-., I': ~ ~~ :.:::.\
O - gn \~ ~~ -:~ : ~~ ~~ ~~ ~~ By:
Requirements '
Jam -I= ~ ~~ ~ ~ ~ ~ I- ~ ~~: ~~ ~~ ~ ~~ f
A- ~ ~ ~ I: ~~:~ ~~ -~ Ace. ~ ~ ~~ ~ ~~ ~ ~~ An.'
~ it,, ~ ~ ~ ~ ~ . ~
f-~ ~ ~~ ~^ ,,,,, ~ ~ ~~ . ., ~ . ~ ~~-~ ~ .~ ~~:~ ~!
,:::.: . ~.~ ~~ ~~ .. . it. ~~, it: .:.,
, .:_ ~ ~ ~~ ~~ ~~ ~ . ~ ~ ~~ .~ :: ~ :#
I: ~~ :~:: ~~ ~ .~ ~ i: ~~:~ i. :. ... ~ air. : ~~ I!
f :~ :~ ~~ -: ~ ~ ~~ : : ~ ~~ ~ I: ~!
Item-L.~I \
Meson Hi, i:
I~-~'rernents \
\
\
.411 Oes3gn FIeq`3i~rnents ~mple¢e
~ ~~ ~:~:! Con Steno
''
.. ~~ ~-~.~ .1
~ I::,
' i/ trial
/ ~ portents
Figure 2-4 Future Combat System "V" Model. Source: Purdy (2001~.
OCR for page 37
SIMULATION-BASED ACQUISITION
37
benefit by examining this environment as an excellent means to implement
SBA concepts at the earliest point in the Acquisition Process."
Barriers to More Widespread Use
Although DOD is committed to maximizing the benefits of M&S
technologies, numerous barriers impede the more widespread use of M&S
in defense applications. These barriers include inadequate allocation of
resources to support SBA, lack of information for acquisition program
managers, obstacles to collaboration between government and industry, the
need to protect intellectual property rights, the lack of dissemination of
information on SBA to the wider M&S academic community, and the need
for standardized education for M&S professionals.
Inadequate Allocation of Resources
Inadequate managerial authority, as well as insufficient financial and
leadership resources, has been allocated to support the achievement of
DOD's stated vision for SBA. The development of HLA and SEDRIS are
examples of centralized investments that have led to cost savings for
individual programs. In contrast, investments in M&S technology,
standards, and tools made by individual programs are direct costs and
difficult to recoup. To stimulate activities that are of value to the larger
community, a variety of approaches should be considered (e.g., programs
for centralized investment; or policies to reward an individual acquisition
which makes an investment that benefits others).
Existing uses of M&S in defense acquisition indicate that efforts to
implement SBA have resulted in uneven applications of the approaches
and capabilities available to program managers. Investments in the
development of M&S technology and standards and investments in the
application of M&S to the defense acquisition process appear unbalanced.
Although DOD has the infrastructure, procedures, and plans to identify,
develop, and maintain needed M&S standards and technologies, the same
level of maturity has not been achieved in applying M&S capabilities to
the defense acquisition process. Reports from program office principals
and from the DOD acquisition and M&S communities indicate that the
important ongoing efforts to develop M&S technology and standards are
achieving success on a broad front, but that there is an important shortfall
in the application of M&S to the defense acquisition life cycle and to the
engineering of systems within the defense acquisition life cycle.
improvements are also needed in the culture associated with the use of
M&S in manufacturing and acquisition.
OCR for page 38
38
MODELING AND SIMULATIONIN MANUFACTURING
Lack of Information' for Program Managers
Currently, program managers lack comprehensive information on
how to integrate M&S with the systems engineering process and
information on available tools to support and realize this integration.
Although the DOD Acquisition Deskbook5 contains specific information
useful to understanding the policy and procedures that support SEA and
the use of M&S in the acquisition process, there is no reference to best
practices as carried out by DOD program offices in applying M&S to the
acquisition process. The Acquisition Deskbook includes a process
discussion that explains the role of M&S in system acquisition and
explains the integration of M&S into the system acquisition life cycle
through descriptions of several acquisition phases in DOD-wide practice.
Although this is an excellent beginning, more is needed on how M&S is
brought to bear on the systems engineering life-cycle process.
DMSO has developed detailed discussions of the role of M&S overall
and in each acquisition phase. This discussion must be augmented by a
discussion of the role of M&S in each activity of the systems engineering
process. Several useful cases exist that would illustrate how specific M&S
approaches and tools were applied to an activity and what benefits were
derived from doing so—for example, the JSF program's use ofthe U.S.
Air Force's Thunder M&S application in order to understand the value of
the JSF in a theater context. Thunder is an accepted campaign-level model
used throughout both the U.S. Air Force and other DOD units to study
force structure and system requirements with a view toward understanding
their contribution to operational outcome. Several campaign-level models
have wide acceptance in DOD and are applied to assess the value of
investment in systems by understanding the return on investment based
upon the effect on operational outcome. The combination of campaign-
level, mission-level, system-level, and subsystem-level models provides a
capability to support system engineering activities at the system,
subsystem, and component levels both in new system development life-
cycle phases and in system modification and system maintenance life-cycle
phases.
Needfor an Integrated Systems Engineering Process
Currently, there is no broadly accepted definition of an integrated
systems engineering process for the development of software-intensive
systems. To overcome this barrier, DOD has assigned the Software
Engineering Institute and the National Defense Industrial Association's
s Available at . Accessed June 2002.
OCR for page 39
SIMULA TION-BASED ACQUISITION
39
System Engineering Committee to define an overarching systems
engineering process for such systems. This effort pulls together industry
best practices, such as the Electronics Industry Association's 632 Standard,
Processes for Engineering a System (EIA, 1999), and the Institute of
Electrical and Electronics Engineers' (IEEE's) 1220 Standard, Application
and Management of the Systems Engineering Process (IEEE, 1998~. This
effort may serve as a model for the development of similar systems
engineering models for other DOD domains.
Similarly, M&S needs to be better integrated into the overall DOD
systems engineering and acquisition process. Methods and standards for
DOD M&S (such as HLA and SEDRIS, while beneficial within the M&S
discipline, need to be better related and integrated into the overall system
acquisition process. For example, it would be beneficial to have tools that
assist in ensuring that the software representations developed in accordance
with the HLA standard are consistent with the requirements documents for
the systems they represent, and that they accurately represent the
operational and systems architectures associated with these systems.
Although there have been attempts at codifying DOD mission
representations (such as the Functional Descriptions of the Mission Space
effort, formerly known as Conceptual Models of the Mission Space), these
efforts are not sufficient to ensure the degree of integration and consistency
that is needed.
Obstacles to Collaboration
Increased collaboration between government and industry has been
recommended by numerous studies as an essential element of the success
of SBA. Specific technical requirements recommended for achieving this
objective include distributed information repositories with search access,
collaboration mechanisms, and security and access control mechanisms for
shared data (Ho]]enbach, 20003; co]]aborative environments (SBATF,
1998; Coo]ahan et a]., 2000~; and data standards for integrated data
environments (Starr, 1998~. Nontechnica] changes are also needed,
including policy and law changes to support the delineation of
responsibilities and contractual sharing of data (Ho]]enbach, 2000) and
changes in the acquisition process to adapt it to the new methods of SBA.
Specific requirements include partnering of combat and weapons system
developers earlier in the concept exploration phase, examination of the
process by which the government trades simulations with industry during
the development process, and examination of the resources required for
this effort (Hollis and Patenaude, l 999~. Another nontechnical issue is that
security classification of the scenarios behind models and simulations limit
OCR for page 40
40
MODELING AND SIMULATIONIN ~NUFAC TURING
their use in academic environments and increase the cost of using them
elsewhere.
Intellectual Proper Rights
Protection of intellectual property rights is also a non technical issue
that presents a potential obstacle to use of M&S in acquisition.
Specifically, the question of how to protect the proprietary interests of
model builders within the SBA process must be addressed. Bidders in
highly competitive military procurement programs may be selected on the
basis of the results of simulations of proposed weapons systems. The
simulations used to provide information for decision making will most
likely be provided by the bidders themselves. Competing bidders, military
decision makers, and elected officials will all have an interest in examining
the simulations for accuracy, while the details of the proposed weapons
systems and the proprietary nature of the modeling methodologies will
need to be protected.
Information Dissemination
The progress of M&S use in defense applications may be hindered by
the insufficient and uneven dissemination of information on SBA to the
broader M&S community. This community is divided into subgroups with
diverse M&S interests, such as simulation, interoperability, industrial
engineering, physical sciences, and biological sciences. While SBA is well
represented at the meetings of some professional organizations, little
awareness of it exists at others. Because the SBA vision covers the
complete spectrum of phases in system acquisition, many ofthese
disciplines have important contributions to make. Although academic
researchers may be more interested in the narrower issues relating to their
own disciplines, it is important that the SBA vision, requirements, and
opportunities for involvement be more widely disseminated.
Education and Training
Information technology is facing exponential growth as a field, with a
corresponding increase in training materials, courses, and certificates in
hardware design, network administration, and programming. M&S has
developed into a field that interacts with a cross-section of science,
business, and engineering applications. M&S professionals, in addition to
requiring a basic information technology background, also require an
understanding of core concepts and skills specific to M&S. The future
OCR for page 41
SIMULA TION-BASED ACQUISITION
development of M&S as a discipline, profession, and industry is strongly
reliant on the growth of necessary M&S education and training.
Validation and Serif cation
41
If simulation-based acquisition is to be credible, it must employ data
and models and simulations that have undergone rigorous validation and
verification. Although numerous efforts have been made by DOD and
professional organizations (e.g., the Military Operations Research Society)
to formulate guidelines that address this issue (Pace and Glasow, 1999), an
efficient, rigorous process remains to be formulated and applied
systematically. This is particularly challenging for future systems-of-
systems in which new doctrine, concepts of operation, and training will
convolve as users experiment with and gain experience with the new
systems.
REVIEW OF ACQUISITION-RELATED STUDIES
ON MODELING AND SIMULATION
During the 1 990s, many studies were sponsored by U.S. government
agencies and industrial organizations in the general area of M&S as it is
related to the manufacture and acquisition of systems. On the basis of its
experience, the committee selected and focused on 10 studies sponsored by
U.S. government agencies or industrial organizations and published since
1994. These studies ranged in focus from design and manufacturing
methods, such as collaborative virtual prototyping (CVP), to broad-based
M&S strategies, to specific acquisition-related M&S areas such as SBA.
After performing its review, the committee found areas of overlap in the
recommendations of the ] 0 studies; these overlapping areas can be grouped
into the following 5 categories: (1) leadership, (2) processes, (3)
technology, (4) motivation, and (5) experimentation. Each study
emphasized these areas to a different extent, depending on the original
objective of the study, the targeted organization, and the general
environment in which the study was conducted. The following subsections
elaborate on these overlapping recommendations (see Appendix B for a
summary of the objectives and major conclusions and recommendations of
each study).
OCR for page 42
42
MODELING AND SIMULATION IN MANUFACTURING
Multiple Recommendations on Leadership
The majority of the 10 studies concluded that significant leadership
attention devoted to advancing the use of M&S in acquisition was
extremely important for its success. These studies called for leadership
either in the form of a separate office or in the form of an individual who
would act as a focal point or champion. This office or individual was
recommended to be at the most senior level of the organization to which
respective studies were directed.
Without exception, the studies recommended investment of funds to
advance the use of M&S in acquisition. These investments, applied
according to integrated plans, should be directed toward establishing the
M&S infrastructure necessary for common use, corporate and enterprise
capabilities, and/or demonstration programs. The studies also recognized
that investment in M&S would have to be made early in the life cycle of a
system, whereas many of the projected benefits would not occur until later.
Overall, the leadership recommendations focused on new organizations,
roles and responsibilities, and funding and investment.
Multiple Recommendations on Processes
As noted above, DOD's definition of SEA is directed toward making
the acquisition process more effective. Therefore, most of the 10 studies
that were focused on DOD and the services addressed the use of M&S as
an integral part of such a process. These studies found that early
involvement of operational users of systems during the development of
systems requirements and design was an important component of increased
effectiveness. Studies characterized by strong industry participation
stressed the importance of earlier industry involvement. For both
government and industry, M&S was seen as an enabler of earlier
participation.
Other common themes were the need to foster cooperation and
collaboration between government and industry and the need to promote
information sharing. Specifics included the need to leverage commercial
practices for DOD applications, the need to use collaboration technologies
effectively, and the need to provide industry with government M&S
technologies. Cultural factors and concerns about proprietary data were
often cited, however, as obstacles to effective information sharing. There
was some recognition that government policy and legislation should be
considered as factors in enabling or hindering collaboration. However,
there did not appear to be consensus on whether specific new policies and
legislation were essential or whether the flexibility in existing policies was
OCR for page 43
SIMULA TION-BASED ACQUISITION
sufficient to allow for more collaborative business practices. Overall, the
process recommendations focused on acquisition processes, business
practices, cooperation and collaboration, information sharing, and policy
and legislation.
Multiple Recommendations on Technology
43
Rapid technological advances in computers, networks, M&S, and
collaboration technologies have been the drivers for the expanded use of
M&S in acquisition. Almost all of the 10 studies reviewed included
recommendations for the application and further development of these
technologies, although several concluded that technological advance is not
the critical factor in advancing the use of M&S in acquisition.
There was consensus among the studies that the development and
application of standards is essential to the effective use of M&S-
especially standards that promote interoperabi]ity among models,
simulations, and databases. The studies recommended standardization of
underlying data schema and protocols for information exchange. Although
progress has been made by DOD (for example, HLA) and by industry (for
example, the Standard for the Exchange of Product Model Data, or STEP),
additional standards are needed, as well as a more general application of
existing standards.
Several studies recommended the development of architectures or
frameworks that would promote collaboration and the reuse of models and
simulations. Recommendations ranged from general collaborative
environment architectures to specific product data representation structures
and model and information repositories. in addition, it was noted that DOD
system performance analyses require architectures that include access to
common threat and environment representations. Although the studies
agreed that many models and simulations already exist, several called for
new or improved representations. Areas of emphasis included total-
ownership-cost models and models representing new warfare areas such as
information warfare and operations other than war.
Finally, the studies acknowledged that certain technical problems
related to M&S have not been solved and that a need exists for additional
basic and applied research. Among the areas recommended for additional
research were data security technologies, especially to accommodate
multilevel security and to protect proprietary data; multiresolution
modeling, including the aggregation and disaggregation operations within
such models; and models of human behavior. Overall, the technology
recommendations focused on tools, standards, architectures, and protection
of classified and proprietary information.
OCR for page 44
44
MODELING AND SIMULA TION IN MANUFACTURING
Multiple Recommendations on Motivation
Several ofthe 10 studies reviewed suggested that cultural issues
related to the acceptance and adoption of new technologies and business
practices may be the largest challenge in the adoption of SBA. These
studies recommended that incentives be established to motivate
government program managers and industry to implement collaboration
and the integrated use of M&S and SBA. Few specific suggestions were
made regarding the types of incentives to be established, although some
specific ideas were mentioned about how best to educate the range of
acquisition stakeholders, from small businesses to senior DOD leadership,
on the benefits and uses of M&S. These ideas included passive techniques,
such as Web pages and information repositories providing lessons learned,
as well as more active techniques, such as conferences, workshops, and
required training courses for the DOD acquisition workforce.
The studies noted that positive measures of expected returns on M&S
investments are needed to motivate DOD organizations and program
managers to make the required investments. They suggested standard
metrics related to improvements in cost, schedule, and performance, as
well as some additional quantified benefits. Overall, the recommendations
on motivation focused on education and training, incentives, metrics, and
return on investment.
Multiple Recommendations on Experimentation
Several industry and government studies emphasized the need for
experimentation to advance M&S and SBA technologies, to develop
standards and infrastructure, and to gain experience in using collaborative
processes and environments. Characterized as demonstrations, pilot
projects, or selected programs, the studies recommended that these
experiments be structured carefully with specific objectives related to
developing metrics and quantifying benefits. If existing programs were
selected, additional funding would be required to support the objectives
specific to M&S and SBA. Overall, the recommendations on
experimentation focused on demonstrations and the use of pilot and
selected programs.
OCR for page 45
SIMULA TION-BASED ACQUISITION
TABLE 2-1 Categories of Recommendations from 10 M&S Simulation-Based
Acquisition Studies
45
Category of Recommendation
Leadership
Specific Recommendations
Establishment of focal point
(senior office or individual)
Investment of funds
Processes
Technology
Motivation
Experimentation
Early involvement of users
Collaboration between government and industry
Leveraging of commercial practices
Development and application of standards
Collaborative architectures or frameworks
Research in data security
Research in multiresolution modeling
Research in aggregation/disaggregation
Human behavior modeling
Incentives to collaboration
Education of stakeholders
Metrics to measure benefits
Pilot projects
Selected programs
NOTE: See Appendix B in this report for the titles and summaries of the objectives and
major recommendations of the I O reports and application of standards.
CONCLUSIONS
The use of M&S in defense acquisition predates digital computers
and network technologies. However, with the rapid advances in these
areas, opportunities for collaboration in the development of defense
systems have been created. DOD has coined the term "simulation-based
acquisition," or SEA, to describe its vision and goals for an enhanced,
more collaborative, simulation-aided acquisition process. Several recent
and emerging DOD acquisition programs such as the U.S. Navy's LPD-
17 and SC-2 1 /DD-2 1 /DD(X) programs, the Joint Strike Fighter program,
and the U.S. Army's Future Combat Systems program—have advanced
and promoted the expanded use of M&S during their acquisition, but no
program has yet achieved the ultimate stated vision and goals for SBA.
OCR for page 46
46
MODELING AND SIMULA TION IN MANUFACTURING
Many barriers remain to more widespread use of M&S in defense
systems acquisition. These barriers include inadequate allocation of
resources, lack of information for acquisition program managers, lack of an
integrated software systems engineering process, issues related to the
protection of inte]]ectua] property rights, poor information dissemination
on SEA to the broader M&S community, and insufficient education and
training for the workforce.
This committee's review of lO government-or industry-sponsored
studies since 1994 on the subject of M&S use in system acquisition
revealed 5 genera] categories of recommendation: leadership, processes,
technology, motivation, and experimentation. Table 2- l summarizes the
more common specific recommendations in these 5 categories from the lO
studies.
Representative terms from entire chapter:
defense acquisition
