National Academies Press: OpenBook

The Role of Experimentation in Building Future Naval Forces (2004)

Chapter: 5 Effectiveness of Experimentation for Future Naval Capabilities

« Previous: 4 Emerging Roles in Experimentation--The Joint Connection
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

5
Effectiveness of Experimentation for Future Naval Capabilities

This chapter assesses the effectiveness of experimentation for future naval capabilities. It addresses the specific questions of the terms of reference, beginning with a review of what has been learned from naval experimentation and an assessment of the success of transitioning results from experiments to the fleet and field. The assessment then naturally leads to an examination of the adequacy of naval experimentation programs, processes used, and the environment and infrastructure that support experimentation. The chapter concludes with an examination of the broader relationship of experimentation within the joint context, specifically looking at the effectiveness of naval experimentation in preparing for joint operations and at the relationship between Service-unique and joint experimentation.

ASSESSMENT OF EXPERIMENTATION RESULTS

The terms of reference ask what has been learned from experimentation thus far. From a historical perspective, experimentation with innovative technologies in military systems and with techniques of warfare has often led to revolutionary changes in how military forces are constituted and how they fight. Examples from the previous century include the use of armored forces in land warfare, the use of submarines and sea-based aviation in naval warfare, the application of ballistic missiles to intercontinental attack capability, the use of nuclear submarines to constitute long-enduring and essentially invulnerable undersea tactical and strategic strike forces, and the use of space systems for observation, communication, and navigation.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

The Navy and Marine Corps have a long history of using experimentation to evolve significant new capabilities. The Navy experimented with submarines, carriers, and PT boats, and with new propulsion systems and new fuels. Experimentation with launching and recovering aircraft from ships began but a few years after aircraft were invented. Similarly, although Marines had been landing from ships in small boats for many decades, even centuries before World War II, the pressures of warfare led to the rapid development of prototypical modern amphibious landing systems. Experimentation has been key to advances in all forms of naval warfare—gunnery, guided missiles, naval ship propulsion, vertical-capable jet aircraft, very short takeoff and landing rotorcraft, and all other activities related to the shaping and operation of naval forces, including the Fleet Antiterrorism Security Teams and the Chemical/Biological Incident Response Force (CBIRF) of today.

In order to assess more recent Service-unique experimentation, the committee focused primarily on the Navy Warfare Development Command (NWDC)-sponsored fleet battle experiments1 (Alpha through India) and on the Marine Corps experimentation efforts beginning with the Hunter Warrior (HW), Urban Warrior (UW), and Capable Warrior (CW) campaigns. Fleet Battle Experiment-Alpha (FBE-A) was conducted in March 1997. The Hunter Warrior series of activities was initiated in 1997. Chapter 3 provides details on these efforts.

FBE-A through FBE-I included a total of nearly 40 separate objectives. Each FBE had between three and eight major objectives, and each major objective had anywhere from one to nine subobjectives. A significant number of these major objectives and subobjectives were realized. The range of investigation of these FBEs was quite extensive, addressing network-centric operations for naval and joint fire power, theater and air missile defense, precision engagement, time critical strike, and defense against asymmetric threats, to name a few areas. Table 3.1 in Chapter 3 provides a synopsis of all FBEs, their objectives, and their results. Collectively assessed, these provide evidence that experimentation is achieving meaningful results.

The three experimentation campaigns HW, UW, and CW also addressed many objectives. HW had 37 objectives, of which 29 were realized. As a campaign, it focused on individuals and combat patrols operating in desert environments. UW addressed individuals and platoon-size operations in urban environments; CW focused on individuals and company-size operations at Camp Pendleton, California. Each of the campaigns required a cycle of more than 3 years. As with the FBEs, these campaigns covered a considerable range in their investigations,

1  

The nominations for and participation in FBEs involve many organizations, as discussed in Chapter 3. For example, these organizations have included the Second, Third, Fourth, Fifth, Sixth, and Seventh Fleets, and regional commanders, who propose experiments through the Navy Component Commanders in their area of responsibility.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

including the use of nonlethal weapons, small unmanned aerial vehicles (UAVs), sensors, intrasquad radios, precision targeting, and the development of a common tactical picture. Each addressed a series of important questions, such as how to extend the effectiveness of a modest forward afloat expeditionary force or how to penetrate and operate in dense urban littorals. In short, many objectives of each campaign were realized and valuable knowledge was derived. The lessons learned from these campaigns were captured in the Marine Corps “X-files.”2

To the terms-of-reference question posed at the beginning of this section, the committee responds that both Navy and Marine Corps experimentation is enabling learning and producing meaningful results directed at promising concepts and technologies in a number of key naval mission areas. However, other questions that remain to be answered in this chapter go to the heart of the matter—which is whether naval experimentation is as effective as it can be and needs to be.

ASSESSMENT OF TRANSITIONING

A key question from the terms of reference is how successful the transitioning of the results of experimentation to the field has been. The committee responds by focusing on (1) doctrine and tactics, techniques, and procedures (TTPs) and (2) fielded capabilities, including acquisition programs. Summaries are provided in Table 5.1, which synopsizes results from the discussions on naval experimentation in Chapter 3. More detailed observations are presented below.

Summary Observations

One of the important outcomes of the Navy’s FBE series of experiments has been the determination as to why certain objectives were not achieved, which allows for an iterative process of improvement. The principal successes of FBEs have been as follows:

2  

“The X-Files contain useful information packaged for rapid reading and easy transport in the cargo pocket of the utility uniform. They convey a synthesis of knowledge gained from experiments with tactics, techniques, and procedures, and some enabling technologies that can help us fight and win battles. Most of them focus on operation in the urban battlespace. They are an evolving body of knowledge that is constantly refined through experimentation…. information in the X-Files is entered into the Marine Corps Combat Development System. It forms the backbone of recommended revisions to Marine Corps doctrine for Military Operations on Urbanized Terrain (MOUT). Knowledge in the X-Files also underpins much of the Basic Urban Skills Training (BUST) program used by the Operating Forces. The X-Files gather, organize, and synthesize knowledge from post training analysis and feedback from Marines, Sailors and other participants in MCWL experiments. They do not contain official doctrine, nor are they policy or standing operating procedures (SOPs).” For further information see <http://www.mcwl.quantico.usmc.mil/x_files.asp>. Accessed November 18, 2003.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

TABLE 5.1 Transitioning Results and Conclusions of Experimentation into the Navy and Marine Corps

Result

U.S. Navy

U.S. Marine Corps

Doctrine and tactics, techniques, and procedures (TTPs)

Some concepts have moved into the field

Many concepts have moved into the field.

Fielded capabilities including acquisition programs

Few successes so far. Naval Fires Network is one notable concept that appears to be in transition to the field.

Numerous successes in transitioning small end items as well as interim capabilities.

 

Transitions are very difficult, and processes for achieving them are seen as poor, even from within the Navy, owing in part to budget pressures and to a lack of processes for new capabilities to compete with programs of record.

Transitions have proven very difficult for expensive capabilities because of budget pressures and ineffective processes for new capabilities to compete with the programs of record.

  • Demonstrations of the feasibility of new operational concepts using surrogate or prototype or existing systems,

  • The adoption by fleet forces of new TTPs, and

  • The development of new doctrine for fleet operations.3

The Naval Fires Network (NFN) appears to be a case of successful experimentation in the process of transition for the Navy. NFN is a network-centric warfare system that provides real-time intelligence correlation, sensor control, target generation, mission planning, and battle damage assessment capabilities. It allows ships and aircraft in a carrier strike group or an expeditionary strike group to share near-real-time and real-time intelligence and targeting information, not only with one another but also with Army and Air Force units in a joint or coalition task force. NFN was first studied in FBE-A and then refined in FBE-I. It has transitioned to the fleet as an interim prototype on two carrier strike groups, owing to intense interest from the CNO and fleet commanders. However, to date it has not become a formal program of record.

Marine Corps successes, based on the Hunter Warrior, Urban Warrior, and Capable Warrior campaigns, are as follows:

3  

In Chapter 3, see the section entitled “Synopsis of Results to Date from Fleet Battle Experiments Alpha Through India.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
  • Concepts, doctrine, and TTPs resulting from experimentation have transitioned successfully to forces in the field.

  • Experimentation has resulted in changes in minor equipment items in the field.4

It is clear that naval experimentation is resulting in new doctrine, TTPs, some new concepts, and some minor end items. It is also clear that there is a serious shortfall in transitioning the results of experimentation into major fielded capabilities. For the Navy, this extends to minor items of equipment and capabilities as well. The committee’s findings with respect to the success of transitioning results of experimentation to the field are as follows:

Finding for Navy: The mechanisms and processes for transitioning the results of experimentation directly to the fleet or to an acquisition program of record are inadequate, and they curtail the effectiveness of experimentation in building future naval forces.

Finding for Marine Corps: The Marine Corps has been successful in transitioning nonmaterial elements of doctrine, organization, training, materiel, leadership, personnel, and facilities (DOTMLPF) and minor equipment. However, it has not been successful in transitioning to combat forces major warfighting capabilities identified during experimentation.

More detailed discussions related to these findings follow.

Transitioning to Naval Doctrine and Tactics, Techniques, and Procedures

One objective for experimentation is to explore or assess new doctrinal concepts and, later, to develop and refine new TTPs. The committee noted that the NWDC is sensitive to the need to coevolve doctrine and TTPs at the same time that new technology is introduced. The Marine Corps Combat Development Command (MCCDC) also emphasizes this strategy—partly because in some early experiments problems had arisen as a result of insufficient attention to retraining participants in the use of new capabilities. The lessons of the Gulf War in 1991 and of the conflicts in Bosnia and Kosovo (where sustained operations required superb command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR)), and in Afghanistan (where Special Forces and Marines operated directly together with the use of long-range bombers) were consistent with studies and games conducted over the years. Without the urgency of an actual war, however, it is unclear that change would have happened,

4  

In Chapter 3, see the subsection entitled “Synopsis of Results of Sea Dragon to Date.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

since at a certain stage, rational change driven by persuasive analysis, modeling, simulation, and gaming is not yet supported by empirical evidence. Nonetheless, the “wild ideas” practiced in experiments such as Hunter Warrior, Urban Warrior, and fleet battle experiments were turned into reality when the time came—because individuals exposed to the ideas and potentials were willing to break with tradition and doctrine to do sensible things. That said, some results produced from Navy and Marine Corps experimentation have led to new concepts, doctrine, and TTPs, as well as new training initiatives.

FBEs have been successful in the process of developing and transitioning into the fleet decision-support concepts and tools intended to increase the speed of command. Other FBEs enabled the collaboration of command echelon decision makers. One specific effort is the invention of a Knowledge Web (K-Web) concept.

As described in Chapter 3, the K-Web involves the application of knowledge-management practices to warfighting, creating a concept of operations in which value-added information (i.e., knowledge) is created and published on the command intranet in real time rather than being coupled to daily briefing cycles. The concept was initially developed in war games, installed as a prototype system on the USS Carl Vinson, and eventually battle-tested during Operation Enduring Freedom, where it was viewed very favorably. The Network-Centric Information Center is working to migrate these tools and transition them to other battle groups. The K-Web demonstrates a significant first step toward a new concept of operation for warfighting and is explicitly designed to support distributed collaboration.

Other examples of successful transitions resulted from the Marine Corps Hunter Warrior, Urban Warrior, and Capable Warrior campaigns, which shaped doctrine and TTPs in the field in the following areas:

  • Introduction of new command relationships,

  • Development of the Combat Decision Range,

  • Construction and use of the urban Close Air Support range (Yodaville),

  • Development of military operations in urban terrain (MOUT) for infantry tactics,

  • Continued refinement of the CBIRF organizational structure, and

  • Construction of the night laboratory for training infantry tactics.

In particular, the results of Urban Warrior have influenced major changes in the doctrine of how to fight in urban terrain. Additionally, many of the training lessons learned have been captured in the X-files, which are distributed throughout the Marine Corps down to the squad level. The development and building of the night laboratory for the training of rifle squads and all lieutenants at the Basic School (the primary school of basic training for all Services) are a direct result of HW and UW. Another training tool, the Combat Decision Range, is also a direct

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

product of the experimentation program. The use of paintball and chalk small arms rounds for urban force-on-force training is also a result of the Marine Corps experiments in urban warfare. The instrumented Close Air Support range (Yodaville) constructed near Yuma, Arizona, is a direct result of UW and is currently used by Air Force, Navy, and Marine Corps pilots.

Transitioning Results to Acquisitions and Fielded Capabilities

In reviewing the experimentation efforts of the NWDC and the MCCDC, the committee concludes that there have not been any major end items of equipment that have moved directly into acquisition as a result of experimentation.

Navy

The Navy’s process for turning concept development over to the acquisition system is viewed from within as being poor5 (discussed in more detail later in this chapter). Instead, one sees examples of the Navy moving directly to the procurement of interim capabilities, sometimes prematurely. Some critics of the Naval Fires Network believe that it was rushed into the fleet with insufficient consideration of alternative ways to achieve the same ends, while supporters view the movement as an opportunity to understand the NFN’s potential and to use that knowledge to guide its further development and effectiveness.

However, as noted in Chapter 3, Navy experimentation results (e.g., with the high-speed vessel6) have influenced some acquisitions, such as the Littoral Combat Ship. The route of entry seems to be through interested individuals in leadership positions, evolutionary upgrades of existing systems, or modification of the design concept or implementation plans of ongoing programs of record.

The NFN has the potential for the most successful transition of the results of an FBE-tested concept, although it is not yet a formal program of record. The successful test of the NFN in FBE-I and its subsequent introduction into the fleet as an interim prototype on two carrier strike groups demonstrate that experimentation may be used as a vehicle for the rapid introduction of new capabilities into the fleet.

5  

Admiral Dennis Blair expressed his concerns on this matter while still serving as U.S. Commander in Chief, Pacific (see ADM Dennis C. Blair, USN, 2002, “Force Transformation in the Pacific,” remarks at U.S. Naval Institute/Armed Forces Communications and Electronics Association Western Conference 2002, San Diego, Calif., January 15, pointing to examples of systems readily available in the civilian economy that have not yet been adapted for use in the operating fleets.

6  

As with its simulated use in FBE-J, the high-speed vessel is serving as a command and control platform and staging base in Operation Iraqi Freedom. See Jason Ma, 2003, “Catamaran Deployed in War: Naval Special OPS Use Joint Venture in Operation Iraqi Freedom,” Inside the Navy, Vol. 16, No. 13, March 31, p. 1.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Marine Corps

The committee observed that the Marine Corps has had some successes in getting interim minor end-item equipment capabilities fielded as a result of experimentation—for instance:

  • Nonlethal weapons sets for deploying Marine Expeditionary Units,

  • Laser eye protection kits for deploying units,

  • PAQ-4 night alignment sight for the M-16,

  • Intrasquad handheld radios for all USMC units,

  • Elbow and knee pads for MOUT operations, and

  • Infrared-treated battle dress uniforms.

Successful results of experimentation for major equipment items have been very difficult, if not impossible, to transition to fielded capabilities. Examples of successfully transitioned results include:

  • Interim fast attack vehicles,

  • Precision targeting systems for forward observer/forward air controllers (FOFACs), and

  • The Dragon Eye UAV.

The interim fast attack vehicle (discussed in Chapter 3) has been fielded to the operating forces, while the program of record—the light strike vehicle—works its way through the acquisition system. To the committee’s knowledge, the debate on displacing the program of record did not occur. The commercial off-the-shelf (COTS) intrasquad radio (experimented with during Urban Warrior) has been fielded as an interim capability, while the formal acquisition system works on the Joint Tactical Radio System.

The forward observer forward air controller system equipment, barely workable during Hunter Warrior, was improved in terms of its reliability, ruggedness, and performance during Urban Warrior; the prototype systems are now deployed with units. Meanwhile, the formal FOFAC program of record—that is, the Target Location and Designation Handoff System (TLDHS) and the Target Handoff System (THS)—is working its way through the aviation acquisition system.

Additional experimental equipment currently in operational use includes the Dragon Eye UAV, which was sent to the First Marine Expeditionary Force in Kuwait to support Operation Iraqi Freedom. This small reconnaissance and surveillance system provides real-time video imagery and can be put in a backpack. An acquisition program for the Dragon Eye is scheduled to follow its extended evaluation by users, although it has already undergone more limited development and operational evaluation, as well as participation in the Millennium Dragon ’02 Marine Corps experiment.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Transition Planning

Logically, one might expect that well-designed and well-executed experimentation campaigns would be so persuasive that the transition of equipment from them into acquisition would be straightforward. In practice, this is clearly not the case. The Marine Corps instituted formal planning to facilitate transition. The Navy does not appear to have done so, relying on personal interactions by leaders to move capabilities forward.

The lack of a formal transition plan is only one impediment to successful transition. Success depends fundamentally on satisfying a requirements process and on having resources made available in advance. That is, if an experiment campaign ends gloriously but no funds have been provided in the Service Program Objectives Memorandum, exploiting the fruits of experimentation proves to be extremely difficult for obvious reasons. A solution should rely on planning and programming in advance and/or on invoking a process that amounts to a competition between the capabilities developed through experimentation and the program of record. Since the program of record represents the results of many compromises and a prioritization by the senior leadership, the notion of supplanting an agreed-to program with a new one emerging from an experiment is difficult. Since time is important, the transition process needs to start early—at the time the experiment campaign is planned, in anticipation of success.

Not all experimental capabilities warrant transitioning by displacing a program of record. Potential capabilities may remain promising but unrealized because their introduction into the field is not as effective a return on investment as is the capability already programmed. Yet if experimentation is to provide results to the fleet, the debate should be allowed. The present process is woefully inadequate; it almost guarantees that the results of successful experimentation will not enter the acquisition process or become the basis of a program of record.

ASSESSMENT OF SPIRAL DEVELOPMENT

There were two sources of motivation for considering spiral development in this report. First, the terms of reference for the study specifically asked the committee to do so. Second, the network-centric operations report of the Naval Studies Board recommended that the Navy and the Marine Corps apply spiral processes within the experimentation framework as a means to develop concepts and processes and to accelerate fielding capabilities.7

All of the groups delivering briefings and written materials to the committee referred to spiral development, but the term carried very different meanings.

7  

Naval Studies Board, National Research Council. 2000. Network-Centric Naval Forces: A Transition Strategy for Enhancing Operational Capabilities, National Academy Press, Washington, D.C.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

Many individuals seemed knowledgeable about the spiral development process. In some cases, what was described as spiral development was actually traditional product improvement in phases.8 Relatively little evidence was seen in naval briefings of an explicit planning process for true spiral development, which emphasizes iterative cycles of prototypes with substantial learning and adaptation based on operator feedback to drive subsequent spirals before delivering a fielded capability.

More typically experiments—such as those using commercially available high-speed vessels—amount to the use of a prototype or surrogate that approximates what will eventually be required. The information from one experiment featuring the vessel is used to shape a subsequent experiment and to answer additional questions. However, such a sequence of events does not constitute a deliberate spiral acquisition. The submarine community used a spiral methodology for the Advanced Rapid COTS Insertion (ARCI) program to address the (then) loss of acoustic advantage in U.S. submarines.9 Another example of experimentation used to support a continuous evolutionary improvement is that of the FOFAC, cited above. Developed under Hunter Warrior, this system was the subject of enhancements during Urban Warrior and resulted in an interim fielded capability.

These things said, the committee was struck by the manner in which naval experimentation is disconnected from the Navy’s acquisition community. In some briefings it was asserted that experimentation was not relevant to acquisition. Only in a few instances was there evidence of a deliberate, planned connection—to apply what was learned in experiments to shape system acquisitions or to turn the results into a coherent acquisition program.10

It appears as if some Navy interest in experimentation is really based on a desire to bypass the acquisition process. This is understandable, given the notoriously long times for normal system acquisition. But the price paid is high: By hopping from concept development to experiments and interim capabilities, the Navy is bypassing the organizations with the expertise to do systematic engineering, capabilities-based planning, and operational assessments. The committee also observed that the Navy’s acquisition components seem to be viewed by some leaders as a major part of the problem, rather than as part of the solution. Nonetheless, the Naval Fires Network could be cited as an example of something that leaped quickly from concept demonstration to directed procurement. The

8  

Virtually all major systems have involved evolutionary development and deployment under names such as “block change” or “planned product improvement strategy.”

9  

In Chapter 3, see the subsection entitled “A Recent Experimentation Program: Advanced Rapid Commercial Off-the-Shelf Insertion.”

10  

The most prominent example cited is that of the Littoral Combat Ship, which is using experimentation with the high-speed vessel to determine key performance parameters.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

basic idea is good and interim capabilities are much to be desired, but a refined design is probably needed for all but the near term. The NFN could be viewed as an example of a capability that could profit from a deliberate spiral development process managed collaboratively by the experimentation and acquisition communities. Such problems can be remedied in part by the advanced planning process described above, since the initial experimentation with the NFN would have been viewed as part of a campaign, with orderly provision for acquisition built in at the start.

The committee concluded that the spiral development process is used at the discretion of the individual manager and is confined to discrete experimentation events or individual acquisition capabilities rather than being applied systematically. An assessment of the effectiveness of this methodology could not be made because its use is too sporadic. The committee believes that the Navy and the Marine Corps have not yet adequately explored spiral processes or spiral development, particularly in the context of experimentation campaigns, as was recommended by the network-centric operations report of the Naval Studies Board.

ASSESSMENT OF THE NAVAL EXPERIMENTATION PROGRAM AND ITS METHODS

The terms of reference specifically ask the committee to determine whether any important questions were omitted from the naval program of experimentation. They also request an assessment of the adequacy of processes and methods, particularly in planning. As discussed below, these are not unrelated questions. In short, there are key omissions from the current program of experimentation, and some of these result from inadequate methods.

Surveying Future Challenges

A key strategic question is whether the naval experimentation programs are focused well strategically. The committee concludes that, at the highest level, the Navy and the Marine Corps are both doing well here. They are both “capabilities-oriented,” as distinct from being oriented toward specific scenarios. This is consistent with DOD guidance11 and long-standing naval traditions. For example, the components of the new naval force strategy Sea Power 21 (e.g., Sea Strike, Sea Shield, and Sea Basing) are quite general. So also, the primary enabler being emphasized (FORCEnet) describes a broad capability rather than something designed for a narrow function suitable in only special cases.

11  

See Donald H. Rumsfeld, Secretary of Defense, 2001, Quadrennial Defense Review Report, Washington, D.C., September 30. The classified defense guidance issued in 2002 reportedly reinforced the emphasis.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

It is less clear whether this breadth of coverage is realized in more explicit detail. The committee did not see evidence of a comprehensive and rigorous survey of capability needs. The naval experimentation program certainly addresses issues that have been highlighted in high-level planning documents, such as assuring access and projecting force even in the presence of asymmetric strategies. However, the translation of such high-level challenges into concrete problems to be worked on and into priorities for doing so appears to be accomplished in a rather unstructured way, without the benefit of rigorous studies and critiques.12

Another concern is that the number of concepts being pursued in experimentation is small and seems to be driven by particular ideas (such as the crosscutting value of network-centric operations) rather than by an attempt to be systematic and to address problems that the Navy knows it is going to face. These would include problems arising either because threats can be foreseen or predicted, or because new capabilities are being pursued and their effects and modus operandi will have to be explored in detail. Arguably, the NWDC and the MCCDC should be responsible for addressing the full range of worries about future naval capabilities, not just one or a few concepts. This is especially so in connection with planning for major transformational changes; history is not encouraging about such matters. Often, nations have worked diligently to develop and perfect concepts that ultimately proved wrongheaded or unsuitable to the war that actually developed.13

A rejoinder is that there are not enough resources to do everything at the same time. In fact, however, it is possible to address a wide range of problems in parallel. This is an essential part of planning under uncertainty, as is required in capabilities-based planning. To be sure, only a few concepts can be addressed with the highly personnel- and fleet-asset-intensive activities that are usually equated with fleet battle experimentation. However, there is no need to pursue everything using large field experiments. Analytical studies and small-scale experiments would suffice for addressing many issues.

Certain important areas are not yet adequately explored in the naval experimentation programs. Some of these are currently gaining definition.14 Many

12  

The concern here is not just the range of political-military scenarios but also the range of assumed circumstances (e.g., the enemy’s strategy and tactics, initial conditions that may include surprise or other setbacks, the nature of both sides’ allies, the real-world effectiveness of weapons and systems never used in such circumstances before, and so on).

13  

See Paul K. Davis, 2002, “Integrating Transformation Programs,” Transforming America’s Military (Hans Binnendijk, ed.), National Defense University Press, Washington, D.C., pp. 193-218.

14  

In January 2003, the CNO requested that the Commander, Fleet Forces Command—as part of the lead role for Sea Trial in support of Sea Power 21—“draft and implement a comprehensive roadmap (by May 2003) that integrates studies, wargames, experimentation, and exercises with evaluation metrics and an execution timeline.” (Commander, Fleet Forces Command. 2003. Sea Trial—Concept Development and Experimentation Campaign Plan (U), Working Paper (draft), Norfolk, Va., May (Classified).)

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

candidate areas for experimentation emerge from the spectrum of problems facing the Navy in implementing Sea Power 21. Others emerge from questions about capabilities expected to be delivered by programs of record; many of these have been experimented with, but not conclusively, and deserve further investigation. They include over-the-horizon, time-critical strike; use of extended-range guided munitions (ERGMs) for long-distance, high-volume, rapid fire support; expanded applications for network-centric capability to deployable undersea sensor arrays; mine/countermine warfare; the use of UAVs to locate and identify targets; operational concepts for cruise missile submarines; Marine operations in built-up areas; sea-basing concepts such as logistic ship configurations with loadable containers; and advanced support and logistics concepts. The experimental work done to date has brought out overarching issues, such as achieving a satisfactory common operating picture (COP); deconfliction;15 and bandwidth size and management. And some important areas not yet explored include Vertical Launch System (VLS) reloading at sea, assault breaching of mine and obstacle fields near and on the beach, and continued decisive operations under impaired network conditions and under unfavorable environmental conditions.16

The committee believes that these areas, some of which were not mentioned in its Navy and Marine Corps briefings, anticipate legitimate issues with many facets and typify the kinds of experiments that the Navy and the Marine Corps should consider moving from the program of record toward the full Sea Power 21 capabilities. Furthermore, many of these individual areas of experimentation would fit into larger experimentation campaigns, as discussed in that context in Chapter 2. The NWDC recently drafted a Sea Trial experimentation campaign plan.17 The committee believes that this is a step in the right direction, though the plan’s impact on the Navy is as yet unclear. Nevertheless, the noted omissions and inadequacies sustain the committee’s concerns about the need for a more robust program and result in a finding for the Navy:

Finding for Navy: As yet, no cohesive experimentation program exists that will move the Navy’s forces to “future” concepts, processes, doctrine, and capabilities.

In further assessing Navy and Marine Corps experimentation, other questions arise: (1) Does concept development address the range of challenges? (2) Are multiple, competing concepts developed and debated for each? (3) Are

15  

Deconfliction is normally understood as “deconfliction of air space” to ensure that two air vehicles (aircraft, missiles, projectiles) do not occupy the same air space at the same time.

16  

An expanded set of candidate areas for future experimentation is provided in Chapter 6.

17  

Commander, Fleet Forces Command. 2003. Sea Trial—Concept Development and Experimentation Campaign Plan (U), Working Paper (draft), Norfolk, Va., May (Classified).

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

the concepts generated well chosen and well conceived? These are considered and answered in the discussion below.

Navy Warfare Development Command

The committee was impressed with how much the Navy Warfare Development Command has accomplished over the past few years. The progress made in a short time is remarkable. One of the laudable, recent changes is that of sharpening the extent to which concepts drive experimentation. That change is likely to improve the coherence of the naval experimentation program as well as to improve the sharpness and quality of the concepts themselves. The command’s vigor and enthusiasm were also high, and the command could point to recent successes related to high-speed vessels and the FBE-J experiments. However, as indicated by the committee’s discussions and reading, it seemed that the number and range of operational concepts being pursued are small, that debate is informal and internal, and that—despite its efforts—the NWDC has not been sufficiently connected to all of the potential stakeholders (e.g., the fleets and the acquisition community). All of these factors underscore the committee’s concerns about the adequacy of the future program of experimentation, already noted in the finding directed to the Navy in the preceding subsection. The CNO’s decision to align the NWDC under the Fleet Forces Command may prove useful in mitigating these problems, but may also result in too great a focus on the near term.

The committee was troubled by the sometimes overly rapid jump from “concept development” (which is sometimes criticized as too “soft” or “conceptual,” without sufficient grounding in engineering and other technical analysis) and longer-term commitment to high-stakes experimentation and even fast-track acquisition. A number of those consulted during the course of this study acknowledged that this was a problem. Unfortunately, the Navy’s acquisition community, which has the knowledge and expertise to translate soft concepts into more rigorous constructs and capability needs, is sometimes only minimally involved, and often not at all.18

One reason that the acquisition community is not more closely involved on a routine basis may be leadership interest in using experimentation to bypass the sometimes-ponderous acquisition system. The desire to speed up innovation is both understandable and laudable. However, there is clear need for a process in which concepts are exposed to initial systems analysis and systems engineering before they are tested in the field and during the field evaluation process. This step, which does not exist now, is needed and could be accomplished by involv-

18  

Arguably, the problem is process—the lack of a clean mechanism for transitioning from successful experimentation to acquisition. Individuals sometimes transcend the system, but success depends on personal relationships and efforts that surmount ongoing processes.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

ing both stakeholders and well-chosen teams from the relevant Systems Commands in carefully crafted (but not necessarily large) experimentation events.

Marine Corps Combat Development Command

Currently, the Marine Corps appears to be operating effectively in its concept formulation—in part because of the nature of the concepts it is exploring and in part because it operates in a tightly integrated system. The organizational structure of the Marine Corps, with the MCCDC responsible to the Commandant for experimentation and also responsible for DOTMLPF integration across all capabilities, facilitates a thorough examination of all results from several perspectives. Requiring that the Marine Requirements Oversight Council (MROC) approve the Marine Corps experimentation plan and review all claimed results helps to keep efforts focused in the intended direction, while causing senior leadership—not just the Commandant—to take responsibility and to share ownership. Also, MROC involvement helps create a culture that is proactive toward experimentation. All of the Marine Corps Service experiments are linked directly to advanced warfighting concepts. The experiment objectives and concept linkage are published in the respective experiment plans.

One major drawback of the current approach is that the Marine Corps experimentation plans are not focused on long-term capabilities for the forces. Instead, experimentation plans and objectives are tied to change and improvement, tending to deal with small-scale issues and to address improved infantry TTPs. They are along the lines of moderately paced evolution consistent with the intuition of current senior leaders. Experimentation needs to address challenges and capability improvements for the near, middle, and long term, although the respective levels of effort can and should vary substantially. For balance, there should be a stronger component of more revolutionary experimentation.19 The preceding observations lead to this finding:

Finding for Marine Corps: The Marine Corps has moved from a balanced program of experimentation campaigns based on near-term, mid-term, and long-term objectives to one of experiments focused on near- and mid-term objectives.

The committee observes that many problems of a long-term nature need continuing attention in experimentation, sustained over time so as to affect capabilities in the field. Examples include the following:

19  

Early aspects of the Sea Dragon exercise program had this character, but it proved difficult to move from experimentation into fielded capability. The current approach is more evolutionary but also more effective organizationally in bringing about continual change.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
  • Sea basing. This concept establishes a commendable new approach for the operation of naval forces. It requires only minimal dependence on the assistance of host nations and their facilities. Close cooperation with the Navy experimentation community and others is required to continue exploration and development of the sea-basing concept. Leaving much of the force’s support at sea will require new designs for ship or other types of offshore platforms that allow selective offloading and reconfiguration of cargo containers on board; at least limited on-board support of large, vertical-lift aircraft, if not more radical solutions; lighterage for over-the-beach logistics in rough sea conditions up to sea-state 3 or 4; and the ability to support forces ashore from many miles out at sea in order to avoid shoreward defenses such as mines and short-range antiship missiles. These are radically new conditions for supporting a naval force ashore. Learning how to implement them effectively will require much experimentation, from the level of the individual ship to that of large-scale forces. Such a program of experimentation remains to be designed, funded, and carried out. And once a problem’s physical parameters have been ascertained by such means, the results will have to be translated into new systems, doctrine, and procedures that will have to enter the naval forces’ budgets, acquisition, training, and operational programs according to schedules designed to bring the capabilities into effect as early as feasible.

  • Littoral combat. U.S. naval power projection is built on the foundation of mastery of the seas—being able to operate where and when required. Today, this mastery exists on the open seas—on “blue water”—but it is not fully realized for “brown water” littoral naval forces. Major problems in littoral waters include enemy resistance in the form of mine warfare, antiship missiles, armed small boats that can attack and damage U.S. logistics ships, and possible use of chemical or biological weapons against near-shore operations of naval forces. In addition, the complex orchestration of Marine Corps landing and support forces, Navy surface and air fire support, and defensive operations to protect the landing forces at sea and ashore will require many changes in the C4ISR system on a joint Navy and Marine Corps basis. These changes will also have to be extended to other Services that may be involved in an operation. Much experimentation remains to be done, from the small-unit to the force level, to devise the system requirements, the TTPs, and the joint C4ISR coordination for full success in such operations.

  • Unconventional warfare. Unconventional warfare, as exemplified by the operations of Marine Expeditionary Units (Special Operations Capable), has been part of naval operating procedures for many years. However, changes in sea basing and brown water littoral operations will require the adaptation of current doctrine and TTPs. Experimentation under these new conditions or as part of support to the Special Operations Forces-related naval forces will be necessary. For instance, with more use of unmanned underwater vehicles, unmanned aerial vehicles, and new modes of landing, it will be necessary to provide fire power

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

and logistics support, commercial transport, pathfinder, and the Seals’ underwater vehicle operations.

Methods—Experimentation on Concepts

Assuming that good concepts and their priorities are established, the next question is whether appropriate methodology is employed to pursue the concepts that undergo systematic experimentation. Chapter 2 describes a canonical approach to experimentation campaigns, which involves an overall campaign strategy, warfighter reviews, wargaming, simulation, and field experiments (as are needed and feasible). Studies and analyses constitute a systematic, overarching activity, rather than merely a preparation for and analysis of data from some particular experiment. Analysis should encompass the development of theory to make sense of the whole, systems analysis and systems engineering, empirical analysis, and, ultimately, policy analysis leading to choices among the concepts regarding their embodiment in doctrine and their further development for fielding.

During the past few years, the Navy, the Marine Corps, other Services, and the U.S. Joint Forces Command (USJFCOM) have all embraced this approach to some degree. Those involved in experimentation programs usually acknowledge the importance of all of the activities (not just the big events). The NWDC and the MCCDC both construct experimentation campaign plans with a mix of the various types of venues and tools. Both organizations also work with USJFCOM to develop the USJFCOM campaign plans mandated by Congress.20 All of these campaign plans reflect considerable learning over the last 6 years or so.

Despite this progress, the committee concludes that preparing for the event of a fleet battle experiment has largely (but not exclusively) been the focus of attention in the Navy’s experimentation program. Studies, analysis, and “writing the book” (i.e., developing definitive knowledge and understanding aimed at applying the results) should play more central roles, but they are underemphasized activities today. The NWDC’s experimentation campaigns require expansion and structure to ensure thorough learning of the relevant phenomena and options for improved capabilities by careful selection of experimentation venues, rather than overemphasis and over-reliance on the success of a major event. This is a relevant and needed change in strategy for the Sea Trial process.

The “big event” of a fleet battle experiment is very important and sometimes critical for exploring scalability and integration, demonstrating potential, and building broad-based enthusiasm and support. But it is neither an end in itself nor, in many cases, an appropriate culmination. After such an event, more remains

20  

See, for example, U.S. Joint Forces Command, 2000, 2001, and 2002, Joint Concept Development and Experimentation Campaign Plan, Norfolk, Va., September. Chapter 4 discusses USJFCOM developments in more detail.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

to be learned, partly from operator feedback, partly from continued experimentation. Furthermore, the circumstances of the big event are often singular and atypical of wartime operations in at least some important respects, such as force structure and threat scenarios.

An interesting contrast is struck between the classic development of nuclear submarines and sea-launched ballistic missiles on the one hand,21 and the current pursuit of FORCEnet, on the other. The former was legendary for its depth, quality, discipline, and engineering prowess. The latter, as best the committee could judge from presentations and materials received, is being pursued in an unstructured way, including relatively ad hoc experimentation with little underlying architecture and design. Today discipline in developing and structuring campaigns is sometimes evident, but oftentimes not.22 The emphasis appears to be on concepts that are sometimes not fully developed, competed, and critiqued and their demonstration under circumstances that do not provide the information required.

The NWDC has made some changes in processes. In the past, concepts were proposed at a high level, building momentum that would extend throughout the experiment planning cycle. The maturation and development of a concept could extend up to the final planning conference for the experiment. Initiatives, derived from concepts, were also kept at a high level, and the objectives for experimentation then followed; these too could be introduced late in the process. Such continuous changes in concepts and objectives placed pressure on the experiment design.

In a recent approach to planning, concepts were developed by Warfare Innovation Development Teams (WIDTs).23 As concepts and doctrine co-evolved, a set of related objectives were developed for the FBE. Concepts and associated objectives were produced at the front end of the experimentation process and were more stable throughout that process. Each team was responsible for a set of activities that consist of war games, meetings, limited-objective experiments (LOEs), exercises, real-world experiences, and so on. FBEs may or may not be the culminating event.

21  

In Chapter 3, see the subsections entitled “A Case History in Past Experimentation: The Early Development of Naval Aviation,” and “Evolution of the Linear Development Model” for a discussion of past successes in developing new capabilities.

22  

Planning for Fleet Battle Exercise-Juliet (FBE-J), which was conducted in 2002, was much more systematic and rigorous than planning for the previous FBEs. However, the focus was on methodology for the big-event experiment, not on preparing a definitive analysis of, say, naval firepower in the network-centric era.

23  

The committee understands that NWDC has recently changed this process somewhat. There are still five WIDTs, now called Sea Strike, Sea Basing, Sea Shield, Information and Warfare Advantage, and Combating Terrorism/Force Protection. The committee believes its comments are applicable to this process even if organized differently.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

The committee believes that these changes hold promise for some positive consequences. They improve stability for good experiment design and place emphasis on selecting the right kind of activity to address the problem at hand. Nevertheless, as indicated in Chapter 3, there are concerns with this process:

  • The process by which a concept becomes worthy of further exploration by a WIDT is not clear.

  • Although more time may be available for better experiment design and planning, the process must be stable enough to produce better results.

  • It is also not clear how feedback and knowledge gained from other events and venues contribute to the definition of new concepts and experimentation objectives, and result in coherent experimentation campaigns.

  • While the importance of campaigns was acknowledged, no examples of how the process results in experimentation campaigns were forthcoming.

  • Participation in the WIDTs was still primarily an internal function of NWDC, without much participation from outside stakeholders.

  • Analysis and objectives were still emphasized at the end of the experimentation process, rather than at the beginning.

  • Cross-concept analysis was further fractured, as there was not a clear relationship between learning from one WIDT to another, or in the development of experiments.

The assessment of experimentation methods leads to another finding for the Navy.

Finding for Navy: There are significant deficiencies in the end-to-end processes of naval experimentation. Shortfalls include the following:

  • Insufficient use of structured experimentation campaigns;

  • Inadequate breadth in exploration of concepts;

  • Insufficient studies and analysis, including their use to determine the best experimentation venue (games; simulations; small, focused experiments; and so on);

  • Inadequate use of spiral processes to build knowledge iteratively; and

  • Inadequate planning and evaluation.

The Influence of Methods on Resources

The committee acknowledges the value of FBEs, both for supporting various objectives and for gaining the enthusiasm and support of influential senior leaders (such as fleet commanders) by making tangible what had previously been abstract and hypothetical. The history of military innovation is replete with

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

examples of analogous demonstrations.24 The big experiments also reflect long-standing traditions of “just trying things.” However inefficient scientifically, such experiments bring warfighters into the process of change, rather than merely allowing work to be done in laboratories and other R&D venues in the hope that it will prove interesting to warfighters eventually. And, finally, there have been quite a number of significant results from fleet battle experiments.25

Nonetheless, the focus on big fleet experiments comes with a significant price tag, typically in the few millions of dollars, though costs were much higher for Navy participation in the recently conducted Millennium Challenge ’02 (MC 02) joint experiment. There are also substantial opportunity costs in terms of the time of required key individuals and critical fleet assets, not just funds. According to some participants the prevailing incentive structure works against systematic effort because the expectation is that large experiments must not fail in the achievement of their objectives, however much it is argued otherwise. Thus, these experiments are often constrained to prevent the range of exploration needed. Even sheer size contributes its own diminution of free play, as noted in Chapter 4. Depending on how the program of joint experiments managed by USJFCOM develops, the problem could worsen, since the process of staffing and otherwise preparing for a major joint exercise/experiment such as MC 02 is all-consuming.

In recent years, FBEs have become progressively more complex and have incorporated more tests and experiments within a limited period of platform and asset availability. As a consequence, it can be argued that the experiments undertaken during FBEs are inherently less complete than those undertaken during LOEs. For example, the operational parameters and attributes of the HSV could not possibly have been completely explored during the few-week period associated with an FBE. An LOE dedicated to an examination of the attributes of the HSV was used in addition to its participation in a FBE. As a result, the Navy has a much broader understanding of the HSV than it would have gained in the course of an FBE. The committee’s conclusion is that the overemphasis on FBEs is not always the best use of the limited resources available to experimentation.

Another by-product of the emphasis on FBEs is the compromise or modification of objectives that results from combining experiments with exercises and maintenance schedules. These activities have, by nature, inherently different pur-

24  

See, for example, Andrew F. Krepenivich, 2002, Lighting the Path Ahead: Field Exercises and Transformation, Center for Strategic and Budgetary Assessment, Washington, D.C.; and Richard O. Hundley, 1999, Past Revolutions, Future Transformations: What Can the History of Revolutions in Military Affairs Tell Us About Transforming the U.S. Military?, RAND, Santa Monica, Calif.

25  

In Chapter 3, see the section entitled “Synopsis of Results to Date from Fleet Battle Experiments Alpha Through India.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

poses and contexts. Exercises, for instance, address readiness and are associated with current doctrine, organizations, and immediate needs. Experimentation deals with evolving DOTMLPF. Owing to the need to align FBEs with the availability of assets such as ship platforms and aerial assets, many experimentation objectives must be substantially modified or jettisoned. Those that remain are examined in a field environment that is not always suited for proper data compilation methods or that does not always readily factor in anomalies and artificialities introduced during the conduct of the experiment. While good experimentation design should temper such effects, these circumstances do often limit the validity of the results—underscoring the concern that the FBE is not always the most effective use of limited resources.

Funding for an FBE is provided by the NWDC, which also must use its coordinating skills to leverage participation, since it does not have the line-item funding to control equipment used for experimentation. Other organizations, such as the Office of Naval Research (ONR), provide support for the NWDC’s planning of FBEs as well as sponsoring individual projects and concepts that are evaluated in an FBE. ONR has nominally intended to provide from $20 million to $40 million per year for experimentation, but in the past, funds have been used for other higher-priority efforts. Such diversion of funds has occurred with other sponsors as well. These perturbations can have severe impacts on the planning, preparation, conduct, and evaluation of FBEs, which typically have long cycle times of 12 to 18 months. This situation is conducive to a poor return on investment, resulting in delays and/or a loss of substantial effort, with high potential for the compromise of objectives.

While the committee deliberated about making recommendations for increased funding for experimentation, changes in already-programmed funding were sufficiently frequent and severe as to make the need for and scope of such recommendations unclear. What is clear is that the Navy is not making effective use of the experimentation resources already programmed. Also clear is a need for the Navy to establish a line item for funding experimentation that reflects the value placed on experimentation, stands on its own merits, and is supported and sustained Navy-wide. As a notable aside, the submarine community’s development squadron (DEVRON 12) organization, well regarded for its experimentation successes, controls its own assets and budget for experimentation, in contrast to the situation at the NWDC.

These observations result in an additional finding.

Finding for Navy: The Navy has not made effective use of resources in its experimentation program.

A rebalancing of resource investment requires an increased emphasis on analyses, studies, games, models and simulations, and limited-objective experi-

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

ments of various types, leading up to field experiments whose scale is carefully and analytically moderated.

Although the Marine Corps emphasizes field experiments of various types (in large part because it is difficult to evaluate infantry concepts in other ways), the concepts are subjected early to a series of formal and informal war games, tabletop discussions by subject-matter experts, seminars, symposiums, and, if possible, modeling and simulation (M&S) prior to field experimentation. Some limited experiments lend themselves well to M&S. The Marine Corps projects Einstein and Isaac26 are efforts that depend heavily on M&S; the results have been folded into major experiments, notably Urban Warrior.

The situation in the Marine Corps has not always been so balanced. Only a few years ago, the Corps was overly reliant on large experiments—even for the purpose of learning lessons that could have been obtained with smaller experiments. Also, as discussed in Chapter 3, early work in the Sea Dragon series was not always well planned, and preparation for experiments was not always adequate. Consequently, some equipment problems occurred that could have been anticipated, mitigated, or avoided. However, the Marine Corps learned and altered its approach to experimentation.

One change that it has made is more systematic preparation for field experiments (e.g., scheduling time to train personnel on new equipment and in the new concepts and procedures before experimentation begins). Another has been and continues to be a growing reliance on smaller experiments (including limited technical assessments, or LTAs). The committee found these changes and trends on the part of the Marine Corps encouraging.

ASSESSMENT OF ENVIRONMENT, INFRASTRUCTURE, AND TOOLS FOR EXPERIMENTATION

Deriving the maximum benefit from experimentation requires a suitable environment, as discussed in Chapter 2.27 One of the specific questions from the terms of reference is how adequate the tools and environments for experimentation are (e.g., the Navy’s modeling and simulation capabilities, integration facilities, and so on). This section answers this question and examines the naval environment, infrastructure, and key tools for experimentation. The results of the assessment are summarized in Table 5.2 and the detailed discussions leading to these conclusions are presented in the subsections that follow.

26  

See the Web site of Andrew Ilachinski, Center for Naval Analyses, at <www.cna.org/isaac>. Accessed October 9, 2003.

27  

See the section entitled “Environment for Experimentation.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

TABLE 5.2 Assessment of Navy and Marine Corps Environment and Infrastructure

Enabler

U.S. Navy

U.S. Marine Corps

Leadership and culture

  • Committed leaders

  • Learning culture

  • Incentives for risk

  • Tolerance of negative results

  • Empowerment

Strong culture of innovation and adaptation to challenge. Top-down leadership is needed to provide minimal coherence for an experimentation program. Innovators are not visibly rewarded in the Navy Warfare Development Command (NWDC) experimentation programs. Concept developers are at the end of their careers.

Strong culture of innovation and adaptation to challenge. Risk taking and horizon of inquiry have varied. May need to spin off and protect a subset of more radical experiments.

Trained and talented people

  • Concept developers

  • Systems analysts

  • Operators

  • Red-team cells

  • Support teams

Shortages in higher-level skills, especially systems analysts, systems engineers, and architects. Potential decommissioning of Third Fleet’s USS Coronado will reduce needed experimentation expertise. Naval Postgraduate School experimentation expertise may be disbanded.

Shortages in higher-level skills, especially systems analysts, systems engineers, and architects.

Information and physical infrastructures

  • Networks

  • Information repositories

  • Architectural frameworks

  • Integration and test facilities

  • Training facilities

  • Places and platforms

Great opportunities ahead for fleet-centered embedded and distributed exercise/experimental command and control (including reachback); NWDC and Marine Corps Combat Development Command need to track actual progress. Next-generation mission scenarios are needed.

Great opportunities ahead; need to track actual progress in improving network-centric environment in the fleet. Next-generation mission scenarios are needed.

Infrastructure good and improving, e.g., new command and control facility at Camp Pendleton, California (Marine Corps Tactical Systems Support Activity).

Necessary ships and airborne command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) assets are often unavailable. USS Coronado is an exceedingly valuable asset but may be decommissioned. Distributed engineering plant may be useful for spiral processing in experimentation.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

Enabler

U.S. Navy

U.S. Marine Corps

Tools

  • Modeling and simulation

  • Prototypes, surrogates, and so on

  • Artificial environments

  • Data capture and dissemination

Simulations have supported major field experiments but are not adequate for representing future warfighting concepts. Focus has been on simulations to support live forces. Significant benefit in working with the U.S. Joint Forces Command (USJFCOM) to develop future simulation environments. Greater emphasis on supporting the end-to-end range of experimentation activities through modeling and gaming capabilities and virtual simulation environments is necessary. Data collection and analysis tools are also needed.

Simulations have supported major field experiments but are not adequate for representing future warfighting concepts. Significant benefit to working with USJFCOM to develop future simulation environments. Data collection and analysis tools are also needed.

Leadership and Culture

The committee concludes that both the Navy and the Marine Corps have strong, dynamic cultures that believe in adaptation and innovation. Furthermore, both Services have used experimentation as an enabler of change for decades.

Navy Leadership

The current Chief of Naval Operations (CNO) is relying on experimentation through Sea Trial initiatives to enable his Capstone Concept Sea Power 21. The connection that he makes between the use of experimentation and the development of such advanced concepts as FORCEnet raises the bar on his expectations for experimentation and thereby promotes greater initiative for innovation. The CNO also realigned the NWDC under the Fleet Forces Command so that it would be tied more closely to the fleet, reversing his predecessor’s decision on the matter. These actions illustrate the CNO’s interest in experimentation, his belief in its relevance to the fleet, and his continuing support for it, including the use of fleet battle experiments. However, the committee has substantial concerns about the lack of an alignment of the experimentation organization (NWDC) and the experimentation program (Sea Trial) directly under the CNO. One lesson learned from past success in experimentation suggests the need for this strong a connec-

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

tion to the most senior leadership. This is particularly pertinent when experimentation is intended to result in significant new capabilities for the field.28

Certainly the realignment of the NWDC enhances the authority of its commander, enabling him to garner more support and provide increased coherence for the experimentation program, though the committee acknowledges that this move under the Fleet Forces Command is relatively recent. In working with the many Navy organizations participating in experimentation, the NWDC needs to use its coordinating skills and abilities to direct and leverage all of the participation it can obtain from organizations and parties that it does not control. The NWDC does not have line-item funding to control equipment to be used for experimentation (for example, it relies on ONR), nor can it require the N7 staff to act on the results of experiments when dealing with Navy “requirements.”

The lack of the direct connection from the NWDC to the CNO, together with another factor, compounds the committee’s concerns: that is, a potential by-product of the realignment is an overemphasis on the near-term needs of the fleet. The likelihood of this was viewed as high by members of the committee because there will be continuing tensions between experimentation and readiness-related activities. These concerns result in the following finding:

Finding for Navy: Continuing Chief of Naval Operations and Vice Chief of Naval Operations sponsorship, leadership, and attention are vital for the overall coordination, direction, prioritization, and execution of the naval experimentation program.

Navy Culture

The Navy’s incentive structure is not uniformly good within the experimentation community.29 In particular, opportunities for promotion from or extended tours in the NWDC do not appear to be realized. Indeed, many officers there appear to be in their last tour. Further, it is not evident how much influence and prestige the position of NWDC’s commander is intended to have. It is hoped that the realignment under the Commander, Fleet Forces Command (CFFC), and the promulgation of Sea Trial will increase these aspects of the position. A contrast is provided by the submarine community with the well-regarded, dedicated submarine development squadron (DEVRON 12), which enjoys high prestige as it

28  

As evidenced by examples discussed earlier in this report—naval aviation, ARCI, Army digitization, and the Air Force Command and Control Center.

29  

Other innovators in the Navy are rewarded. For example, members of the CNO’s Strategic Studies Group at Newport, Rhode Island, are regarded as prime candidates for promotion and future leadership positions. They also work on concept development but may or may not be regarded as part of the experimentation community.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

pursues continuous experimentation and improvement. The new commander of the U.S. Joint Forces Command is one alumnus of that activity.

Marine Corps Leadership

The Commandant of the Marine Corps has also taken measures to improve the effectiveness of experimentation. He has created the MROC, which, in addition to its other responsibilities, provides early and continuing attention to experimentation by the most senior commanders. This also makes experimentation more institutional and less dependent on the Commandant. By implementing process changes in a series of MROC meetings, the Assistant Commandant has also provided strong guidance and direction to wed the Service experimentation to that of the joint community.

Marine Corps Culture

One questionable change within the Marine Corps in recent years has been increased emphasis on near-term and mid-term objectives in experimentation at the cost of long-term objectives, noted in a prior finding. This change reduces the prospects for risk taking in experimentation, whereas more risk taking is needed to achieve objectives that are transformational. Longer-term change traditionally depends on pursuing highly disruptive technologies and concepts, and the people advocating such departures, even through experimentation, are themselves often viewed as disruptive. There is a need to encourage and reward those who advocate high-risk possibilities, while protecting and balancing innovative but high-risk experimentation.

Despite the need just noted, incentive structures appear to be better for individuals associated with experimentation in the Marine Corps than in the Navy. This may be due to the tight-loop connection with the Commandant and other senior commanders.

Trained and Talented People

Both Naval Services have an abundance of high-quality personnel. The Navy lacks a cadre of experimentation experts for in-depth studies connected with experimentation (most such experts could be civilians). Systematic and rigorous experimentation is a complex and technical undertaking that requires dedicated expertise.30 Both the NWDC and the Third Fleet have dedicated personnel for

30  

This can be seen from perusing illustrative reports on experimentation. See, for example, Joint Advanced Warfighting Program of the Institute for Defense Analyses, 2000, The Joint Experiment J99OJ: Attack Operations Against Critical Mobile Targets, Institute for Defense Analyses, Arlington, Va. (limited distribution). The committee also saw good examples of this in discussions with

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

concept formulation, some experiment planning, and experiments themselves. Nonetheless, experimentation programs of the Navy and the Marine Corps have shortages of personnel with certain high-level skills, notably, systems analysts, systems engineers, and systems architects.

A cadre for in-depth work in experimental design had been constructed at the U.S. Naval Postgraduate School for Fleet Battle Experiment-Juliet, but it may be disbanded in the near future—apparently because its activities were not seen as fitting in to the school’s mission. Also, at the time of this writing there was a movement by the Navy to decommission the USS Coronado, which would impact the Third Fleet’s personnel support.31 One option that has been mentioned is reducing the Coronado’s status to that of a reserve ship, which could reduce yearly costs by perhaps $10 million to $20 million since it would no longer be required to have the same level of readiness as that of an active-duty command ship. That might be a reasonable alternative, but the committee was concerned that such a change would reduce the prestige and career-enhancing aspects of being assigned to experimentation—as well as removing a useful platform.

Recent activities are also having an effect on the numbers of Marine personnel engaged in experimentation. The Service is experiencing increased competition for its personnel assets, primarily from the demands engendered by large-scale joint experimentation and also by the level of participation required by the new joint processes. The Navy is also affected, though less so owing to its size. The press on the Marine Corps staff has been especially severe, since its resources are small and the priorities of the joint process have at times been duplicative and/ or conflicting and therefore inordinately demanding, although the latter difficulty may be being mitigated.32 The committee views these impacts as negative, since they result in fewer resources available for other activities so important to good experimentation, such as analyses. The committee also notes this as another example of the strain that can be put on available personnel by large experimentation events.

Information and Physical Infrastructure

The committee believes by and large that the quality of the information infrastructure is improving, both for naval experimentation and in the forces

   

Dr. Shelly Gallup of the Naval Postgraduate School. See also David S. Alberts, Richard E. Hayes, John E. Kirzl, Leedom K. Dennis, and Daniel T. Maxwell, 2002, Code of Best Practice for Experimentation, DOD Command and Control Research Program, Office of the Assistant Secretary of Defense (Networks and Information Integration), Washington, D.C., July, Ch. 6. Available online at <http://www.dodccrp.org/>. Accessed October 7, 2003.

31  

The USS Coronado is slated for decommissioning in FY 2005.

32  

In Chapter 4, see the subsection entitled “The Role of Joint Experimentation in Preparing for Joint Operations.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

themselves. On the basis of recent experiments and infrastructure developments at the Navy’s Distributed Engineering Plant and the Marine Corps Tactical Systems Support Activity (MCTSSA), discussed below, the networking environment appears adequate for purposes of conducting the experiments. Significant shortcomings exist in the network for supporting operational missions (and hence operational missions as examined in experiments), but that is a different matter from the experimentation infrastructure. Nonetheless, the adequacy of the experimentation information infrastructure should be reassessed as network-centric experimentation moves forward and as key concepts under Sea Trial are explored. Experimentation with network-centric operations may create tremendous but as yet undefined challenges that will have to be accommodated through additional infrastructure and tools support.

The Navy has invested successfully in the Distributed Engineering Plant (DEP)33 for the resolution of critical fleet interoperability issues. These are issues spawned by the growth in tactical networking capabilities such as Link-11, Link-16, and the cooperative engagement capability (CEC). While the main mission of the DEP has been the testing and certification of battle groups before they deploy, it is providing support in system design and development, prototyping for the CEC, and validation of operational processes and TTPs. Consequently, it may be a potential source of infrastructure for experimentation, particularly in spiral development. Its adequacy to support tool interoperability and integration should be assessed as future experimentation campaigns are developed.

Navy

Currently the Navy lacks some key platforms for experimentation. The submarine community usually has adequate access to required platforms for experimentation because it alone controls those platforms. However, there are some problems related to the availability (and budget) for experimentation with ships and C4ISR assets and with manned and unmanned aerial systems such as the U-2 and Global Hawk that are owned by the other Services. The Third Fleet’s USS Coronado is valuable because it does provide a dedicated command-ship platform, but, as noted earlier, this asset is currently scheduled for decommissioning in the interest of cost cutting.

No clear solution is apparent for having access as needed to aerial and non-Naval Service C4ISR assets, but their importance is growing and the experimen-

33  

The DEP is a founded on shore-based sites that replicate the hardware, software, connectivity, and environment of the ship and combat systems and interconnect them to replicate a battle group. See Jeffrey H. McConnell, Technical Coordinator, Navy Distributed Engineering Plant, Naval Surface Warfare Center, Dahlgren, 2002, “The Navy Distributed Engineering Plant—Supporting Force Systems Engineering,” viewgraphs available online at <http://www.dtic.mil/ndia/2002systems/mcconnell2a4.pdf>. Accessed October 9, 2003.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

tation program needs access to these systems. The NWDC’s new position under CFFC may improve prospects in this area.

Marine Corps

The Marine Corps appears to have adequate physical infrastructure (especially with the addition of MCTSSA, the new facility at Camp Pendleton for command and control), but with the current and projected growth of C4ISR systems, additional options may eventually be required. Also, the Marine Corps has the same problem as the Navy with respect to access to some C4ISR and aerial assets that are not owned by the Service.

The other Services and the U.S. Joint Forces Command are increasing their investments in linked computer simulations and ranges, as well as in artificial and virtual environments. As standards for interoperability among facilities and ranges are developed, the Marine Corps may need to invest in these areas to offset the projected rise in operations and personnel tempo for people and equipment. These investments would not only permit both training and experimenting in virtual, realistic environments but would also reduce costs associated with equipment transportation, maintenance, and duplication of excess personnel and equipment (hedging against contingencies), as well as personnel transportation.

A relevant example is that of the Army’s long-term investment in systems with which to train personnel for operations similar to that in Bosnia. One effort, at the University of Southern California, is researching how to have a trainee (e.g., a lieutenant) interact with a virtual-reality “world.” This world has relatively realistic, animated, artificial-intelligence beings representing, for example, villagers and soldiers at an intersection where a traffic accident involving U.S. forces has occurred. The trainee must assess the situation and interact with the “people” (who respond to the lieutenant’s voice, show emotion, ask questions, and take actions).34

Tools

Tools for Analytical Work

It was clear from presentations and documentation provided to the committee, as well as from discussions with presenters, that the Navy’s analytical work

34  

See William Swartout, 2002, “Creating Human-Oriented Simulation: The Challenge of the Holodeck,” presented at the Grand Challenges for Modeling and Simulation Seminar at Schloss Dagstuhl International Conference and Research Center for Computer Science, Wadern, Germany, August 26-30. Available online at <http://www.informatik.uni-rostock.de/~lin/GC/report/Swartout.html>. Accessed October 9, 2003.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

underlying the experimentation program is currently unbalanced in favor of relatively detailed models and simulations tied to the large experiments. Its analytical work is not well suited to broad and rigorous capabilities-based planning,35 as distinct from working through particular scenarios in more detail. Consequently, there is a shortfall in families of models and games.

Figure 5.1 elaborates on why families of models and games approaches are needed by showing that different members have different strengths. Low-resolution models, if designed for the purpose, can be excellent for exploratory analysis and design—covering a breadth of cases, but not in much detail. In contrast, entity-level simulation can provide a much richer depiction of some underlying cause-effect relationships, i.e., of the underlying phenomena. However, such detailed simulations are not appropriate for broad, design-level exploration,36 nor even for doing a good job in FBE-style experimentation.37 Nor are they currently detailed and accurate enough to represent some important issues (e.g., performance of U-2-based C4ISR as a function of operational circumstances, pilots, weather, and so on), which need to be observed with real platforms.38 War games can bring in human warfighters, who are essential in some play and represent potential users. Field tests with live forces, supplemented by simulations of actual battle, can provide the ultimate experimentation—short of war itself—but are inherently limited in many respects. And, again, they are not very good at supporting understanding of the breadth of possibilities for drawing generalizations, both of which require more abstracted modeling and analysis.

Improving the balance of analytical work underlying the experimentation program would require a realignment of effort within some of the analytical

35  

Capabilities-based planning has been mandated since the 2001 Quadrennial Defense Review. For high-level technical discussion, see Paul K. Davis, 2002, Analytic Architecture for Capabilities-Based Planning, Mission-System Analysis, and Transformation, RAND, Santa Monica, Calif.

36  

Important, less-abstracted exploration can be accomplished with more detailed human-machine simulations and could be accomplished by real-world operators in the fleet if the tools for doing so were adequately embedded in operational systems, permitting continuous experimentation along with training. This type of exploration is especially suitable for exploiting concepts and technology in near-term improvements. Higher-level exploration—for example, across scenarios and case spaces—is different in character.

37  

For an interesting discussion by one of the developers of the well-respected Naval Simulation System, see William Stevens, 2000, “Use of Modeling and Simulation (M&S) in Support of the Assessment of Information Technology (IT) and Network Centric Warfare (NCW) Systems and Concepts,” 5th International Symposium on Command and Control Research and Technology (ICCRT), held at Australia War Memorial, Canberra, Australia, October 24-26, and sponsored by DOD Command and Control Research Program (CCRP), Office of the Assistant Secretary of Defense (Network and Information Integration), Washington, D.C., and Australian Department of Defence, Defence Science and Technology Organisation.

38  

This important point contradicts the notion that simulations can do nearly everything. It was stressed in discussions with Shelly Gallup of the U.S. Naval Postgraduate School, on the basis of extensive experience in fleet battle experiments.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

FIGURE 5.1 Comparative strengths within a family of models and games.

groups supporting experimentation, or the creation of new groups. For example, the NWDC’s modeling and simulation group has made enormous strides in recent years in establishing the capability to simulate many aspects of operations sufficiently well so that live and simulated play can be intermixed and so that live players can be stimulated by and can interact with simulations in much the same way as with real-world command and control systems.39 However, these efforts are extraordinarily demanding, in terms of both energy and resources. As a result, the group acknowledges that it does very little of the lower-resolution analysis associated with exploration or systems analysis. One way or another, the Navy needs to supplement these capabilities with other skills (some of which exist elsewhere within the Navy community).

The large and complex simulation work is essential if experimentation is to connect and resonate well with fleet operations and if transitions are to occur easily; it is also essential in many instances in which it provides analytical insights not achievable in other ways (at least currently). Elsewhere this chapter discusses the need to emphasize analytical work and experimentation activities that offer good and correct alternatives to large-scale events. The issue is one of balance.

39  

Discussions with Guy Purser of the NWDC and Annette Ratzenberger of USJFCOM.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Tools for Simulation

Properly applied, modeling and simulation can both accelerate the pace of experimentation and reduce its cost. Consequently, it is important to both Service-unique and joint experimentation. There is a need and an opportunity to develop naval use of virtual environments (no interaction with field forces) and constructive environments (some field involvement) for experimentation purposes. As the Navy works through the concepts required for Sea Power 21, virtual and constructive simulations can be extremely useful for developing, exploring, and testing the concepts before trying them out with actual forces.

In assessing the need to expand naval simulation infrastructure for Sea Trial as well as that for future joint experimentation,40 specific shortcomings in the naval simulation environment emerge. Most simulations exercised by the Navy in the past have involved some forces and have been structured within the context of large field experiments. However, there is a need now for a more robust set of M&S capabilities to align with the many activities involved in the Sea Trial and joint experimentation campaigns, beginning with exploratory concept development.

After MC 02, USJFCOM noted that the DOD’s existing M&S capabilities were inadequate to represent future operational concepts. These capabilities do not account for information operations, model new organizations, or capture asymmetric warfare strategies. Also observed after MC 02 was that data analysis tools were too limited and were inadequate for dealing with the large amounts of data collected in MC 02.

In response to these shortfalls in the joint infrastructure, USJFCOM will be addressing an entire range of enhancements through its continuous experimentation environment to enable more flexible, higher-fidelity M&S and through the Joint Simulation System training tool to provide realistic, large-scale, simulations placing humans in operational environments. The development and use of such a common simulation environment are necessary as well as desirable, although the Navy and the Marine Corps will also need their own naval-specific simulations. Such an interoperable environment brings twofold benefits: (1) a consistent environment for both joint and naval-only play and (2) the cost economies of developing one environment to serve two (or more) communities.

The key observations of the committee’s assessment of naval simulation environments are based on future needs for Service-unique and joint experimentation. They are as follows:

  • While the current modeling and simulation environments have successfully supported major field experiments, there are deficiencies in M&S capabilities

40  

In Chapter 4, see the subsections entitled “Joint Forces Command’s Emerging Modeling and Simulation Infrastructure” and “Naval and Joint Linkages in Simulation.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

to support a full range of experimentation activities, such as concept development, games, and small experiments; in the areas of compatibility across joint and Service simulations; and in the representation of future warfighting environments.

  • The greater use of simulation relative to live forces would allow more experimentation in the future, especially for concepts that apply above the tactical level. In particular, the Navy should make greater use of virtual and constructive simulations for developing, exploring, and testing concepts of operation before trying them out with actual forces.

  • Exploring tactical interactions in simulation environments is necessary, given the increased importance of tactical cross-Service interoperation in current and future warfighting concepts. Both the Navy and the Marine Corps will need to participate in efforts to realize such simulation environments.

  • Significant development of tools for building, validating, and verifying models; for generating scenarios; for populating databases; and for collecting and analyzing data is necessary. Furthermore such tools have to function, interoperate, and integrate into various environments and frameworks, as future experimentation campaigns are defined and executed.

These observations and the earlier discussions on platform availability and needed modeling capabilities lead to two findings.

Finding for Navy: The infrastructure and tools required for the experimentation campaigns of the future, including those for Sea Trial and joint experimentation, are inadequate. Primary shortfalls include the following: limited availability of ship platforms (compounded by the potential decommissioning of the USS Coronado) and airborne command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) assets; lack of modeling and simulation capabilities that support a full set of experimentation campaign activities, explore tactical-level interactions, and reflect next-generation warfighting environments; and lack of tools for building, validating, and verifying models; for generating scenarios and populating databases; and for collecting and analyzing data.

Finding for Marine Corps: The infrastructure and tools required for the experimentation campaigns of the future are inadequate. Primary deficiencies include the following: lack of modeling and simulation (M&S) capabilities that support a full set of experimentation campaign activities, explore tactical-level interactions, and reflect next-generation warfighting environments; and lack of tools for building, validating, and verifying models; for generating scenarios and populating databases; and for collecting and analyzing data.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

EXPERIMENTATION FOR BUILDING NAVAL FORCES FOR JOINT OPERATIONS

All recent U.S. military operations have been joint. Consequently the terms of reference for the study directed an examination of the role of experimentation in building future naval forces to operate in the joint environment. This section assesses experimentation relative to its influence on joint operations and with respect to preparedness for joint operations. Various experimentation venues are included—USJFCOM-sponsored experimentation activities, the Combatant Commands and cross-Service activities, as well as Naval Service activities.

Building Naval Forces to Support Joint Operations

Ultimately, the whole point of joint experimentation is better joint operations. Experimentation serves to help develop and refine concepts that lead to capabilities, which in turn are reflected in operations. Today, in support of joint operations, each Service brings its own core warfighting capabilities to the fight. Therefore, joint experimentation and Service experimentation both serve to build capabilities for joint operations. As indicated in Chapter 4, joint experimentation performs three primary functions in improving joint operations:

  • Developing of DOTMLPF for the Services and the joint community,

  • Improving the understanding of other Services’ capabilities, and

  • Contributing to the less tangible but important aspect of socialization.

To note progress first: The committee was consistently impressed by the major changes in thinking that have occurred among naval personnel within the past few years. With respect to understanding other Services’ capabilities and to socialization, the Navy and Marine Corps officers now appear to have internalized the fact that they will almost always be fighting in joint contexts, often with much more integrated operations than would have been conceived of only a few years ago. This attitude is an important development in the evolution of U.S. military forces.

Experimentation is being planned more in terms of a joint context and with positive interest and much effort. The Navy and the Marine Corps realize that they may conduct key, theater-opening activities, with more massive Army and Air Force deployments following as quickly as possible. They also recognize the need to adapt to further shifts in traditional roles, as happened in Afghanistan when the Navy was present throughout the conflict and the Marines entered after the Army Special Operations Forces.

Although the joint experiments are in some respects a burden for the individual Services, they are also opportunities and, in any case, unavoidable realities. As a result, both the Navy and the Marine Corps are active participants and are

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

increasingly building their own experiment programs around joint scenarios and working to align them with the schedule of joint exercises/experiments.41 These efforts have led to increased collaboration.

There is also progress with respect to joint concept development,42 although the migration of new concepts into actual operations is not so easy to trace, perhaps because it is too soon. However, the committee heard specific anecdotal examples of successes applied in Operation Enduring Freedom (OEF) that stemmed from activities connected with preparation for the USJFCOM MC 02 experiment; it heard of other successes in OEF stemming from the Hunter Warrior Advanced Warfighting Experiment. Lessons learned and equipment from experiments (specifically, the Marine Corps fast attack vehicles) were successfully used during combat in OEF.43 There is also evidence that operational experience is helping to guide experimentation planning by USJFCOM.

Joint experimentation is still evolving and growing in importance. As of this writing, little development of DOTMLPF had occurred from USJFCOM-sponsored experimentation. Nonetheless, a few USJFCOM recommendations were close to having moved through the JROC approval process (e.g., the Standing Joint Force Headquarters and the Joint Enroute Mission Planning Rehearsal System—Near Term).

Collaboration in Concept Development

Although the Naval Services and the joint community are working toward joint concepts, the committee believes that interactions are not as collaborative as they need to be. As new warfighting concepts are developed, coordination among the Services and the joint community must ensure that individual concepts are compatible and supporting. This kind of coordination sets the stage for experimentation campaigns that develop capabilities and prepare forces for joint operations.

Concept development is focused for the Navy and the Marine Corps, respectively, in the Navy Warfare Development Command and the Marine Corps Combat Development Command. On the joint side, it occurs at the U.S. Joint Forces Command and, more recently, as part of the Joint Warfighting Capabilities Assessment (JWCA) process on the Joint Staff.

41  

See Table 3.1 in Chapter 3 and Tables 4.3 and 4.5 in Chapter 4. For instance, FBE-A included Air Force assets, as did FBE-D which added those of the Army. FBE-I was done in conjunction with Kernel Blitz, which had the participation of all four Services and the Marine Corps Capable Warrior experiment. The Navy had 10 experiment objectives incorporated in USJFCOM MC 02, including FBE-J, and the Marine Corps had 3, including Millennium Dragon ’02.

42  

See Table 4.2 in Chapter 4.

43  

In Chapter 4, see the subsection entitled “The Role of Joint Experimentation in Preparing for Joint Operations.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

While the NWDC and the MCCDC interact in concept development with USJFCOM and (along with other elements of the Navy) with the JWCA process, the committee believes that this interaction is not close enough. The committee’s observations in this regard are as follows:

  • The naval concepts being developed are generally couched in a joint context, but the extent of specific interaction with the joint community or other Services in developing and refining these concepts did not appear to be significant.

  • The development of joint concepts by USJFCOM and the JWCA process does involve Naval Service representation, but even so, joint concepts seem to be created from scratch. They do not appear in a substantive way to build on the naval concepts that are being developed by the NWDC and the MCCDC. Likewise, there does not appear to be a detailed correspondence between the joint and naval-developed concepts.

  • The joint community itself does not speak with a single voice, in that the various processes are developing different concepts. However, there is reason to anticipate that such differences will be resolved.44

Experimentation Led by the U.S. Joint Forces Command

USJFCOM has conducted a number of joint experiments of different types in which the Navy and Marine Corps have participated. These have included war games, constructive simulations, human-in-the-loop simulations, limited-objective experiments, and large-scale field experiments.45 In addition, USJFCOM is planning a significant program of future experimentation across all types. This program will be characterized by two paths: (1) prototype refinement and validation, which have a nearer-term focus, and (2) concept development, which has a longer-term perspective.46 The Navy and the Marine Corps are, of course, expected to be active participants in these future experiments.

To date, most discussion of USJFCOM-led experimentation has centered on the congressionally mandated field experiment Millennium Challenge ’02, which was much larger than any prior joint experiment. The Navy and the Marine Corps each conducted its own major experiments—Fleet Battle Experiment-Juliet and Millennium Dragon ’02, respectively—within the overall experiment. On the

44  

As this study was being completed, the Joint Staff was drafting the Joint Capstone Concept and the Joint Capabilities Integration and Development System document (CJCSI 3170.01C). Completion of the Joint Capstone Concept and its adoption by USJFCOM and realization of the processes specified in CJCSI 3170.01C could do away with the differences noted here.

45  

See Table 4.1 in Chapter 4.

46  

In Chapter 4, see the subsections entitled “Future Experimentation of the Joint Forces Command” and “Emerging Strategy of the Joint Forces Command for Experimentation Campaigns.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

basis of briefings and other information provided by the Navy, Marine Corps, and USJFCOM, the committee makes the following general observations on Millennium Challenge ’02:

  • The value of such a large experiment relative to its cost47 was questioned by a number of briefers and by many committee members. Some felt that funding could be put to better use in a series of smaller experiments both of the joint and the Service-specific types. Others noted, however, that such an event made the assets of other Services available, something that is difficult to achieve in the context of smaller experiments because of all the demands on operational assets. There was full agreement that it is extremely important to select the venue that matches the objectives desired. A larger-scale field experiment may be warranted to test integration, scalability, and a complex set of interactions, but the question remains—How large a scale is necessary?

  • Such large, high-visibility experiments were characterized by many as demonstrations rather than as experiments, with little room for true exploration or failure. If that is the case, these large events may provide some opportunities to showcase unfunded but “ready-for-prime-time” equipment capabilities in hopes of garnering support and supplemental funding. Others noted, however, that a series of smaller events conducted over the 2 years leading up to Millennium Challenge ’02 had allowed for greater exploration and assessment.

  • Since the conduct of Millennium Challenge ’02 in July and August 2002, the general sense in the community appears to be that further experiments of this size are unlikely. Unofficial statements from USJFCOM indicate that it will not conduct any more major joint experiments like Millennium Challenge ’02 for the foreseeable future, but will focus its efforts instead on smaller, more frequent events.48 The Navy and the Marine Corps should be active participants with USJFCOM in the planning now going on to define these future experiments, both for near-term prototype development and longer-term concept development.

Experimentation in the Combatant Commands

Joint experimentation also takes place in the Combatant Commands.49 The issues involved in joint force operations are so important and large in scope that it is only natural that all Combatant Commands are involved in addressing them

47  

The committee repeatedly heard that the total cost of MC 02 was approximately $250 million, including the costs of the Services and of USJFCOM, but it has not been able to verify this number.

48  

Statement attributed to ADM Edmund Giambastiani, Commander of USJFCOM; see Anne Plummer, 2003, “Chief Says More Risks to Be Taken: USJFCOM Says No More Large-Scale Events Like Millennium Challenge,” Inside the Pentagon, March 27, p. 3.

49  

In Chapter 4, see the section entitled “Experimentation in the Combatant Commands.”

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

through experimentation. This has typically been done in combination with command post and field exercises.50

The Navy and Marine Corps and the Combatant Commands do cooperate in conducting joint experimentation within exercises conducted in the Combatant Commands—Kernel Blitz is one such example. However, there does not appear to be a coordinated program between the Services and Combatant Commands to conduct experimentation in a systematic way.

More collaboration of the Navy and Marine Corps and the Combatant Commands to systematically develop programs of joint experimentation appears highly desirable. Past experiences have been valuable, including many that applied to command, control, and communications; that pertained to developing and refining procedures or prototype systems; or that involved coalition nations. However, as indicated in Chapter 4, a number of activities would have to be carried out to achieve a greater degree of coordination in, and systematic expansion of, joint experimentation.

One possible mechanism for this coordination could be between the Combatant Command and the Service component commands assigned to that Combatant Command (e.g., either the appropriate Navy fleet command or the numbered fleet commands under that fleet command). The component commands could then interact as necessary with other elements of their respective Service (e.g., with the NWDC and the MCCDC).

Cross-Service Experimentation

Not all experimentation involving two or more Services need take place under the joint umbrella. Direct Service interaction can be very valuable. In the committee’s view, experimentation involving two or more Services is best suited for operations at the tactical level, since higher-level (component and Joint Task Force) operations are primarily joint. Useful concepts developed in this manner can be fed into the joint arena. There is a need to investigate joint interactions at a tactical level, given the growing intensity in recent operations, such as that in Afghanistan. While the Navy and Marine Corps have devoted increased attention to cross-Service experimentation, more is needed.

Direct Service interaction can occur in two general ways—between Service centers involved in concept development and between deployed forces engaged in exercises and experiments. The Navy Warfare Development Command and the Marine Corps Combat Development Command have regular interactions with one another. In addition, the Navy Network Warfare Command has noted that its proximity to corresponding Air Force organizations (e.g., Air Combat Command

50  

Table 4.5 in Chapter 4 provides several examples.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

and the Air Force C2ISR Center) offers the opportunity for collaboration. While other, substantive interactions between Service centers may exist, none was apparent to the committee in its investigations. Increased interaction could be very valuable—to provide cross-fertilization in concept development and to begin building joint concepts from the ground up.

The Navy indicated in briefings to the committee that other Services (e.g., the Air Force) are involved in its fleet exercises and that it would like to increase such involvement. Recent Marine Corps experiments, with their highly tactical focus, do not appear to have a significant joint or cross-Service perspective. However, future opportunities would appear to exist—for example, coordinated operations with Army Special Operations Forces and air support from Navy and Air Force aircraft.

One recurrent difficulty with cross-Service experimentation is the need for the simultaneous availability of assets from two Services. Mutual scheduling is difficult. Ideally, the elements of both Services should be on exercises or engaged in other more formal experiments at the same time. This, however, requires cross-Service coordination of force deployment schedules. Such coordination would have to be carried out at the top levels of the Services—possibly with USJFCOM acting as an intermediary. This would be necessary because the deployment schedules have Service-wide impacts relating to such factors as accomplishing operational missions, training, personnel rotation, and platform overhaul and upgrade. Such a coordination mechanism is not in place today.

Balancing Service and Joint Experimentation

Since naval operations will become more joint in coming years, linkages between naval and USJFCOM-sponsored experiments, and indeed the full range of joint experimental campaigns, should be carefully planned. This spectrum of activities runs from the earliest concept development, through analysis, war games, and simulations, and leads ultimately to LOEs and large field experiments. An orderly progression of these activities will greatly aid in the efficient use of Service and joint resources.

In all of these phases, joint experimentation coexists with Service-specific experimentation because proficient Service core capabilities are a prerequisite for joint warfighting. Alternately, joint experimentation and the attendant joint concept development may require that the Services develop new capabilities. Service experimentation programs should progress in an orderly fashion so that they can feed into and contribute to joint experimentation while also deriving benefits from the joint activities—all the better for both Service and joint capabilities.

USJFCOM has laid out a detailed and intense joint experimentation campaign. The Services support and participate in this effort while in turn performing their own Service-unique experimentation activities. These multiple responsibili-

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×

ties stretch the already resource-limited Service-specific experimentation efforts. It has already been noted that the MC 02 event significantly stressed the staff resources of the Marine Corps. If, in the worst case, USJFCOM’s activities interfere with or prevent the normal progression of Service experimentation, they can actually be counterproductive to developing improved joint capabilities. Thus, a balance must be maintained between joint and Service-unique experimentation, with events and objectives synchronized within the respective experimentation campaigns. To date there is no mechanism in place for achieving this balance.

Summary with Findings

In assessing the role of experimentation in building forces for joint operations, the committee noted progress both for the Navy and the Marine Corps. The following limitations and opportunities also exist in the current situation:

  • There is a need for expanded and more synergistic collaboration in joint concept development.

    There are opportunities for expanding joint experimentation through the Combatant Commands, but a coordination mechanism is needed to develop programs of experimentation systematically.

  • There are opportunities and a need for more cross-Service experimentation, including investigating joint interactions at the tactical level (particularly important for the Marine Corps), given recent operations such as that in Afghanistan. However, a mechanism is required to schedule the simultaneous availability of assets for the Services involved.

  • There is a need to balance and synchronize joint and Service-unique experimentation, given the demands placed on the resources available.

These observations lead to the following finding.

Finding for the Naval Services: The naval and joint experimentation programs are not yet adequately aligned and synchronized, nor is there sufficient correlation between them. More synergistic collaboration in joint concept development is needed. Service-unique experimentation has been and could continue to be affected by large-scale joint experimentation, but no formal mechanism exists for striking a proper balance between joint and naval experimentation.

Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 141
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 142
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 143
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 144
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 145
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 146
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 147
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 148
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 149
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 150
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 151
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 152
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 153
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 154
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 155
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 156
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 157
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 158
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 159
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 160
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 161
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 162
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 163
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 164
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 165
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 166
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 167
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 168
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 169
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 170
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 171
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 172
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 173
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 174
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 175
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 176
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 177
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 178
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 179
Suggested Citation:"5 Effectiveness of Experimentation for Future Naval Capabilities." National Research Council. 2004. The Role of Experimentation in Building Future Naval Forces. Washington, DC: The National Academies Press. doi: 10.17226/11125.
×
Page 180
Next: 6 Recommendations for Improving the Overall Effectiveness of Naval Experimentation »
The Role of Experimentation in Building Future Naval Forces Get This Book
×
Buy Paperback | $55.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

The Department of Defense is in the process of transforming the nation’s armed forces to meet the military challenges of the 21st century. Currently, the opportunity exists to carry out experiments at individual and joint service levels to facilitate this transformation. Experimentation, which involves a spectrum of activities including analyses, war games, modeling and simulation, small focused experiments, and large field events among other things, provides the means to enhance naval and joint force development. To assist the Navy in this effort, the Chief of Naval Operations (CNO) asked the National Research Council (NRC) to conduct a study to examine the role of experimentation in building future naval forces to operate in the joint environment. The NRC formed the Committee for the Role of Experimentation in Building Future Naval Forces to perform the study.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!