5
Effectiveness of Experimentation for Future Naval Capabilities

This chapter assesses the effectiveness of experimentation for future naval capabilities. It addresses the specific questions of the terms of reference, beginning with a review of what has been learned from naval experimentation and an assessment of the success of transitioning results from experiments to the fleet and field. The assessment then naturally leads to an examination of the adequacy of naval experimentation programs, processes used, and the environment and infrastructure that support experimentation. The chapter concludes with an examination of the broader relationship of experimentation within the joint context, specifically looking at the effectiveness of naval experimentation in preparing for joint operations and at the relationship between Service-unique and joint experimentation.

ASSESSMENT OF EXPERIMENTATION RESULTS

The terms of reference ask what has been learned from experimentation thus far. From a historical perspective, experimentation with innovative technologies in military systems and with techniques of warfare has often led to revolutionary changes in how military forces are constituted and how they fight. Examples from the previous century include the use of armored forces in land warfare, the use of submarines and sea-based aviation in naval warfare, the application of ballistic missiles to intercontinental attack capability, the use of nuclear submarines to constitute long-enduring and essentially invulnerable undersea tactical and strategic strike forces, and the use of space systems for observation, communication, and navigation.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces 5 Effectiveness of Experimentation for Future Naval Capabilities This chapter assesses the effectiveness of experimentation for future naval capabilities. It addresses the specific questions of the terms of reference, beginning with a review of what has been learned from naval experimentation and an assessment of the success of transitioning results from experiments to the fleet and field. The assessment then naturally leads to an examination of the adequacy of naval experimentation programs, processes used, and the environment and infrastructure that support experimentation. The chapter concludes with an examination of the broader relationship of experimentation within the joint context, specifically looking at the effectiveness of naval experimentation in preparing for joint operations and at the relationship between Service-unique and joint experimentation. ASSESSMENT OF EXPERIMENTATION RESULTS The terms of reference ask what has been learned from experimentation thus far. From a historical perspective, experimentation with innovative technologies in military systems and with techniques of warfare has often led to revolutionary changes in how military forces are constituted and how they fight. Examples from the previous century include the use of armored forces in land warfare, the use of submarines and sea-based aviation in naval warfare, the application of ballistic missiles to intercontinental attack capability, the use of nuclear submarines to constitute long-enduring and essentially invulnerable undersea tactical and strategic strike forces, and the use of space systems for observation, communication, and navigation.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces The Navy and Marine Corps have a long history of using experimentation to evolve significant new capabilities. The Navy experimented with submarines, carriers, and PT boats, and with new propulsion systems and new fuels. Experimentation with launching and recovering aircraft from ships began but a few years after aircraft were invented. Similarly, although Marines had been landing from ships in small boats for many decades, even centuries before World War II, the pressures of warfare led to the rapid development of prototypical modern amphibious landing systems. Experimentation has been key to advances in all forms of naval warfare—gunnery, guided missiles, naval ship propulsion, vertical-capable jet aircraft, very short takeoff and landing rotorcraft, and all other activities related to the shaping and operation of naval forces, including the Fleet Antiterrorism Security Teams and the Chemical/Biological Incident Response Force (CBIRF) of today. In order to assess more recent Service-unique experimentation, the committee focused primarily on the Navy Warfare Development Command (NWDC)-sponsored fleet battle experiments1 (Alpha through India) and on the Marine Corps experimentation efforts beginning with the Hunter Warrior (HW), Urban Warrior (UW), and Capable Warrior (CW) campaigns. Fleet Battle Experiment-Alpha (FBE-A) was conducted in March 1997. The Hunter Warrior series of activities was initiated in 1997. Chapter 3 provides details on these efforts. FBE-A through FBE-I included a total of nearly 40 separate objectives. Each FBE had between three and eight major objectives, and each major objective had anywhere from one to nine subobjectives. A significant number of these major objectives and subobjectives were realized. The range of investigation of these FBEs was quite extensive, addressing network-centric operations for naval and joint fire power, theater and air missile defense, precision engagement, time critical strike, and defense against asymmetric threats, to name a few areas. Table 3.1 in Chapter 3 provides a synopsis of all FBEs, their objectives, and their results. Collectively assessed, these provide evidence that experimentation is achieving meaningful results. The three experimentation campaigns HW, UW, and CW also addressed many objectives. HW had 37 objectives, of which 29 were realized. As a campaign, it focused on individuals and combat patrols operating in desert environments. UW addressed individuals and platoon-size operations in urban environments; CW focused on individuals and company-size operations at Camp Pendleton, California. Each of the campaigns required a cycle of more than 3 years. As with the FBEs, these campaigns covered a considerable range in their investigations, 1   The nominations for and participation in FBEs involve many organizations, as discussed in Chapter 3. For example, these organizations have included the Second, Third, Fourth, Fifth, Sixth, and Seventh Fleets, and regional commanders, who propose experiments through the Navy Component Commanders in their area of responsibility.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces including the use of nonlethal weapons, small unmanned aerial vehicles (UAVs), sensors, intrasquad radios, precision targeting, and the development of a common tactical picture. Each addressed a series of important questions, such as how to extend the effectiveness of a modest forward afloat expeditionary force or how to penetrate and operate in dense urban littorals. In short, many objectives of each campaign were realized and valuable knowledge was derived. The lessons learned from these campaigns were captured in the Marine Corps “X-files.”2 To the terms-of-reference question posed at the beginning of this section, the committee responds that both Navy and Marine Corps experimentation is enabling learning and producing meaningful results directed at promising concepts and technologies in a number of key naval mission areas. However, other questions that remain to be answered in this chapter go to the heart of the matter—which is whether naval experimentation is as effective as it can be and needs to be. ASSESSMENT OF TRANSITIONING A key question from the terms of reference is how successful the transitioning of the results of experimentation to the field has been. The committee responds by focusing on (1) doctrine and tactics, techniques, and procedures (TTPs) and (2) fielded capabilities, including acquisition programs. Summaries are provided in Table 5.1, which synopsizes results from the discussions on naval experimentation in Chapter 3. More detailed observations are presented below. Summary Observations One of the important outcomes of the Navy’s FBE series of experiments has been the determination as to why certain objectives were not achieved, which allows for an iterative process of improvement. The principal successes of FBEs have been as follows: 2   “The X-Files contain useful information packaged for rapid reading and easy transport in the cargo pocket of the utility uniform. They convey a synthesis of knowledge gained from experiments with tactics, techniques, and procedures, and some enabling technologies that can help us fight and win battles. Most of them focus on operation in the urban battlespace. They are an evolving body of knowledge that is constantly refined through experimentation…. information in the X-Files is entered into the Marine Corps Combat Development System. It forms the backbone of recommended revisions to Marine Corps doctrine for Military Operations on Urbanized Terrain (MOUT). Knowledge in the X-Files also underpins much of the Basic Urban Skills Training (BUST) program used by the Operating Forces. The X-Files gather, organize, and synthesize knowledge from post training analysis and feedback from Marines, Sailors and other participants in MCWL experiments. They do not contain official doctrine, nor are they policy or standing operating procedures (SOPs).” For further information see <http://www.mcwl.quantico.usmc.mil/x_files.asp>. Accessed November 18, 2003.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces TABLE 5.1 Transitioning Results and Conclusions of Experimentation into the Navy and Marine Corps Result U.S. Navy U.S. Marine Corps Doctrine and tactics, techniques, and procedures (TTPs) Some concepts have moved into the field Many concepts have moved into the field. Fielded capabilities including acquisition programs Few successes so far. Naval Fires Network is one notable concept that appears to be in transition to the field. Numerous successes in transitioning small end items as well as interim capabilities.   Transitions are very difficult, and processes for achieving them are seen as poor, even from within the Navy, owing in part to budget pressures and to a lack of processes for new capabilities to compete with programs of record. Transitions have proven very difficult for expensive capabilities because of budget pressures and ineffective processes for new capabilities to compete with the programs of record. Demonstrations of the feasibility of new operational concepts using surrogate or prototype or existing systems, The adoption by fleet forces of new TTPs, and The development of new doctrine for fleet operations.3 The Naval Fires Network (NFN) appears to be a case of successful experimentation in the process of transition for the Navy. NFN is a network-centric warfare system that provides real-time intelligence correlation, sensor control, target generation, mission planning, and battle damage assessment capabilities. It allows ships and aircraft in a carrier strike group or an expeditionary strike group to share near-real-time and real-time intelligence and targeting information, not only with one another but also with Army and Air Force units in a joint or coalition task force. NFN was first studied in FBE-A and then refined in FBE-I. It has transitioned to the fleet as an interim prototype on two carrier strike groups, owing to intense interest from the CNO and fleet commanders. However, to date it has not become a formal program of record. Marine Corps successes, based on the Hunter Warrior, Urban Warrior, and Capable Warrior campaigns, are as follows: 3   In Chapter 3, see the section entitled “Synopsis of Results to Date from Fleet Battle Experiments Alpha Through India.”

OCR for page 141
The Role of Experimentation in Building Future Naval Forces Concepts, doctrine, and TTPs resulting from experimentation have transitioned successfully to forces in the field. Experimentation has resulted in changes in minor equipment items in the field.4 It is clear that naval experimentation is resulting in new doctrine, TTPs, some new concepts, and some minor end items. It is also clear that there is a serious shortfall in transitioning the results of experimentation into major fielded capabilities. For the Navy, this extends to minor items of equipment and capabilities as well. The committee’s findings with respect to the success of transitioning results of experimentation to the field are as follows: Finding for Navy: The mechanisms and processes for transitioning the results of experimentation directly to the fleet or to an acquisition program of record are inadequate, and they curtail the effectiveness of experimentation in building future naval forces. Finding for Marine Corps: The Marine Corps has been successful in transitioning nonmaterial elements of doctrine, organization, training, materiel, leadership, personnel, and facilities (DOTMLPF) and minor equipment. However, it has not been successful in transitioning to combat forces major warfighting capabilities identified during experimentation. More detailed discussions related to these findings follow. Transitioning to Naval Doctrine and Tactics, Techniques, and Procedures One objective for experimentation is to explore or assess new doctrinal concepts and, later, to develop and refine new TTPs. The committee noted that the NWDC is sensitive to the need to coevolve doctrine and TTPs at the same time that new technology is introduced. The Marine Corps Combat Development Command (MCCDC) also emphasizes this strategy—partly because in some early experiments problems had arisen as a result of insufficient attention to retraining participants in the use of new capabilities. The lessons of the Gulf War in 1991 and of the conflicts in Bosnia and Kosovo (where sustained operations required superb command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR)), and in Afghanistan (where Special Forces and Marines operated directly together with the use of long-range bombers) were consistent with studies and games conducted over the years. Without the urgency of an actual war, however, it is unclear that change would have happened, 4   In Chapter 3, see the subsection entitled “Synopsis of Results of Sea Dragon to Date.”

OCR for page 141
The Role of Experimentation in Building Future Naval Forces since at a certain stage, rational change driven by persuasive analysis, modeling, simulation, and gaming is not yet supported by empirical evidence. Nonetheless, the “wild ideas” practiced in experiments such as Hunter Warrior, Urban Warrior, and fleet battle experiments were turned into reality when the time came—because individuals exposed to the ideas and potentials were willing to break with tradition and doctrine to do sensible things. That said, some results produced from Navy and Marine Corps experimentation have led to new concepts, doctrine, and TTPs, as well as new training initiatives. FBEs have been successful in the process of developing and transitioning into the fleet decision-support concepts and tools intended to increase the speed of command. Other FBEs enabled the collaboration of command echelon decision makers. One specific effort is the invention of a Knowledge Web (K-Web) concept. As described in Chapter 3, the K-Web involves the application of knowledge-management practices to warfighting, creating a concept of operations in which value-added information (i.e., knowledge) is created and published on the command intranet in real time rather than being coupled to daily briefing cycles. The concept was initially developed in war games, installed as a prototype system on the USS Carl Vinson, and eventually battle-tested during Operation Enduring Freedom, where it was viewed very favorably. The Network-Centric Information Center is working to migrate these tools and transition them to other battle groups. The K-Web demonstrates a significant first step toward a new concept of operation for warfighting and is explicitly designed to support distributed collaboration. Other examples of successful transitions resulted from the Marine Corps Hunter Warrior, Urban Warrior, and Capable Warrior campaigns, which shaped doctrine and TTPs in the field in the following areas: Introduction of new command relationships, Development of the Combat Decision Range, Construction and use of the urban Close Air Support range (Yodaville), Development of military operations in urban terrain (MOUT) for infantry tactics, Continued refinement of the CBIRF organizational structure, and Construction of the night laboratory for training infantry tactics. In particular, the results of Urban Warrior have influenced major changes in the doctrine of how to fight in urban terrain. Additionally, many of the training lessons learned have been captured in the X-files, which are distributed throughout the Marine Corps down to the squad level. The development and building of the night laboratory for the training of rifle squads and all lieutenants at the Basic School (the primary school of basic training for all Services) are a direct result of HW and UW. Another training tool, the Combat Decision Range, is also a direct

OCR for page 141
The Role of Experimentation in Building Future Naval Forces product of the experimentation program. The use of paintball and chalk small arms rounds for urban force-on-force training is also a result of the Marine Corps experiments in urban warfare. The instrumented Close Air Support range (Yodaville) constructed near Yuma, Arizona, is a direct result of UW and is currently used by Air Force, Navy, and Marine Corps pilots. Transitioning Results to Acquisitions and Fielded Capabilities In reviewing the experimentation efforts of the NWDC and the MCCDC, the committee concludes that there have not been any major end items of equipment that have moved directly into acquisition as a result of experimentation. Navy The Navy’s process for turning concept development over to the acquisition system is viewed from within as being poor5 (discussed in more detail later in this chapter). Instead, one sees examples of the Navy moving directly to the procurement of interim capabilities, sometimes prematurely. Some critics of the Naval Fires Network believe that it was rushed into the fleet with insufficient consideration of alternative ways to achieve the same ends, while supporters view the movement as an opportunity to understand the NFN’s potential and to use that knowledge to guide its further development and effectiveness. However, as noted in Chapter 3, Navy experimentation results (e.g., with the high-speed vessel6) have influenced some acquisitions, such as the Littoral Combat Ship. The route of entry seems to be through interested individuals in leadership positions, evolutionary upgrades of existing systems, or modification of the design concept or implementation plans of ongoing programs of record. The NFN has the potential for the most successful transition of the results of an FBE-tested concept, although it is not yet a formal program of record. The successful test of the NFN in FBE-I and its subsequent introduction into the fleet as an interim prototype on two carrier strike groups demonstrate that experimentation may be used as a vehicle for the rapid introduction of new capabilities into the fleet. 5   Admiral Dennis Blair expressed his concerns on this matter while still serving as U.S. Commander in Chief, Pacific (see ADM Dennis C. Blair, USN, 2002, “Force Transformation in the Pacific,” remarks at U.S. Naval Institute/Armed Forces Communications and Electronics Association Western Conference 2002, San Diego, Calif., January 15, pointing to examples of systems readily available in the civilian economy that have not yet been adapted for use in the operating fleets. 6   As with its simulated use in FBE-J, the high-speed vessel is serving as a command and control platform and staging base in Operation Iraqi Freedom. See Jason Ma, 2003, “Catamaran Deployed in War: Naval Special OPS Use Joint Venture in Operation Iraqi Freedom,” Inside the Navy, Vol. 16, No. 13, March 31, p. 1.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces Marine Corps The committee observed that the Marine Corps has had some successes in getting interim minor end-item equipment capabilities fielded as a result of experimentation—for instance: Nonlethal weapons sets for deploying Marine Expeditionary Units, Laser eye protection kits for deploying units, PAQ-4 night alignment sight for the M-16, Intrasquad handheld radios for all USMC units, Elbow and knee pads for MOUT operations, and Infrared-treated battle dress uniforms. Successful results of experimentation for major equipment items have been very difficult, if not impossible, to transition to fielded capabilities. Examples of successfully transitioned results include: Interim fast attack vehicles, Precision targeting systems for forward observer/forward air controllers (FOFACs), and The Dragon Eye UAV. The interim fast attack vehicle (discussed in Chapter 3) has been fielded to the operating forces, while the program of record—the light strike vehicle—works its way through the acquisition system. To the committee’s knowledge, the debate on displacing the program of record did not occur. The commercial off-the-shelf (COTS) intrasquad radio (experimented with during Urban Warrior) has been fielded as an interim capability, while the formal acquisition system works on the Joint Tactical Radio System. The forward observer forward air controller system equipment, barely workable during Hunter Warrior, was improved in terms of its reliability, ruggedness, and performance during Urban Warrior; the prototype systems are now deployed with units. Meanwhile, the formal FOFAC program of record—that is, the Target Location and Designation Handoff System (TLDHS) and the Target Handoff System (THS)—is working its way through the aviation acquisition system. Additional experimental equipment currently in operational use includes the Dragon Eye UAV, which was sent to the First Marine Expeditionary Force in Kuwait to support Operation Iraqi Freedom. This small reconnaissance and surveillance system provides real-time video imagery and can be put in a backpack. An acquisition program for the Dragon Eye is scheduled to follow its extended evaluation by users, although it has already undergone more limited development and operational evaluation, as well as participation in the Millennium Dragon ’02 Marine Corps experiment.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces Transition Planning Logically, one might expect that well-designed and well-executed experimentation campaigns would be so persuasive that the transition of equipment from them into acquisition would be straightforward. In practice, this is clearly not the case. The Marine Corps instituted formal planning to facilitate transition. The Navy does not appear to have done so, relying on personal interactions by leaders to move capabilities forward. The lack of a formal transition plan is only one impediment to successful transition. Success depends fundamentally on satisfying a requirements process and on having resources made available in advance. That is, if an experiment campaign ends gloriously but no funds have been provided in the Service Program Objectives Memorandum, exploiting the fruits of experimentation proves to be extremely difficult for obvious reasons. A solution should rely on planning and programming in advance and/or on invoking a process that amounts to a competition between the capabilities developed through experimentation and the program of record. Since the program of record represents the results of many compromises and a prioritization by the senior leadership, the notion of supplanting an agreed-to program with a new one emerging from an experiment is difficult. Since time is important, the transition process needs to start early—at the time the experiment campaign is planned, in anticipation of success. Not all experimental capabilities warrant transitioning by displacing a program of record. Potential capabilities may remain promising but unrealized because their introduction into the field is not as effective a return on investment as is the capability already programmed. Yet if experimentation is to provide results to the fleet, the debate should be allowed. The present process is woefully inadequate; it almost guarantees that the results of successful experimentation will not enter the acquisition process or become the basis of a program of record. ASSESSMENT OF SPIRAL DEVELOPMENT There were two sources of motivation for considering spiral development in this report. First, the terms of reference for the study specifically asked the committee to do so. Second, the network-centric operations report of the Naval Studies Board recommended that the Navy and the Marine Corps apply spiral processes within the experimentation framework as a means to develop concepts and processes and to accelerate fielding capabilities.7 All of the groups delivering briefings and written materials to the committee referred to spiral development, but the term carried very different meanings. 7   Naval Studies Board, National Research Council. 2000. Network-Centric Naval Forces: A Transition Strategy for Enhancing Operational Capabilities, National Academy Press, Washington, D.C.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces Many individuals seemed knowledgeable about the spiral development process. In some cases, what was described as spiral development was actually traditional product improvement in phases.8 Relatively little evidence was seen in naval briefings of an explicit planning process for true spiral development, which emphasizes iterative cycles of prototypes with substantial learning and adaptation based on operator feedback to drive subsequent spirals before delivering a fielded capability. More typically experiments—such as those using commercially available high-speed vessels—amount to the use of a prototype or surrogate that approximates what will eventually be required. The information from one experiment featuring the vessel is used to shape a subsequent experiment and to answer additional questions. However, such a sequence of events does not constitute a deliberate spiral acquisition. The submarine community used a spiral methodology for the Advanced Rapid COTS Insertion (ARCI) program to address the (then) loss of acoustic advantage in U.S. submarines.9 Another example of experimentation used to support a continuous evolutionary improvement is that of the FOFAC, cited above. Developed under Hunter Warrior, this system was the subject of enhancements during Urban Warrior and resulted in an interim fielded capability. These things said, the committee was struck by the manner in which naval experimentation is disconnected from the Navy’s acquisition community. In some briefings it was asserted that experimentation was not relevant to acquisition. Only in a few instances was there evidence of a deliberate, planned connection—to apply what was learned in experiments to shape system acquisitions or to turn the results into a coherent acquisition program.10 It appears as if some Navy interest in experimentation is really based on a desire to bypass the acquisition process. This is understandable, given the notoriously long times for normal system acquisition. But the price paid is high: By hopping from concept development to experiments and interim capabilities, the Navy is bypassing the organizations with the expertise to do systematic engineering, capabilities-based planning, and operational assessments. The committee also observed that the Navy’s acquisition components seem to be viewed by some leaders as a major part of the problem, rather than as part of the solution. Nonetheless, the Naval Fires Network could be cited as an example of something that leaped quickly from concept demonstration to directed procurement. The 8   Virtually all major systems have involved evolutionary development and deployment under names such as “block change” or “planned product improvement strategy.” 9   In Chapter 3, see the subsection entitled “A Recent Experimentation Program: Advanced Rapid Commercial Off-the-Shelf Insertion.” 10   The most prominent example cited is that of the Littoral Combat Ship, which is using experimentation with the high-speed vessel to determine key performance parameters.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces basic idea is good and interim capabilities are much to be desired, but a refined design is probably needed for all but the near term. The NFN could be viewed as an example of a capability that could profit from a deliberate spiral development process managed collaboratively by the experimentation and acquisition communities. Such problems can be remedied in part by the advanced planning process described above, since the initial experimentation with the NFN would have been viewed as part of a campaign, with orderly provision for acquisition built in at the start. The committee concluded that the spiral development process is used at the discretion of the individual manager and is confined to discrete experimentation events or individual acquisition capabilities rather than being applied systematically. An assessment of the effectiveness of this methodology could not be made because its use is too sporadic. The committee believes that the Navy and the Marine Corps have not yet adequately explored spiral processes or spiral development, particularly in the context of experimentation campaigns, as was recommended by the network-centric operations report of the Naval Studies Board. ASSESSMENT OF THE NAVAL EXPERIMENTATION PROGRAM AND ITS METHODS The terms of reference specifically ask the committee to determine whether any important questions were omitted from the naval program of experimentation. They also request an assessment of the adequacy of processes and methods, particularly in planning. As discussed below, these are not unrelated questions. In short, there are key omissions from the current program of experimentation, and some of these result from inadequate methods. Surveying Future Challenges A key strategic question is whether the naval experimentation programs are focused well strategically. The committee concludes that, at the highest level, the Navy and the Marine Corps are both doing well here. They are both “capabilities-oriented,” as distinct from being oriented toward specific scenarios. This is consistent with DOD guidance11 and long-standing naval traditions. For example, the components of the new naval force strategy Sea Power 21 (e.g., Sea Strike, Sea Shield, and Sea Basing) are quite general. So also, the primary enabler being emphasized (FORCEnet) describes a broad capability rather than something designed for a narrow function suitable in only special cases. 11   See Donald H. Rumsfeld, Secretary of Defense, 2001, Quadrennial Defense Review Report, Washington, D.C., September 30. The classified defense guidance issued in 2002 reportedly reinforced the emphasis.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces underlying the experimentation program is currently unbalanced in favor of relatively detailed models and simulations tied to the large experiments. Its analytical work is not well suited to broad and rigorous capabilities-based planning,35 as distinct from working through particular scenarios in more detail. Consequently, there is a shortfall in families of models and games. Figure 5.1 elaborates on why families of models and games approaches are needed by showing that different members have different strengths. Low-resolution models, if designed for the purpose, can be excellent for exploratory analysis and design—covering a breadth of cases, but not in much detail. In contrast, entity-level simulation can provide a much richer depiction of some underlying cause-effect relationships, i.e., of the underlying phenomena. However, such detailed simulations are not appropriate for broad, design-level exploration,36 nor even for doing a good job in FBE-style experimentation.37 Nor are they currently detailed and accurate enough to represent some important issues (e.g., performance of U-2-based C4ISR as a function of operational circumstances, pilots, weather, and so on), which need to be observed with real platforms.38 War games can bring in human warfighters, who are essential in some play and represent potential users. Field tests with live forces, supplemented by simulations of actual battle, can provide the ultimate experimentation—short of war itself—but are inherently limited in many respects. And, again, they are not very good at supporting understanding of the breadth of possibilities for drawing generalizations, both of which require more abstracted modeling and analysis. Improving the balance of analytical work underlying the experimentation program would require a realignment of effort within some of the analytical 35   Capabilities-based planning has been mandated since the 2001 Quadrennial Defense Review. For high-level technical discussion, see Paul K. Davis, 2002, Analytic Architecture for Capabilities-Based Planning, Mission-System Analysis, and Transformation, RAND, Santa Monica, Calif. 36   Important, less-abstracted exploration can be accomplished with more detailed human-machine simulations and could be accomplished by real-world operators in the fleet if the tools for doing so were adequately embedded in operational systems, permitting continuous experimentation along with training. This type of exploration is especially suitable for exploiting concepts and technology in near-term improvements. Higher-level exploration—for example, across scenarios and case spaces—is different in character. 37   For an interesting discussion by one of the developers of the well-respected Naval Simulation System, see William Stevens, 2000, “Use of Modeling and Simulation (M&S) in Support of the Assessment of Information Technology (IT) and Network Centric Warfare (NCW) Systems and Concepts,” 5th International Symposium on Command and Control Research and Technology (ICCRT), held at Australia War Memorial, Canberra, Australia, October 24-26, and sponsored by DOD Command and Control Research Program (CCRP), Office of the Assistant Secretary of Defense (Network and Information Integration), Washington, D.C., and Australian Department of Defence, Defence Science and Technology Organisation. 38   This important point contradicts the notion that simulations can do nearly everything. It was stressed in discussions with Shelly Gallup of the U.S. Naval Postgraduate School, on the basis of extensive experience in fleet battle experiments.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces FIGURE 5.1 Comparative strengths within a family of models and games. groups supporting experimentation, or the creation of new groups. For example, the NWDC’s modeling and simulation group has made enormous strides in recent years in establishing the capability to simulate many aspects of operations sufficiently well so that live and simulated play can be intermixed and so that live players can be stimulated by and can interact with simulations in much the same way as with real-world command and control systems.39 However, these efforts are extraordinarily demanding, in terms of both energy and resources. As a result, the group acknowledges that it does very little of the lower-resolution analysis associated with exploration or systems analysis. One way or another, the Navy needs to supplement these capabilities with other skills (some of which exist elsewhere within the Navy community). The large and complex simulation work is essential if experimentation is to connect and resonate well with fleet operations and if transitions are to occur easily; it is also essential in many instances in which it provides analytical insights not achievable in other ways (at least currently). Elsewhere this chapter discusses the need to emphasize analytical work and experimentation activities that offer good and correct alternatives to large-scale events. The issue is one of balance. 39   Discussions with Guy Purser of the NWDC and Annette Ratzenberger of USJFCOM.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces Tools for Simulation Properly applied, modeling and simulation can both accelerate the pace of experimentation and reduce its cost. Consequently, it is important to both Service-unique and joint experimentation. There is a need and an opportunity to develop naval use of virtual environments (no interaction with field forces) and constructive environments (some field involvement) for experimentation purposes. As the Navy works through the concepts required for Sea Power 21, virtual and constructive simulations can be extremely useful for developing, exploring, and testing the concepts before trying them out with actual forces. In assessing the need to expand naval simulation infrastructure for Sea Trial as well as that for future joint experimentation,40 specific shortcomings in the naval simulation environment emerge. Most simulations exercised by the Navy in the past have involved some forces and have been structured within the context of large field experiments. However, there is a need now for a more robust set of M&S capabilities to align with the many activities involved in the Sea Trial and joint experimentation campaigns, beginning with exploratory concept development. After MC 02, USJFCOM noted that the DOD’s existing M&S capabilities were inadequate to represent future operational concepts. These capabilities do not account for information operations, model new organizations, or capture asymmetric warfare strategies. Also observed after MC 02 was that data analysis tools were too limited and were inadequate for dealing with the large amounts of data collected in MC 02. In response to these shortfalls in the joint infrastructure, USJFCOM will be addressing an entire range of enhancements through its continuous experimentation environment to enable more flexible, higher-fidelity M&S and through the Joint Simulation System training tool to provide realistic, large-scale, simulations placing humans in operational environments. The development and use of such a common simulation environment are necessary as well as desirable, although the Navy and the Marine Corps will also need their own naval-specific simulations. Such an interoperable environment brings twofold benefits: (1) a consistent environment for both joint and naval-only play and (2) the cost economies of developing one environment to serve two (or more) communities. The key observations of the committee’s assessment of naval simulation environments are based on future needs for Service-unique and joint experimentation. They are as follows: While the current modeling and simulation environments have successfully supported major field experiments, there are deficiencies in M&S capabilities 40   In Chapter 4, see the subsections entitled “Joint Forces Command’s Emerging Modeling and Simulation Infrastructure” and “Naval and Joint Linkages in Simulation.”

OCR for page 141
The Role of Experimentation in Building Future Naval Forces to support a full range of experimentation activities, such as concept development, games, and small experiments; in the areas of compatibility across joint and Service simulations; and in the representation of future warfighting environments. The greater use of simulation relative to live forces would allow more experimentation in the future, especially for concepts that apply above the tactical level. In particular, the Navy should make greater use of virtual and constructive simulations for developing, exploring, and testing concepts of operation before trying them out with actual forces. Exploring tactical interactions in simulation environments is necessary, given the increased importance of tactical cross-Service interoperation in current and future warfighting concepts. Both the Navy and the Marine Corps will need to participate in efforts to realize such simulation environments. Significant development of tools for building, validating, and verifying models; for generating scenarios; for populating databases; and for collecting and analyzing data is necessary. Furthermore such tools have to function, interoperate, and integrate into various environments and frameworks, as future experimentation campaigns are defined and executed. These observations and the earlier discussions on platform availability and needed modeling capabilities lead to two findings. Finding for Navy: The infrastructure and tools required for the experimentation campaigns of the future, including those for Sea Trial and joint experimentation, are inadequate. Primary shortfalls include the following: limited availability of ship platforms (compounded by the potential decommissioning of the USS Coronado) and airborne command, control, communications, computers, intelligence, surveillance, and reconnaissance (C4ISR) assets; lack of modeling and simulation capabilities that support a full set of experimentation campaign activities, explore tactical-level interactions, and reflect next-generation warfighting environments; and lack of tools for building, validating, and verifying models; for generating scenarios and populating databases; and for collecting and analyzing data. Finding for Marine Corps: The infrastructure and tools required for the experimentation campaigns of the future are inadequate. Primary deficiencies include the following: lack of modeling and simulation (M&S) capabilities that support a full set of experimentation campaign activities, explore tactical-level interactions, and reflect next-generation warfighting environments; and lack of tools for building, validating, and verifying models; for generating scenarios and populating databases; and for collecting and analyzing data.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces EXPERIMENTATION FOR BUILDING NAVAL FORCES FOR JOINT OPERATIONS All recent U.S. military operations have been joint. Consequently the terms of reference for the study directed an examination of the role of experimentation in building future naval forces to operate in the joint environment. This section assesses experimentation relative to its influence on joint operations and with respect to preparedness for joint operations. Various experimentation venues are included—USJFCOM-sponsored experimentation activities, the Combatant Commands and cross-Service activities, as well as Naval Service activities. Building Naval Forces to Support Joint Operations Ultimately, the whole point of joint experimentation is better joint operations. Experimentation serves to help develop and refine concepts that lead to capabilities, which in turn are reflected in operations. Today, in support of joint operations, each Service brings its own core warfighting capabilities to the fight. Therefore, joint experimentation and Service experimentation both serve to build capabilities for joint operations. As indicated in Chapter 4, joint experimentation performs three primary functions in improving joint operations: Developing of DOTMLPF for the Services and the joint community, Improving the understanding of other Services’ capabilities, and Contributing to the less tangible but important aspect of socialization. To note progress first: The committee was consistently impressed by the major changes in thinking that have occurred among naval personnel within the past few years. With respect to understanding other Services’ capabilities and to socialization, the Navy and Marine Corps officers now appear to have internalized the fact that they will almost always be fighting in joint contexts, often with much more integrated operations than would have been conceived of only a few years ago. This attitude is an important development in the evolution of U.S. military forces. Experimentation is being planned more in terms of a joint context and with positive interest and much effort. The Navy and the Marine Corps realize that they may conduct key, theater-opening activities, with more massive Army and Air Force deployments following as quickly as possible. They also recognize the need to adapt to further shifts in traditional roles, as happened in Afghanistan when the Navy was present throughout the conflict and the Marines entered after the Army Special Operations Forces. Although the joint experiments are in some respects a burden for the individual Services, they are also opportunities and, in any case, unavoidable realities. As a result, both the Navy and the Marine Corps are active participants and are

OCR for page 141
The Role of Experimentation in Building Future Naval Forces increasingly building their own experiment programs around joint scenarios and working to align them with the schedule of joint exercises/experiments.41 These efforts have led to increased collaboration. There is also progress with respect to joint concept development,42 although the migration of new concepts into actual operations is not so easy to trace, perhaps because it is too soon. However, the committee heard specific anecdotal examples of successes applied in Operation Enduring Freedom (OEF) that stemmed from activities connected with preparation for the USJFCOM MC 02 experiment; it heard of other successes in OEF stemming from the Hunter Warrior Advanced Warfighting Experiment. Lessons learned and equipment from experiments (specifically, the Marine Corps fast attack vehicles) were successfully used during combat in OEF.43 There is also evidence that operational experience is helping to guide experimentation planning by USJFCOM. Joint experimentation is still evolving and growing in importance. As of this writing, little development of DOTMLPF had occurred from USJFCOM-sponsored experimentation. Nonetheless, a few USJFCOM recommendations were close to having moved through the JROC approval process (e.g., the Standing Joint Force Headquarters and the Joint Enroute Mission Planning Rehearsal System—Near Term). Collaboration in Concept Development Although the Naval Services and the joint community are working toward joint concepts, the committee believes that interactions are not as collaborative as they need to be. As new warfighting concepts are developed, coordination among the Services and the joint community must ensure that individual concepts are compatible and supporting. This kind of coordination sets the stage for experimentation campaigns that develop capabilities and prepare forces for joint operations. Concept development is focused for the Navy and the Marine Corps, respectively, in the Navy Warfare Development Command and the Marine Corps Combat Development Command. On the joint side, it occurs at the U.S. Joint Forces Command and, more recently, as part of the Joint Warfighting Capabilities Assessment (JWCA) process on the Joint Staff. 41   See Table 3.1 in Chapter 3 and Tables 4.3 and 4.5 in Chapter 4. For instance, FBE-A included Air Force assets, as did FBE-D which added those of the Army. FBE-I was done in conjunction with Kernel Blitz, which had the participation of all four Services and the Marine Corps Capable Warrior experiment. The Navy had 10 experiment objectives incorporated in USJFCOM MC 02, including FBE-J, and the Marine Corps had 3, including Millennium Dragon ’02. 42   See Table 4.2 in Chapter 4. 43   In Chapter 4, see the subsection entitled “The Role of Joint Experimentation in Preparing for Joint Operations.”

OCR for page 141
The Role of Experimentation in Building Future Naval Forces While the NWDC and the MCCDC interact in concept development with USJFCOM and (along with other elements of the Navy) with the JWCA process, the committee believes that this interaction is not close enough. The committee’s observations in this regard are as follows: The naval concepts being developed are generally couched in a joint context, but the extent of specific interaction with the joint community or other Services in developing and refining these concepts did not appear to be significant. The development of joint concepts by USJFCOM and the JWCA process does involve Naval Service representation, but even so, joint concepts seem to be created from scratch. They do not appear in a substantive way to build on the naval concepts that are being developed by the NWDC and the MCCDC. Likewise, there does not appear to be a detailed correspondence between the joint and naval-developed concepts. The joint community itself does not speak with a single voice, in that the various processes are developing different concepts. However, there is reason to anticipate that such differences will be resolved.44 Experimentation Led by the U.S. Joint Forces Command USJFCOM has conducted a number of joint experiments of different types in which the Navy and Marine Corps have participated. These have included war games, constructive simulations, human-in-the-loop simulations, limited-objective experiments, and large-scale field experiments.45 In addition, USJFCOM is planning a significant program of future experimentation across all types. This program will be characterized by two paths: (1) prototype refinement and validation, which have a nearer-term focus, and (2) concept development, which has a longer-term perspective.46 The Navy and the Marine Corps are, of course, expected to be active participants in these future experiments. To date, most discussion of USJFCOM-led experimentation has centered on the congressionally mandated field experiment Millennium Challenge ’02, which was much larger than any prior joint experiment. The Navy and the Marine Corps each conducted its own major experiments—Fleet Battle Experiment-Juliet and Millennium Dragon ’02, respectively—within the overall experiment. On the 44   As this study was being completed, the Joint Staff was drafting the Joint Capstone Concept and the Joint Capabilities Integration and Development System document (CJCSI 3170.01C). Completion of the Joint Capstone Concept and its adoption by USJFCOM and realization of the processes specified in CJCSI 3170.01C could do away with the differences noted here. 45   See Table 4.1 in Chapter 4. 46   In Chapter 4, see the subsections entitled “Future Experimentation of the Joint Forces Command” and “Emerging Strategy of the Joint Forces Command for Experimentation Campaigns.”

OCR for page 141
The Role of Experimentation in Building Future Naval Forces basis of briefings and other information provided by the Navy, Marine Corps, and USJFCOM, the committee makes the following general observations on Millennium Challenge ’02: The value of such a large experiment relative to its cost47 was questioned by a number of briefers and by many committee members. Some felt that funding could be put to better use in a series of smaller experiments both of the joint and the Service-specific types. Others noted, however, that such an event made the assets of other Services available, something that is difficult to achieve in the context of smaller experiments because of all the demands on operational assets. There was full agreement that it is extremely important to select the venue that matches the objectives desired. A larger-scale field experiment may be warranted to test integration, scalability, and a complex set of interactions, but the question remains—How large a scale is necessary? Such large, high-visibility experiments were characterized by many as demonstrations rather than as experiments, with little room for true exploration or failure. If that is the case, these large events may provide some opportunities to showcase unfunded but “ready-for-prime-time” equipment capabilities in hopes of garnering support and supplemental funding. Others noted, however, that a series of smaller events conducted over the 2 years leading up to Millennium Challenge ’02 had allowed for greater exploration and assessment. Since the conduct of Millennium Challenge ’02 in July and August 2002, the general sense in the community appears to be that further experiments of this size are unlikely. Unofficial statements from USJFCOM indicate that it will not conduct any more major joint experiments like Millennium Challenge ’02 for the foreseeable future, but will focus its efforts instead on smaller, more frequent events.48 The Navy and the Marine Corps should be active participants with USJFCOM in the planning now going on to define these future experiments, both for near-term prototype development and longer-term concept development. Experimentation in the Combatant Commands Joint experimentation also takes place in the Combatant Commands.49 The issues involved in joint force operations are so important and large in scope that it is only natural that all Combatant Commands are involved in addressing them 47   The committee repeatedly heard that the total cost of MC 02 was approximately $250 million, including the costs of the Services and of USJFCOM, but it has not been able to verify this number. 48   Statement attributed to ADM Edmund Giambastiani, Commander of USJFCOM; see Anne Plummer, 2003, “Chief Says More Risks to Be Taken: USJFCOM Says No More Large-Scale Events Like Millennium Challenge,” Inside the Pentagon, March 27, p. 3. 49   In Chapter 4, see the section entitled “Experimentation in the Combatant Commands.”

OCR for page 141
The Role of Experimentation in Building Future Naval Forces through experimentation. This has typically been done in combination with command post and field exercises.50 The Navy and Marine Corps and the Combatant Commands do cooperate in conducting joint experimentation within exercises conducted in the Combatant Commands—Kernel Blitz is one such example. However, there does not appear to be a coordinated program between the Services and Combatant Commands to conduct experimentation in a systematic way. More collaboration of the Navy and Marine Corps and the Combatant Commands to systematically develop programs of joint experimentation appears highly desirable. Past experiences have been valuable, including many that applied to command, control, and communications; that pertained to developing and refining procedures or prototype systems; or that involved coalition nations. However, as indicated in Chapter 4, a number of activities would have to be carried out to achieve a greater degree of coordination in, and systematic expansion of, joint experimentation. One possible mechanism for this coordination could be between the Combatant Command and the Service component commands assigned to that Combatant Command (e.g., either the appropriate Navy fleet command or the numbered fleet commands under that fleet command). The component commands could then interact as necessary with other elements of their respective Service (e.g., with the NWDC and the MCCDC). Cross-Service Experimentation Not all experimentation involving two or more Services need take place under the joint umbrella. Direct Service interaction can be very valuable. In the committee’s view, experimentation involving two or more Services is best suited for operations at the tactical level, since higher-level (component and Joint Task Force) operations are primarily joint. Useful concepts developed in this manner can be fed into the joint arena. There is a need to investigate joint interactions at a tactical level, given the growing intensity in recent operations, such as that in Afghanistan. While the Navy and Marine Corps have devoted increased attention to cross-Service experimentation, more is needed. Direct Service interaction can occur in two general ways—between Service centers involved in concept development and between deployed forces engaged in exercises and experiments. The Navy Warfare Development Command and the Marine Corps Combat Development Command have regular interactions with one another. In addition, the Navy Network Warfare Command has noted that its proximity to corresponding Air Force organizations (e.g., Air Combat Command 50   Table 4.5 in Chapter 4 provides several examples.

OCR for page 141
The Role of Experimentation in Building Future Naval Forces and the Air Force C2ISR Center) offers the opportunity for collaboration. While other, substantive interactions between Service centers may exist, none was apparent to the committee in its investigations. Increased interaction could be very valuable—to provide cross-fertilization in concept development and to begin building joint concepts from the ground up. The Navy indicated in briefings to the committee that other Services (e.g., the Air Force) are involved in its fleet exercises and that it would like to increase such involvement. Recent Marine Corps experiments, with their highly tactical focus, do not appear to have a significant joint or cross-Service perspective. However, future opportunities would appear to exist—for example, coordinated operations with Army Special Operations Forces and air support from Navy and Air Force aircraft. One recurrent difficulty with cross-Service experimentation is the need for the simultaneous availability of assets from two Services. Mutual scheduling is difficult. Ideally, the elements of both Services should be on exercises or engaged in other more formal experiments at the same time. This, however, requires cross-Service coordination of force deployment schedules. Such coordination would have to be carried out at the top levels of the Services—possibly with USJFCOM acting as an intermediary. This would be necessary because the deployment schedules have Service-wide impacts relating to such factors as accomplishing operational missions, training, personnel rotation, and platform overhaul and upgrade. Such a coordination mechanism is not in place today. Balancing Service and Joint Experimentation Since naval operations will become more joint in coming years, linkages between naval and USJFCOM-sponsored experiments, and indeed the full range of joint experimental campaigns, should be carefully planned. This spectrum of activities runs from the earliest concept development, through analysis, war games, and simulations, and leads ultimately to LOEs and large field experiments. An orderly progression of these activities will greatly aid in the efficient use of Service and joint resources. In all of these phases, joint experimentation coexists with Service-specific experimentation because proficient Service core capabilities are a prerequisite for joint warfighting. Alternately, joint experimentation and the attendant joint concept development may require that the Services develop new capabilities. Service experimentation programs should progress in an orderly fashion so that they can feed into and contribute to joint experimentation while also deriving benefits from the joint activities—all the better for both Service and joint capabilities. USJFCOM has laid out a detailed and intense joint experimentation campaign. The Services support and participate in this effort while in turn performing their own Service-unique experimentation activities. These multiple responsibili-

OCR for page 141
The Role of Experimentation in Building Future Naval Forces ties stretch the already resource-limited Service-specific experimentation efforts. It has already been noted that the MC 02 event significantly stressed the staff resources of the Marine Corps. If, in the worst case, USJFCOM’s activities interfere with or prevent the normal progression of Service experimentation, they can actually be counterproductive to developing improved joint capabilities. Thus, a balance must be maintained between joint and Service-unique experimentation, with events and objectives synchronized within the respective experimentation campaigns. To date there is no mechanism in place for achieving this balance. Summary with Findings In assessing the role of experimentation in building forces for joint operations, the committee noted progress both for the Navy and the Marine Corps. The following limitations and opportunities also exist in the current situation: There is a need for expanded and more synergistic collaboration in joint concept development. There are opportunities for expanding joint experimentation through the Combatant Commands, but a coordination mechanism is needed to develop programs of experimentation systematically. There are opportunities and a need for more cross-Service experimentation, including investigating joint interactions at the tactical level (particularly important for the Marine Corps), given recent operations such as that in Afghanistan. However, a mechanism is required to schedule the simultaneous availability of assets for the Services involved. There is a need to balance and synchronize joint and Service-unique experimentation, given the demands placed on the resources available. These observations lead to the following finding. Finding for the Naval Services: The naval and joint experimentation programs are not yet adequately aligned and synchronized, nor is there sufficient correlation between them. More synergistic collaboration in joint concept development is needed. Service-unique experimentation has been and could continue to be affected by large-scale joint experimentation, but no formal mechanism exists for striking a proper balance between joint and naval experimentation.