7

Challenges in Assimilating and Exploiting M&S Technology

TRADITIONAL CHALLENGES

As noted earlier, M&S is an enabling technology with great potential. However, whenever such a cross-cutting technology is introduced, there are organizational and managerial challenges. It is commonplace for the organization to measure the value of investments against the wrong yardsticks (e.g., saving money in narrow domains, as distinct from changing the very way the organization does business and improving effectiveness for mainstream missions). It is also common for investments to go awry because the new technology is procured and used as an add-on without sufficient buy-in and influence by the organization's core work and workers, because too much is done by committee without leaders and champions who understand the core business, or because the educational groundwork has not been laid.

IMPLICATIONS FOR THE NAVY

Observations

Despite much ongoing success, all of these problems are visible with respect to M&S in the DOD's components, including the Navy and Marines. As examples,

  • While substantively broad and ambitious, the Navy's plan for M&S was, as of late 1996, rather “defensive,” with viewgraph emphasis on cost savings and



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force 7 Challenges in Assimilating and Exploiting M&S Technology TRADITIONAL CHALLENGES As noted earlier, M&S is an enabling technology with great potential. However, whenever such a cross-cutting technology is introduced, there are organizational and managerial challenges. It is commonplace for the organization to measure the value of investments against the wrong yardsticks (e.g., saving money in narrow domains, as distinct from changing the very way the organization does business and improving effectiveness for mainstream missions). It is also common for investments to go awry because the new technology is procured and used as an add-on without sufficient buy-in and influence by the organization's core work and workers, because too much is done by committee without leaders and champions who understand the core business, or because the educational groundwork has not been laid. IMPLICATIONS FOR THE NAVY Observations Despite much ongoing success, all of these problems are visible with respect to M&S in the DOD's components, including the Navy and Marines. As examples, While substantively broad and ambitious, the Navy's plan for M&S was, as of late 1996, rather “defensive,” with viewgraph emphasis on cost savings and

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force the absence of redundancies. This reflected the attitudes of senior audiences to whom the material was being presented, audiences who look skeptically at M&S. There were numerous expressions of concern by office and program directors to the effect that proper investments in M&S would require larger sums of money than available within their own domain alone —ven though such investments would have large long-term benefits. This was of particular concern with respect to simulation-based acquisition. The Navy's coordination office for M&S has been organizationally weak and may not be well located—especially if the intention is to link M&S to warfighters and decision makers concerned with force structure. The perception exists that the Army and, to some extent, the Air Force have “stolen a march” on the Navy in exploiting distributed interactive simulation (DIS) and in laying the groundwork for the revolutionary changes it will make possible. Some of this perception appears to be due to the Army and Air Force having put together coordinating offices and programs earlier and having communicated their efforts broadly. The Navy clearly possesses expertise in DIS and has begun to use it (e.g., the Kernel Blitz exercise and various activities within the Naval Research and Development Division (NRaD), San Diego, California). However, the perception does generally seem to be correct. Another aspect of the situation that matters is background. Until recently, at least, the Department of the Navy has been relatively aloof from the last decade's activities with distributed war gaming and advanced distributed simulation. Initially, this stance apparently reflected decisions not to invest in what seemed to be unfocused “technologists-going-crazy” activities. Panel members could understand and to some extent sympathize with that judgment. But the situation is far different in 1997 from what it was a decade ago. In future decades—and surely by 2035—M&S, including advanced distributed simulation, will be altogether ubiquitous and crucial. The Department of the Navy as a User and Consumer of M&S A different set of problems relates to the Department of the Navy as a user and consumer of M&S and model-based analysis. The Navy needs to review itself with respect to these matters. While a review of such matters was outside the panel's charge, and the panel drew no conclusions, it notes that there are some troubling reports (see, for example, Calvin et al. (1995)). If the Navy does have problems being a good consumer of M&S, especially high-level M&S at the mission and campaign levels, the problems have nothing to do with technical training or experience in a broad sense. Indeed, the Department of the Navy has many officers educated and trained in technical specialties such as propulsion and aerodynamics. Further, the Department of the Navy is a generally good consumer of M&S and model-based analysis at the component

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force level (e.g., physics- and engineering-level work related to weapon system design). The problem, if it exists, seems to be that not enough officers have been focused on the special problems of higher-level M&S, such as those associated with force-on-force analysis, joint-task-force-level analysis, systems analysis, and force planning. It may be that this is in part a consequence of decisions taken in the early 1980s to eliminate OP-96 within the Department of the Navy's organization and to otherwise downgrade the role of operational analysis and its underpinnings, as evidenced, for example, by the then-controversial actions taken regarding the Center for Naval Analyses and its management. A BACKGROUND OF LEADERSHIP ALOOFNESS FROM M&S An important background observation is that the Navy's leadership has generally been relatively uninterested in M&S in the past. There were good reasons for this aloofness. Why should the leadership pay special attention to M&S as long as components were getting the job done—building ships and aircraft, training, and operating—using models as appropriate? After all, M&S is just a tool. A further consideration here was probably the belief that the M&S technology programs being championed by DARPA were not yet of immediate and great value to the Navy, but could be a sink for money. Yet another concern, which many of the panelists share as the result of their experiences, is that putting too much emphasis on a support activity such as M&S often proves less effective than having core activities “pick up” the support as needed. Finally, the panel notes that much of this report's discussion focuses on mission and campaign-level (theater-level) models, which historically have been of relatively little interest to senior naval officers because they have generally dealt poorly with naval forces. Deterministic models referring to half a carrier battlegroup launching strike missions seemed inappropriate. Further, there was little treatment in the joint models of the intricacies of antisubmarine warfare, over-the-horizon reconnaissance and surveillance, electronic warfare, the differences in operating conditions in different waters, and so on. The models, then, did not seem very good, but they also posed little danger since experienced military officers and analysts could watch over their use and make corrections for foolishness. And, to put the matter bluntly, the Navy was not at the center of attention in related studies and exercises dealing with major conflicts: its role was greater in presence operations, peacetime missions of all kinds, strategic nuclear and counter-SSBN operations, and lesser conflicts involving naval forces. In summary, there have been many reasons for the relative lack of interest in M&S by Navy leadership. Further, the strategy taken may well have been optimal for many years: a strategy emphasizing decentralization and viewing M&S as merely a tool to be used as needed by those responsible for “real” tasks.

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force NEW CIRCUMSTANCES AND THE NEED FOR TECHNOLOGY-DRIVEN ATTITUDE CHANGES Against this background, the panel believes that it is time for a fundamental change in the way the Department of the Navy (and DOD) thinks about M&S. Its potential value is enormous for acquisition, training, and operations. However, major problems are arising, and many forecasters are being much too optimistic about how quickly difficulties will be overcome. The issues are more substantive than technological; they involve model content and quality. In the past, models were best seen as mere tools to be developed and used in specific contexts. The people working on problems had also built or overseen the building of the models, were aware of their limitations, and could deal with them. Also, the results could be observed and assessed by military officers drawing on doctrine, lore, and personal experience for intuition about what was and was not credible simulation behavior. That era is passing. The nature of warfare, and even of peacetime military operations, is changing drastically. Intuition based on traditional doctrine —often dating back to World War II—will no longer be dependable. So it is that all the Services are busily engaged in battle-lab experiments (e.g., the Marines' Sea Dragon activities). One thing is clear, and that is the increased importance of jointness. This is no longer something for lip service, because U.S. military doctrine is now shifting rather dramatically toward a systems-of-systems approach in which jointness is essential (Shalikashvili, 1996). All of this requires extensive simulation, which will become thoroughly embedded in worldwide and theaterwide command-and-control systems. Naval forces have a prime role in many of the envisioned activities, including the theater-opening campaign and theater missile defense. 1 Unfortunately, M&S is becoming extremely complex. The originators of the models will not be present to watch over their use, and many applications of M&S will be made by people who did not develop the models in the first place. For all of these reasons, it is increasingly critical that the most rigorous efforts are made to improve the quality of M&S as judged not only by “validity” (which can be assessed only in context and with discussion of how uncertainty is handled), but also by transparency, flexibility, and in-context testability. At the same time, if the Department of the Navy is to benefit fully from the great potential of M&S—especially in exploiting what is sometimes called the revolution in business affairs—it must arrange for suitable investments. As suggested in Figure 7.1 , taken together, these considerations imply the need for a technology-driven attitude shift represented by a strategic commitment to exploiting M&S technology. This should lead to strategy, policy, and investments. Key elements of such a strategy would include officer education, vigor- 1   See Naval Studies Board (1996) for discussion of naval forces in regional conflicts. See also the Defense Science Board (1996c).

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force FIGURE 7.1 Despite hype, it is time to look at M&S differently. ous and rigorous pursuit of M&S quality, research to better understand the phenomena of and doctrine needed for future conflict, and fundamental research on M&S to generate systems that are more understandable and testable. The other element is large-scale investment in simulation-based acquisition, which can— over the next two to three decades—revolutionize acquisition. ISSUES FOR THE DEPARTMENT OF THE NAVY Conclusions on Managerial Issues To reiterate, and go into more detail, the panel concludes that the Department of the Navy needs to make a strategic commitment to M &S. This is not merely a technology study's expression of the view that the Department of the Navy should “pay more attention to M&S.” Instead, it is a considered judgment about strategy in the information era. Although the panel members have expertise in M&S, the panel also includes a good deal of expertise in strategic planning and organizational behavior. What would such a commitment mean? The panel's answer, based significantly on panelists' experience in their own organizations and knowledge of developments in industry, is that such a strategic commitment will mean ear-marking capital and continuing resources for the following:

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force Simulation-based design, acquisition (major investments), Research on the scientific knowledge base, Research on M&S and analytic methods, including “simulation science” for complex systems and tools to improve transparency, flexibility, and adaptability of models, Education, and Links to related joint worlds. It will also require creating processes for continuing M&S management: Appropriate organizational structure, Common infrastructure development and support, M&S module development and support, Ongoing configuration management, and Coherent and realistic verification, validation, and accreditation of models and simulations. More Specific Recommendations Moving to concrete recommendations, the starting point is to suggest that the Department of the Navy organize for a strategic approach to M&S. Box 7.1 itemizes the recommended actions. The panel does not suggest details about organization, except to observe that whoever is responsible for M&S leadership needs to be in a position of influence. The panel also believes that the Chief of Naval Operations (CNO) would be well served by having a dedicated analytical BOX 7.1 Recommended Managerial Actions Organize for effective shepherding through transition. Establish crack “analytic team” to serve CNO. Conduct studies. Focus, interpret, and filter M&S. Fund key efforts top-down, not by trading within budget categories: focus on innovation and reengineering, not auditing or stamping out redundancy. *Support and exploit DOD's emerging M&S infrastructure: Embrace HLA and activities capitalizing on it. *Invest in developing and nurturing the knowledge base: Support research tagged for key warfare areas and key technical problems. Couple research to operators. Invest in officer education for building, managing, and using M&S (and analysis).

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force organization. Such an office could also provide intellectual guidance to M&S activities, which too often appear to have a technology-push momentum with only vague notions about how to accomplish objectives supporting core missions. The panel urges that the Department of the Navy establish funding mechanisms and funding procedures that assure an appropriately “ strategic” approach to major issues such as simulation-based acquisition and other activities that would help enable reengineering. This would mean providing some capital investment from the top down, both for long-lead-time investments and common-good investments. The items marked with an asterisk in Box 7.1 indicate subjects on which the panel focused. For reasons elaborated elsewhere in the report, the panel recommends that the Department of the Navy leadership embrace the infrastructure initiatives associated with DOD's high-level architecture (HLA) and that it specifically endorse and assure research activities to improve the validity of models, especially those dealing with combat, logistics, and so on (as distinct from, say, sensor-level models where there is a stronger current knowledge base). The Department of the Navy—and DOD more generally—are severely underinvested here, especially since new-era warfare involves new, complex, and unfamiliar operations. Major errors are possible, and scientists, not just computer scientists, are needed here. A Market Approach to Improving Model Quality and Credibility Although the panel has discussed verification, validation, and accreditation (Chapter 4 ), long experience suggests that bureaucratic processes to “enforce VV&A,” while potentially valuable, are unlikely to do the job (and could reduce efficiencies). A more effective approach is to exploit natural market forces, that is, by having model users play a more vigorous role. This may sound like nothing new, but the reality is that model users have traditionally found it very difficult to review and test any but simple M&S, except by observing behaviors in some standard cases. Much more can be done with: Leadership demanding and valuing continued rigorous attention to M&S quality from the outset of projects; Enforced standards for documenting conceptual model (as distinct from the program), the structural relationship between that and the program, and how to review databases (consumers can do tittle without such basic information); Routine use of outside scientific panels to review conceptual models —in part on their own terms and in part to advise the Navy on whether the models and modeling approaches being used exploit the then-current state of expertise; Model-building technology greatly improving Transparency and the ability of users to query the system with particular questions,

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force The ability to do “exploratory analyses” as a routine part of model testing for a given context, and Increasingly competent software agents for verification testing and even domain-specific knowledge testing; Routine mechanisms for “beta testing” by appropriate communities, such as the Naval War College and operating commands. Establishing Research Programs This report does not recommend any particular organizational approach for assuring the needed research. However, some observations are appropriate. It is important to distinguish between the warfare-area research and the more fundamental research. The latter could rather naturally fall into the domain of the Office of Naval Research and possible Navy-DARPA cooperative efforts. The warfare-area research, however, creates some challenges. Several approaches are possible: In each of a set of key warfare areas, require “program managers” (or other relevant managers) to create a research component. Create a higher-level cross-cutting office that would support a portfolio of applied warfare-area research in close cooperation with the above managers as well as warfighters and doctrine developers. Ask some existing organization or combination of organizations to play this role (e.g., the Center for Naval Analyses and the Applied Physics Laboratory, Johns Hopkins University). The first approach might maximize the closeness of relationship between researchers and warriors. On the other hand, the fit might not be natural and the research component might get short shrift. Also, the second approach would be likely to have integrative advantages and economies of scale. The third approach may be considered a variant of the second. Centralization Issues There are many issues that the panel has not attempted to resolve. One of the more important involves degree of centralization. How centralized should the Department of the Navy management of M&S be? Centralization has advantages. For example, it can improve ability to do comprehensive planning and coordination, but it may also lead to bureaucratic frictions and stifle both creativity and problem solving. Decentralization has great advantages. It allows program managers to address their own problems with minimal external considerations. However, this may inhibit realizing efficiencies and reduce effective

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force coordination. Centralized funding makes common-good efforts easier, but the resulting programs may be hobby horses, failures, or irrelevant because of being too distant from where the real action is, within the programs and in operations. It seems clear to the panel that the Department of the Navy needs a good deal more centralization than it has today (almost none), but finding the right balance is inherently difficult. 2 It is even more difficult for outsiders to judge. Education of Officers A key element of any assimilation effort must be an increased emphasis on educating young officers. The effective exploitation of M&S depends upon the experience, knowledge, and wisdom of its practitioners, hence upon their education. The panel recommends Navy investment in such education at all levels: for those who acquire and design M&S tools and also for those who rely on them to guide acquisition, training, and operations. Some of the education should be in the form of enhanced master's and Ph.D.-level programs. Other aspects should include short courses tailored for officers needing refresher courses, technology updates, and preparation for next assignments involving M&S management. One new educational activity is a master 's degree program at the Naval Postgraduate School with OPNAV endorsement. It will emphasize both computer technology (e.g., virtual simulation) and also human-computer interaction modeling, with a strong component of operations analysis. This and other programs—and a competition of programs is important—could make a significant difference over time. Significantly here, the panel does not necessarily recommend an emphasis on computer science alone, but rather a priority on increasing the supply of young officers with rigorous training in the “hard” sciences or engineering that includes solid exposure to modern M&S, including software engineering. It is common for crack teams in industry doing projects with advanced M&S to be composed mostly of engineers, mathematicians, operations researchers, and scientists. Realism suggests that master's-level education is much more likely to create wise consumers than practitioners. Thus, the Navy will wish to consider how many of its personnel should go on to obtain Ph.D.s and how best to link up with the best expertise in the university and private domains. 2   The panel was divided on the recommendations of Calvin et al. (1995) regarding model management. Panel members were sympathetic to this Center for Naval Analyses report's diagnosis of problems (e.g., chronic inattention to verification and validation of models, and inadequate analytical sophistication regarding what model-base analyses are and are not sound), but there was considerable doubt about the prescriptions, which seemed to some of the panel to call for an excessively centralized approach that would generate bureaucracy and associated frictions.

OCR for page 91
Technology for the United States Navy and Marine Corps, 2000-2035: Becoming a 21st-Century Force Education of Analysts and Model Builders Interestingly, the challenge is not just to educate more naval forces officers in M&S, but also to educate more of the analysts and model builders in military operations. This has seldom been recognized as a requirement, but it seems quite important to the panel. One mechanism for doing this is to strongly support the DMSO's efforts on common models of the mission space (CMMS), something that the JSIMS and JWARS programs are currently cooperating on. Other mechanisms could be more formal. For example, the panel can imagine the Naval War College taking a lead role in arranging orientation meetings in doctrine, field visits, and relatively in-depth discussion of the emerging CMMS.