4
Defense Modeling, Simulation, and Games

INTRODUCTION

Modeling, simulation, and games (MS&G) facilitate a broad set of capabilities for analysis focused on defense and national security. The United States uses modeling and simulation (M&S) to advance our understanding of human genetics, weather prediction, crop failure, ballistic performance of new weapons, aerodynamic optimization of fighter aircraft and other warrior, weapons, and information system performance against strategic, operational, and tactical threats and adversaries. Additionally, there has been an increasing dependence on serious games (see Chapter 3) to provide training aids to our soldiers and engaging learning environments to keep our youth interested in education as they step onto a path guaranteeing a continuation of our nation’s innovation cycle. Finally, we observe a growing potential for social impact with national security considerations as games become more interconnected and create their own cultures.

However, the United States as an entity is not the only country, group, or nonstate actor looking to leverage the advantages of MS&G to satisfy strategic and tactical goals, whether they be academic, economic, or military in nature. As the basic components of M&S technology (computer architecture, software, and algorithms) become commodity in many respects, widespread use of these technologies will no longer be controlled by a powerful few (Lardinois, 2009; Ohmae, 1995).

Particular disciplines within the realms of MS&G have deeper potential impacts. Scientific modeling and simulation, long driven by advances in computer architecture and a quest for realistic representation and problem solving, is gaining wider penetration and practical use outside its traditional community as games leverage realistic models such as physics engines (Ponder et al., 2003). Furthering this penetration, open-source communities are distributing prepackaged scientific knowledge in a consumable form for reintegration in code bases that can be leveraged by other parties with minimal investment (Tirole and Lerner, 2000; Bonaccorsi and Rossi, 2003). It is only natural that gains realized in the commercial sectors are being redeployed to the defense industry in the form of virtual reality training environments and kinetic and nonkinetic weapons development and deployment.

This chapter will discuss the defense and national security implications of modeling, simulation,



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 55
4 Defense Modeling, Simulation, and Games INTRODuCTION Modeling, simulation, and games (MS&G) facilitate a broad set of capabilities for analysis focused on defense and national security. The United States uses modeling and simulation (M&S) to advance our understanding of human genetics, weather prediction, crop failure, ballistic performance of new weapons, aerodynamic optimization of fighter aircraft and other warrior, weapons, and information system performance against strategic, operational, and tactical threats and adversaries. Additionally, there has been an increasing dependence on serious games (see Chapter 3) to provide training aids to our soldiers and engaging learning environments to keep our youth interested in education as they step onto a path guaranteeing a continuation of our nation’s innovation cycle. Finally, we observe a growing potential for social impact with national security considerations as games become more interconnected and create their own cultures. However, the United States as an entity is not the only country, group, or nonstate actor looking to leverage the advantages of MS&G to satisfy strategic and tactical goals, whether they be academic, economic, or military in nature. As the basic components of M&S technology (computer architecture, software, and algorithms) become commodity in many respects, widespread use of these technologies will no longer be controlled by a powerful few (Lardinois, 2009; Ohmae, 1995). Particular disciplines within the realms of MS&G have deeper potential impacts. Scientific modeling and simulation, long driven by advances in computer architecture and a quest for realistic representation and problem solving, is gaining wider penetration and practical use outside its traditional community as games leverage realistic models such as physics engines (Ponder et al., 2003). Furthering this penetra- tion, open-source communities are distributing prepackaged scientific knowledge in a consumable form for reintegration in code bases that can be leveraged by other parties with minimal investment (Tirole and Lerner, 2000; Bonaccorsi and Rossi, 2003). It is only natural that gains realized in the commercial sectors are being redeployed to the defense industry in the form of virtual reality training environments and kinetic and nonkinetic weapons development and deployment. This chapter will discuss the defense and national security implications of modeling, simulation, 

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION and games. In particular, the topics of scientific modeling and simulation, cyber and kinetic warfare, propaganda through games, and war gaming will be explored. SCIENTIFIC MODELINg AND SIMuLATION The increased power of computers arising from faster chips and memory, better bandwidth, better algorithms and architecture, and other improvements described in Chapter 2 means that increased levels of realism, accuracy, and fidelity can be included in models, whether a model is used for science and engineering or for games. The level of detail included in a simulated model must be optimized for the complexity of the prob- lem, the hardware platform that will be used to solve the model, the level of qualitative versus quantitative accuracy required, and the time one is willing to wait for the solution. As computer speed increases, the accuracy and fidelity of the solution can also increase while the time to solution decreases. In science, recent increases in computing hardware speeds and capabilities mean that more detailed underlying physics (e.g., fluid flow, thermodynamics, molecular interactions) can be considered within models of radar cross sections, airfoils, weather and climate, and materials, respectively, providing a greater level of predictive accuracy. There are particular fields in which huge breakthroughs in simulation capabilities are having a positive effect, as mentioned in the World Technology Evaluation Center report (Oden et al., 2006; WTEC, 2009). Examples include the life sciences and medicine, where the ability to perform simulations of complex biological molecules such as proteins on millisecond timescales with atomistic resolution is now possible. Verification and validation (V&V) and uncertainty quantification (UQ) are essential elements of modeling and simulation and are critical for proper risk assessment. Despite this, systematic efforts to develop these elements and integrate them into scientific modeling and simulation are not yet prevalent in either academic research programs or industrial research efforts. The 1991 Army Science Board noted that the Army’s use of “unvalidated simulations” should end its use of M&S until V&V is addressed. The report claimed the Army had been overly focused on the “pretty graphics” and fast run time of the sim (Lynn et al., 1991). A more recent study (WTEC, 2009) found that the United States leads only marginally in V&V and UQ through the efforts of major Department of Energy programs (ASC/ASCI/ PSAAP1). Progress in the ability to automate V&V and UQ for scientific simulation would likely show up in the typical academic literature, but may also be developed in-house as proprietary efforts, especially in industrial labs. General methods and strategies developed for V&V and UQ for a given scientific problem are expected to be transferable to some extent to other domains. Rapid progress in this area would give a major competitive edge to the level of predictability of simulation and the accurate assess- ment of risk. See Chart 4-1 for a representation of this scenario according to the technology warning methodology detailed in Appendix C. The ability to include physics-based models in simulation can be exploited to increase the “realism” of games through increased detail and more accurate levels of fidelity of the models solved as part of the games program. Examples of real-world intersections between games and M&S are detailed as case studies in Boxes 4-1 and 4-2. While the open-source movement is seen by many as critical for leveraging scientific, engineering, and business productivity code bases, there is still the inherit danger in free access to information by those with nefarious purposes. In fact, some individuals in the U.S. military, homeland security, and 1ASC stands for Advanced Simulation and Computing, ASCI is the ASC Accelerated Strategic Computing Initiative, and PSAAP is the Predictive Science Academic Alliance program.

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES CHART 4-1 Automation of Verification and Validation (V&V) and Uncertainty Quantification (UQ) Technology Observables V&V and UQ are essential elements of Evidence of progress may be found in: modeling and simulation and are critical for • Published academic research proper risk assessment, yet systematic efforts • Industrial lab output to develop these elements and integrate them into scientific modeling and simulation are not prevalent in either academic research Confidence intervals are expected to be provided with programs or industrial research efforts. a simulation study. Hurricane tracking has reached this level of maturity. Accessibility Maturity Consequence Level 1a Technology watch General methods and strategies developed for V&V and UQ for a given scientific problem are expected to be transferable to some extent to other domains. Rapid progress in this area would give a major competitive edge to the level of predictability of simulation and the accurate assessment of risk. aMany V&V and UQ technologies and tools exist for the physical sciences but are not widely used. This will change through customer expectations. Investments are required to mature V&V and UQ for the social sciences and data-driven applications. BOX 4-1 Case Study 1: Kinetic Kill Physics Serious Game The intersection of games and scientific simulation is intriguing, particularly in the field of serious games (see Chapter 3 discussion). Given the popularity of games throughout the world, it is likely that a gaming interface to scientific simulation tools would create more users of the tools. If properly designed, it would be possible to perform useful scientific simulations without being an expert in the field. Given the great number of Internet users searching for unique and perhaps free gaming environments, consider a mythical example of a serious game of two players in an environment of kinetic kill physics. Behind the scenes could be one of many production-capable hydrodynamics codes. The interface presents Player 1 with the design tools, applicable materials, and parameters to develop explosively fired projectiles (EFPs). Player 2 is provided a tank to design multilayer armor based on both modern and evolutionary materials. Unknown to Player 2, the parameters are limited to the possibilities of what is publically known and speculated about Country A’s latest deployed tank in Country B. The players design, shoot, and defend. Points are awarded, and players can reach higher levels of distinction by winning against multiple oppo- nents. While two players score points against each other, a nation or nonnation-state collects data on the most effective EFPs. In effect, the Internet is used to optimize EFPs against Country A, free of charge. In the example above, the hydrocode could be either an open-source code or one constructed by open-source software libraries. Open source is a celebrated trend for many researchers and is helping accelerate all forms of software development. Open source allows for collaboration across the globe, helps eliminate “reinventing the wheel,” and facilitates rapid application development. A catalog of many popular open-source projects can be found at http://www.ohloh.net, which claims more than 3 billion lines of code by 200,000 developers. Open source also enables leveling the playing field across the globe for applica- tion software.

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION BOX 4-2 Case Study 2: U.S. Asset Assault Simulation Another example of exploiting the new wave of Internet capabilities and open-source software would be in planning an assault on U.S. assets. It is not difficult to imagine a rogue group coupling Google Street View with computer-aided design software to provide a working model for an accurate simulation of a kinetic weapon or weapon of mass destruction attack. One can even imagine virtual-world war games providing accurate geometries of U.S. assets. Players across the globe could work in teams to defend or take out assets. Tactics and results could be recorded for future military plans. By adopting an open-source strategy, countries with small scientific software investments can rapidly catch up to the big players in terms of a software infrastructure, including parallel software capability. One can find open-source biological and chemical software and, perhaps more importantly, the major building blocks for scientific software: finite element libraries, matrix solution libraries, visualization software, and more. A search on “parallel program” results in greater than 200 products. Fortunately, useful simulations for kinetic and nuclear weapons are highly dependent on accurate material behavior models. To date, ac- curate material behavior models have required a delicate, costly experimental program to provide data to validate theory. These models tend to be considered important intellectual property and are protected as proprietary, export controlled, or classified. A watch sign for heightened threats in this area would be the appearance of highly accurate and open- source material behavior models or the emergence of robust technologies to accurately link length scales between atomistic models to continuum models. Unfortunately, models for crude but effective distribution of chemical, biological, or radiological agents are easily obtainable in the open literature. This scenario is represented as technology warning in Chart 4-2. CHART 4-2 Vulnerability of Physical Assets Given Open-Source Software and Information Technology Observables Heightened risk may be found given: Internet-based capabilities using open-source software (e.g., coupling Google Earth with • he emergence of well-modeled simulations that Google SketchUp) for simulation of a kinetic T attack). The virtual war game could be run allow users to adjust the critical coefficients to tune using physics-based modeling of the objects a model and turn a mid-fidelity model into a high- and materials and geolocation similar to the fidelity model rapidly; • he appearance of highly accurate and open-source Defense Advanced Research Projects T Agency’s RealWorld. material behavior models; or • he emergence of technologies to accurately T link length scales between atomistic models to continuum models. Accessibility Maturity Consequence Level 1 Technology warning A major competitive edge to the level of predictability of simulation and the accurate assessment of risk. Moreover, increased vulnerability of physical assets.

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES intelligence communities do find open-source information such as Google Earth to be a threat in the same way that any open-source intelligence can be used to exploit weaknesses in defenses. Access to information will always be creatively used and exploited. This is not to say that there is an overwhelming movement to ban the use of this incredibly helpful technology; rather it is a source of information that needs to be considered controlled by the enemy and can be used against us. Entities that create powerful software that can be readily used to manufacture weapons, artificial intelligence, or other preventable means should carefully consider the impact those advances could bring if openly distributed. As noted in Box 4-2, a rising threat is the accessibility of open-source software and information concerning U.S. assets. Chart 4-2 further details the possible consequences of this open access. CyBER AND KINETIC WARFARE Practically everything that happens in the real world is mirrored in cyberspace. For national security plan- ners, this includes propaganda, espionage, reconnaissance, targeting, and—to a limited extent—warfare itself. (Geers, 2008) There is a growing movement to use a wide range of modeling, simulation, and game technologies to advance the state of the art of cyber and kinetic warfare. The existence of mature virtual worlds, in terms of capability and user base, continues to add credence to the notion of cyberspace not only as a medium of information transport but also as a point of organization and even social identity. From within those boundaries we can begin to expect an increase in all forms of activity, including those influenced by modeling, simulation, and especially games. As the capabilities of tools, virtual worlds, and acces- sibility gain momentum there will be further influence on the way state and nonstate actors wage war. The impact on national security of kinetic warfare is obvious, historic, and dramatic. On the other hand, the migration of espionage, reconnaissance, and propaganda to the cyber realm—enabled by autonomous connections between people and networks (i.e., the Internet)—is still less than 30 years old. In conflicts from Chechnya in 1994 up through the current conflicts in the Middle East, cyber warfare has become an important component and force multiplier of the ground warfare effort (Lewis, 2002; Geers, 2008). The fields of modeling, simulation, and games have the potential to yield an asymmetric advantage in cyber warfare. Enemies could model cyber capabilities for preplanning of electronic and cyber warfare campaigns. Automation of attackers in this domain is a critical area of interest as distributed computer architectures add more power to thousands of mobile devices. Multiplayer online games, as well as more ephemeral games, can be used to augment the distribution of propaganda or viruses. See Chart 4-3 for a summary of these concerns in the technology warning representation. Additionally, technology may evolve to a level of realism and interconnection that would allow actual cyber warfare attacks to be launched and controlled from a game environment. The application of artificial intelligence, agent-based simulation, and realistic cyber environments provides the basic building blocks of a computational analytic framework that allows users to explore the space of potential outcomes. Finally, the drive for ever more powerful computer architectures may produce a complete separation from the current trajectory, of ever more capable human-controlled attacks, and change the face of cyber warfare efforts completely, such as through entirely automated cyber warfare campaigns aimed not just at spreading viruses but also intelligently planning and carrying out online attacks and defenses that are integrated with kinetic and psychological operations activities of the real world.

OCR for page 55
0 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION CHART 4-3 Cyber Warfare Through Virtual Worlds Technology Observables The existence of mature virtual worlds, in Games that introduce real connections to real life terms of both capability and user base, (e.g., external Internet connectively allowing links to continues to add credence to the notion Supervisory Control and Data Acquisition systems), of cyberspace not only as a medium of particularly in the form of virtual worlds. information transport but also as a point of organization and even social identity. Accessibility Maturity Consequence Level 2 Technology watch Enemies could model cyber capabilities for preplanning of electronic and cyber warfare campaigns. Multiplayer online games, as well as fun, viral, ephemeral games, can be used to augment the distribution of propaganda or viruses. Computer Security While much of the expansive topic of computer security is left out of this report because of the abundance of cyber security publications commissioned by the National Academies (such as NRC, 2009), the committee thinks it is worth mentioning here that computer security also can be modeled, simulated, and then acted on. Agent-based cyber warfare draws extreme parallelism from game engine artificial intelligence (AI) design. Put in simplest terms, information assurance and cyber security as a whole are a game of moves and countermoves between two antagonists where two sides theoretically consist of teams of agents (Hamilton et al., 2002). Therefore, with intelligent algorithms and accurate models of an opposing side’s computer systems and information safeguards, an attacking force could launch massively parallel and highly interactive attacks with high confidence of success due to the fidelity of the preceding simulations. The threat of presimulated automated attack is summarized in Chart 4-4. Finding 4-1: Improvements in and the deployment of agent-based simulation technology—that is, tech- nology that simulates the actions and interactions of autonomous characters and/or systems such that an that understanding or a view into the simulated behavior or system can be obtained—as the underpinning of as game artifical intelligence systems could be a source of significant vulnerability to the extent that the United States falls behind in this area. CyBER PROPAgANDA THROugH gAMES The Internet has undeniably provided the world’s largest international information distribution forum. With information comes disinformation and all manner of propaganda, opinion, saboteurs, and simple mistakes (Chttenden, 2006). Common to nearly all forms of electronically distributed information is the return on investment as it “may be attempted for a fraction of the cost—and risk—of any other information collection or manipulation strategy” (Geers, 2008). The Geers article only scratches the surface of how propaganda can be used through games to affect internal opinions and actions.

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES CHART 4-4 Presimulated Automated Computer Attack Technology Observables Computer security can also be modeled, Computer security test-bed environments that allow simulated, and then acted on. Information automated RED-team/BLUE-team testing of large-scale assurance and cyber security as a whole are computer networks. a series of moves and countermoves. Accessibility Maturity Consequence Level 2 Technology watch With intelligent algorithms and accurate models of an opposing side’s computer systems and information safeguards, an attacking force could launch massively parallel and highly interactive attacks with high confidence of success due to the fidelity of the preceding simulations. Political Manipulation Through games on the Internet Large-scale games create a new venue for political manipulation. While some cases of direct politi- cal manipulation of an individual have been popularized by fiction such as The Manchurian Candidate, it is the manipulation of large numbers of people that can potentially be accomplished through Internet media. Enabled by the Internet, political messages can be quickly distributed to the world population with little validation beyond the recipients’ own social filters. Games, modeling, and simulation present a new dynamic in the use of the Internet as a source of propaganda. There are three main components that need to exist in order for a game to have an effect on its user base. First, the computer game should be well designed to distribute a political message, either covertly or overtly. In the former case, the game may be more likely to convert new people to an ideology. In the latter, it is more likely to target a population in order to reinforce a message or derive an action from a willing user base. The Serbian resistance and pro-democracy game A Force More Powerful, for example, teaches the strategy of nonviolent conflict (kohler, 2005). The second requirement to make a game more effective is to adequately model the behaviors of the game’s actors in order to optimize the messages being sent. Effects of this could be significant if properly orchestrated by a foreign power or commercial entity. A third challenge for game designers is to create a game that is compelling and fun to play in order to keep audiences attracted to it for a substantial enough period of time to successfully convey desired messages. WAR gAMES The Evolution of War games War games have been used since ancient times to assess the nature of conflict for opposing and threatening forces. These games typically had two purposes: post event to assess the impact of a differ- ent decision or choice associated with an action taken (i.e., if we had done this instead of that, would the result have changed?) or planning for a future course of action (if we do this, what do we think

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION will happen?). Chess is one of the oldest surviving ancient war games, dating back some 2,000 years (Dunnigan, 2000). Regardless of the game, war games share some common attributes: individual pieces (people, equip- ment, installations) with distinct capabilities, predefined and agreed-on rules of employment; knowl- edge of their power and effects on other player’s pieces; and predefined end states to “win.” One side “moves” its game pieces—typically through a preplanned strategy—to achieve a successful objective in an environment that provides barriers to success by an opposing player going through the exact same process. Playing the game provides the player with the ability to think, react, understand, and make better decisions about move and countermove options and to develop better strategic choices about the use of his or her different capabilities in different ways. The Prussian Army is credited as the first organized army to use true realistic war games as a method to better train, plan, and assess a plan of operations for strengths and weaknesses (Dunnigan, 2000). Moving around small game pieces and equipment on a battlefield map against known or suspected enemy strongholds provided operational insight on the best order of battle in a tactical environment. Massive logistical problems in early World War II led to the development of more complex tabletop simulations for larger scale operations in an effort to solve the age-old problem of getting the right stuff to the right place through the right channels. As the scale and scope of the battlefield have expanded, the need for higher degrees of scientific modeling has grown exponentially. No longer can a table of game pieces on a small topography effec- tively capture planning and execution insights into the effects of moving and employing thousands of “pieces” (e.g., people, equipment) across a global map. Understanding the strategic effects of many simultaneous operational battles encompassing thousands of tactical moves has required an even more complex set of rules and attributes—and a lot more scientific computing power. While the initial use of military war games was largely facilitated to study the art of warfare and tactical maneuver, modern war games have taken on a different military value, serving far more: • As a predictive tool to assess specific combat capabilities and their impacts when employed with other combat capabilities; • As an assessment of the total number of weapons systems from man to machine and their priori- tization, timing, delay, and impact associated with strategic choice among different theaters of operations; • As a means to study the impacts of attrition due to losses on the battlefield; • To assess different concepts of operations using existing capabilities; • For the continued practice and value of “mission rehearsal” that has remained a constant since the initial inception of war gaming; and • For development of new CONOPs (concept of operations), tactics, and strategies. Huge, inflexible models were built over time to support the need for simulation decision-level data. The models and their conventional rules began to create artificial barriers to innovative and creative combat rules. As the battlefield grew more complex, the games themselves expanded into massive exercises that required extensive planning, large war game staffs and institutions, and very large and, complex models with increasingly rigid rules. At the same time, actual warfare increasingly took on a more asymmetric tone, where conventional rules of war were increasingly irrelevant as they were replaced by a far more adaptive and rapidly changing enemy. Over time, traditional war games became missions in themselves, leading to each service developing

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES its own games, staffs, budgets, contractors, and contractor-developed models to support the increas - ing dependency on these games. Large service war games can take a minimum of one to two years to complete from the start of preparation planning to the final report and out briefs. Each of these massive exercises includes extensive game preparation and manpower support to assess a very limited number of issues (between three and five objectives), culminating in hundreds of players physically traveling to and attending a week-long paper exercise (with limited, if any, use of commercial gaming visualiza- tion tools). Participants are given time to execute two to three game moves using primitive models with limited to no adaptability for ingenuity, surprise, and innovation. The staff size alone (both government and contractors) ranges from 20 to 50 people to support game players in the range of several hundred. The logistical burden of this model of gaming is quite onerous. Once the game has started, creative and unanticipated moves can often stop the play until Game Control inserts instructions prohibiting such moves. Some of these games are supported by decades-old models built and rigidly applied to specific rule sets. These limit the use of any weapons system to a defined method and as a result allow no room for on-the-fly adaptation and creativity. The logistics of game and scenario setup typically cost millions of dollars per single-event game and are not often reusable for subsequent games with different scenarios. In addition, because these games must be played in one physical location, there are travel, housing, and other costs to support anywhere from 200 to 500 players. In many ways, lack of adaptation to massively multiplayer online war games has created a new “cadre” of the same players for each and every different service game, limiting the input from a broader community whose jobs and missions do not allow attendance and participation in person. Major Applications of Military War games Today’s military war games are typically used and valued to accomplish four major efforts: • Test new CONOPs, • Assess the impact of resource decisions on future technologies, • Advocate for resources, and • Provide next-generation training support. War games to Test Concepts of Operations War games and their results are often used to assess or test new CONOPs as mission rehearsal, training, or “what if” drilling against a specific enemy in a specific region of the world. According to Peter Schnorr and David Perme (2004), the use of simulations for training today’s warfighters in today’s asymmetrical warfare environments requires investments in advanced technologies that allow more real-time interactive modeling and simulation solutions “on a more unorthodox and spatially complex scale.” Schorr and Perme are convinced that the use of intelligent agents and expert systems can aid the development and execution of the simulation events but not without the capability for more tailorable models and rule sets. An intelligent agent capability might be useful in the collection of data, to create ad hoc queries, and to adjust for lessons learned in each simulation run. Making use of emergent service-based architectures incorporating the flexibility presented by the use of agents and expert systems will allow current and future simulation solutions to behave more unpredictably and erratically, like today’s enemy.

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION War games to Assess Resource Impact War games are now used to assess the impact of different resource decisions if investments are made in future technologies and capabilities (some of which do not exist today but have had enough research and development (R&D) investment and science to believe them achievable) with significant adjustments in budget against potential future threats or vulnerabilities. These “future capability” games are often the environment in which to explore the art of the possible and to assess the value of adjusting force structure or technology plans. These games also provide a mechanism to consider future threats. When building BLUE (coop- erative) and RED (adversarial) forces, the game designers must do a thorough review of friendly and threatening capabilities for the next 15 to 20 years. They gather data on actors throughout the world with R&D investments in hostile or lethal capabilities. Game designers also search for indications of friendly vulnerabilities to develop RED capabilities for their toolbox. There have been attempts to support these games using models, but there have been limited examples of success. More often, these archaic models are criticized for their inability to truly assess the capability impacts where specific technology is still in the “imagine” phase. Instead, the value of these games has become the ability to discover promising new technologies and assess their transfor- mational nature on existing warfare as well as unexpected vulnerabilities that require new capabilities to remedy. The Air Force’s Future Capabilities Game is a good example of this improved type of game. In the 2004 game, the Air Force sought ways to refresh its war games methodology to derive better results for less cost and to reduce the logistical game footprint. Game designers sought ways to distribute the game to more than one location to drive down costs and to reach a broader player who, due to mission and job requirements, could not commit to a week-long, full-time war game at a site far from his station (Rolleston, 2009). This 2004 war game reluctantly eliminated rigid models for the first time and gave the game execution a wider free-play environment with a new self-assessment process. The focus this time was not on the cadre of experienced gamers, but on rapidly gathering insights (via statistical data) from a new category of players: future senior leaders and warfighters. This change was made in anticipation that these new players would take what they learned and experienced in this more fluid adaptive environment and apply it to real situations in the future. The one thing lacking was an intelligent agent and adaptive model that operated in real time with each play providing immediate feedback to the gamers. Both the Air Force’s 2004 and 2005 games cited, as a primary objective, “to educate senior Air Force leaders on emerging technologies, innovative system concepts, and emerging Red [adversarial] capabilities” (USAF, 2003). Instead of emphasizing models that would directly determine a winner up-front, time and investment were made in describing, designing, and distributing a comprehensive toolbox (with graphics, pictures, and digital videos of the capability in action and attributes for each discovered new technology, much like today’s gaming industry provides for its players). Additionally, more effort was allocated in the postgame period to digitally visualize the war game’s outcome for a more game-like experience. This approach to out briefing the results resulted in better visual and experiential understanding for both senior Air Force leaders and members of the labs and scientific communities who participated. 2 The revised war game was considered to have significantly more impact on these senior Air Force deci- sions makers, who drew far more from the lessons and threats they could see “in play” than from the previous years’ war games. The archaic model-constrained data of the previous games were considered to have inhibited ingenuity and creativity. These small changes in the Air Force game hint at the great 2Personal communication between Mort Rolleston and Committee Member Allison Hickey on July 8, 2009.

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES achievement potential possible if all services considered adapting the best practices associated with today’s commercial massively multiplayer war games, which are conducted in a virtual environment 24 hours a day with missions constantly impacted, lessons constantly learned, and players adjusting the rules of play at a moment’s notice. War games to Advocate for Resources Finally, the use of war games has grown as a tool to influence decisions in national defense reviews like the congressionally mandated Quadrennial Defense Review for the games’ ability to demonstrate (with some magnitude of science depending on the service model and its assumptions) one service’s capabilities over another in the ever-growing competition for limited defense budgets. Each service uses its preferred models to provide advantage to its forces and weapons systems over another service. Today’s military war games play critical roles in service budgets and force structure decision processes. Throughout the committee’s exploration of the breadth of technology drivers in modeling, simula- tion, and games, there has been a clear demonstration of the benefits that can be derived from widespread adoption of large-scale leaps in commercial development. However, most defense organizations are consumed and trapped by their past success with outdated methods and they largely ignore the leaps in technology, culture, and practice of the growth in other sectors, like the gaming industry. The use of simulation support for operational decision making has been highly desirable but thus far unachiev- able because the databases that drive the simulations could not be developed and updated fast enough (Schnorr and Perme, 2004). The U.S. Armed Forces have been slow to adopt automated and intelligent warfare simulation envi- ronments. The analogous environments to the pop culture War Games movie are barely in existence (e.g., “move,” “countermove,” “constrained choices”), instead performed through expensive multiyear exercises that simulate relatively few moves between two actor nations. The April 2002 U.S. Army’s Vigilant Warriors war game held at Carlisle Barracks, in Pennsylvania, for example, included a group of 500 U.S. and allied military and civilian personnel, sequestered in rooms filled with wall-mounted maps, telephones, and computer terminals, conducting “tabletop” exercises, all while focused on using tomorrow’s expected military capabilities to mitigate several global crisis scenarios set in the year 2020 (Gilmore, 2002). Recommendation 4-1: Military war games should exploit the significant growth and lessons of seri- ous games to leverage experiential aspects of large multiplayer joint war games. A more real-time, immediate-feedback exploration environment can then be assessed using rapidly updated algorithms, parameters, and coefficients that reflect behavioral and policy implications. The use of serious games to explore strategy or technology implications can be valuable to strategic and long-range concepts of operations, weapons system acquisition, and threat assessment and response and far more effective in providing constant assessment of new technology and CONOPs opportunities, as well as more real- time threat warnings to those who consistently monitor for these issues. This same virtual sandbox can provide rapid assessment of these capabilities in a more affordable virtual method and reduce the manpower-extensive planning, logistical, and cost requirements typically required by large war games that only occur on a two-year basis. Creating an accessible war gaming environment with the dynamic virtual world and persistence like World of Warcraft would provide a far nimbler and more effective approach to conduct real-time assessments of breaking news on emerging threats and therefore improve our ability to counter these threat potentials far earlier in the process. In addition, virtual war gaming environments can be crafted

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION on levels from strategic to tactical and across the full spectrum of military response. These games are used today to train and test tactics, techniques, and procedures (TTPs) and as effective tools for mission rehearsal, cultural development and experience, and familiarization training. War games to Provide Next-generation Training Support A particular nexus of MS&G is in the form of future training environments.3 While modeling and simulation have historically focused more on the accuracy of models while the games industry has focused on the user experience, the two properties are not mutually exclusive in many fields, especially training. As an extension to education, leveraging accurate (validated) computer games has the potential to help improve all aspects of U.S. competitive advantage from medical, basic educational, vocational, and military skills training. While nearly all studies surveyed by the committee mentioned a continued lack of in-depth or systematic studies to fully evaluate the effectiveness of video games in training, many have seen an increased level of participation on the part of the students (Annetta et al., 2009; Carnevale, 2005). In fact, the level of interest from the educational community as a whole shows potential that continued interest in the promise of improved training and education will continue to fund research into analyzing the effectiveness of education (skill acquisition through games is covered further in Chapter 3). However, as indicated by the 1991 Army Science Board survey of M&S initiatives, those investing in the effectiveness of games need to be aware of the validity of the models they are built on (Lynn et al., 1991). Educational games may provide great benefits, but should not be created at the expense of the lesson content. Conversely, games can be used to create the data foundation of future scientific models. As discussed by Orkin and Roy (2007), the use of games in analyzing social behavior in scenario settings is an incred- ibly powerful tool to help understand human behavior in certain situations. Particularly, the massively distributable nature of electronic games combined with the ease of their “instrumentation” allows for a degree of study not efficiently accomplishable in the real world. As shown in Figure 4-1, the game Zero Hour: America’s Medic is designed to teach first responders how to react to large-scale disaster scenarios. This training tool could alternatively be designed and instrumented in a manner that would allow researchers to profile how humans actually behave in such situations. This feedback loop could be a critical step in increasing the efficacy of computer-game-based social models. This discussion and its associated technology watch are summarized in Chart 4-5. Combining the concepts of games for training and for the introspective use of modeling behavior, some researchers are working on using AI systems to augment interactive media offerings in education (Moreno-Ger et al., 2007). IMS’s Learning Design specification is designed to learn from its users’ interaction with the system to change its game offerings to the student as well as the game interactions 3The committee was directed by the study’s sponsor to not produce an in-depth report on modeling and simulation for train- ing military forces since this has been a topic of many previous reports. For additional information, see Impact of Adanced Distributed Simulation on Readiness, Training, and Prototyping, 1993, Defense Science Board, Office of the Undersecretary of Defense for Acquisition, Technology, and Logistics, Washington, DC, January, available online at http://www.acq.osd. mil/dsb/reports/srp.pdf); Modeling and Simulation: Linking Entertainment and Defense, 1997, NRC, National Academy Press, Washington, DC, available online at http://www.nap.edu/catalog.php?record_id=5830); and Adanced Modeling and Simula- tion for Analyzing Combat Concepts in the st Century, 1999, DSB, Office of the Undersecretary of Defense for Acquisition, Technology, and Logistics, Washington, DC, May, available online at http://www.acq.osd.mil/dsb/reports/advancedmodeling. pdf. Web sites last accessed September 21, 2009.

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES FIGURE 4-1 Screen shot from the game Zero Hour: America’s Medic, which allows first responders to role play various disaster scenarios, such as an earthquake, a bombing, or a sarin gas attack. SOURCE: Image courtesy of Virtual Heroes, a Division of Applied Research Associates, Inc. themselves based on the interpreted preferences of the student. Taking this concept into the future, the system would ideally be used to isolate the most effective ways to train a subject. U.S. leaders and industry should consider the benefits that can come from proper use of serious games technology. However, it should be understood that this technology is not likely to stay with the United States, and will likely even be developed elsewhere in some arenas. Already, the Chinese People’s Liberation Army has begun to embrace its own “virtual training systems and tools” (Mulvenon, 2008). Enhanced Military Simulation Despite the lack of widespread games technology used directly in war games, there has been increased use of actual games by the military. One of the most compelling cases for game-based aids to the military is for basic skill and situational training. While games might never be a true replacement for acquiring skill through physical experience, they might very well function as force multipliers for training—helping to overcome the physical bandwidth limitations or steep costs of many live train- ing facilities. Bandwidth constraints, particularly in the tactical battlespace, are a factor that must be addressed if training and planning games are to be deployed to the field. Cooper’s law permits a degree of optimism about the future, but it might be worthwhile to monitor technological progress in this spe-

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION CHART 4-5 Cultural and Behavioral Modeling Through Virtual Environments Technology Observables To detect whether games and other virtual Games, training environments, and virtual environments are becoming a threat, watch for: worlds may be specifically instrumented to log data about how people (or a particular • ass investment in widespread behavioral modeling culture) respond in certain scenarios or M situations. Were adversaries to obtain such through game analysis; • Research in using games for cultural analysis; and capabilities, there might be ramifications • ames and other social media that are created to for psychological operations (PSYOPS). G These environments also present a new appeal to particular demographics while promoting dynamic in the use of the Internet as a an agenda. source of propaganda. Effects of this could be significant if properly orchestrated by a foreign power or commercial entity. Accessibility Maturity Consequence Level 2 Technology watch Effective cultural and behavioral modeling through games and virtual environments could be used to profile the effectiveness of planned terror campaigns—a potential PSYOPS concern—and present opportunities for political manipulation. cific domain, particularly as it pertains to embedded training, distributed planning, and command and control. Bandwidth constraints translate to having to deploy rendering capability (graphical or game play) further on the edges of the network. Any organization or state has the potential to effectively offer richer experiences that permeate the population as a result of increased bandwidth (e.g., South korea). The U.S. Army has demonstrated the highest interest of all the services in leveraging commercial gaming capabilities, through its new Games for Training program, and appears to take this approach seriously, as supported by the funding of different initiatives (Turse, 2003). In fact, the 2008 release of the Army Field Manual specifically called out video games as “serious training tools” (McLaughlin, 2009). Recent Department of Defense (DoD)-sponsored game programs include: • America’s Army (2002); • DARWARS Ambush (2003); • Full Spectrum Warrior (2004); • Stability Operations: Winning the Peace (2005); • Virtual Battlespace  (VBS; 2007); • Game After Ambush (2009); and • RealWorld (2009). The U.S. Army is prepared to invest $50 million in video games to train soldiers for combat. The funds will be used over five years beginning in 2010, as part of its Games for Training program (Jenkins, 2008). The Army and other services are investigating all forms of interactive media and games to augment a wide range of goals, from engaging scenarios to help service members cope with combat stress, to

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES constructing immersive environments aimed at providing a wide range of realistic scenarios for tactical engagement and role play (see Figure 4-2 for an example). Despite increased optimism about games, a number of questions have yet to be answered, such as: • How can enemies use games to learn our tactics and play against us? • Can virtual worlds really teach people to be better shooters or small ambush squads? • Can games be used to train and adapt rapidly to CONOPs capabilities and to leverage command and control training capabilities for joint and coalition partners? • Can online or virtual reality platforms provide for a massive increase in militia-style reserve forces? (This question is of particular concern for guerilla tactics.) Massively multiplayer online role-playing games (MMORPGs) may provide insights into effec- tive new campaigns for terror, PSyOPS, and mass reaction to certain catalysts that can be developed in online environments to provide rapid evaluation of new TTPs, as indicated in the “Global Industry Trends” section of Chapter 3. Finding 4-2: While the United States continues to leverage superior training as part of its ability to maintain asymmetric advantages over potential adversaries, these same potential adversaries may develop the ability to train and adapt CONOPs based on prolific access to Western game genres and actors. FIGURE 4-2 Scenario from the University of Southern California Institute for Creative Technologies’ Flatworld prototype for tactical engagement and role play. SOURCE: Image courtesy of the University of Southern California Institute for Creative Technologies. Photograph by Bradley Newman.

OCR for page 55
0 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION Finding 4-3: In the global war on terrorism, American forces have frequently prevailed in direct-fire exchanges, often attributed to better squad coordination and training—a skill commercial multiplayer gamers practice and develop virtually and routinely. Though it is not expected that this advantage will be maintained when engaged with a sophisticated potential adversary in the field, the United States is at risk of losing its advantage due to the advanced training environments and distributed nature of simula- tion and online gaming either currently or in the near future. Recommendation 4-2: DoD should strongly consider migrating at least one of its Title 10 war games to the emerging architectures of the commercial gaming industry. The Air Force’s Future Capabilities game is a potential candidate to make this leap in order to leverage flexible game play, minimize logistical footprint, and optimize mass player participation across the joint forces. FINAL THOugHTS In recent decades the United States has had a demonstrated lead in technical and computational inno- vation. However, the Internet has created an environment for open innovation, bringing disparate groups together, democratizing access to state-of-the-art technologies, and providing for different approaches and views on problem solving. As a result, individuals or small groups now have tools to exert influence. The Internet is having a significant impact on culture and moreover is accelerating technology develop- ment, communication, and productivity. Social change itself has been accelerated through globalization of attitudes, values, fashions, lifestyles, and languages (Mack, 2009). Lastly, the Internet is accelerating the pace of business. Those who can adapt are more productive, connected, and prepared. The dynamics of which country is leading in cyber technology are also surprising. Currently, China has more Internet users than any other country; the United States is second. However, the United States is thirteenth in terms of Internet penetration (as defined by Internet connections per resident), and U.S. broadband penetration is beginning to slow and plateau, while in Indonesia and Latin America it is expected to show high relative growth (Mack, 2009). Take-away Warnings The United States is not alone in its pursuit of MS&G excellence. Many nations are actively invest- ing in the next generation of computational power. While active research in fields such as game artificial intelligence, the creation of algorithms, and future hardware platforms is often benign, the capability for new advances in warfare is also being created. The basic scientific knowledge of ballistic penetrators is being packaged into open-source academic software and creating the game AI’s that can be reused as smart agent automation for cyber security attacks. Adversaries may leapfrog U.S. capabilities as we continue to rely on mid-1960s tried-and-true, noncomputational analysis due to institutional and political inertia, exemplified by the growing divide between rapidly accelerating cyber technology and U.S. executive leadership expertise in the area. Some suggest that we are already exiting the information age and are now moving into the conceptual age. yet some senior leaders have yet to adopt e-mail or use the Web. This knowledge was exploited in the 2008 U.S. presidential race when candidate Senator John McCain was cast as out of touch with technology after admitting to not using e-mail. In the 2007 U.S. Air Force Future Capabilities Game, one of the findings was that “players found it difficult to develop/integrate cyberspace into campaign plans due to . . . understanding of capabilities and how to integrate with other domains” (Rolleston, 2009).

OCR for page 55
 DEFENSE MODELING, SIMULATION, AND GAMES Many government agencies and contractors have very strict rules for Internet use at work. Some have no Internet access. One must question if a cost-benefit study has been conducted by U.S. agen- cies on Internet use. On the one hand, inappropriate computer use and viruses might be eliminated, but it is difficult to understand how these institutions can stay on the leading edge without learning and experiencing where the rest of the world is headed. Many enlightened companies now encourage their employees to invest time in learning new cyber tools (e.g., Virtual Worlds; Anderson, 2009). At the same time, an obstacle to incorporating games and simulations into the government and military is resistance to what are perceived as “toys.” The pursuit of fidelity for fidelity’s sake, the stipulation to avoid “negative training,” and the lack of understanding of what could be used for focused training hamper adoption of these technologies. Take-away Opportunities Despite challenges, the United States is in a position to take advantage of exciting new applications in the field of MS&G. As one of the world leaders in the creation of serious and nonserious games, our nation possesses a plethora of the talent needed to leverage the full potential to train our troops and educate our population and to do so in a way that may augment or complement traditional styles of learn- ing. Additionally, environments can be created in which to experiment with new ideas and concepts in a fashion that will continue to outpace existing forms of manufacture and development. These revolutions will not happen without proper resourcing, and U.S. leadership must make sure that hurdles to success are removed and innovation in these areas is encouraged. REFERENCES Published Annetta, Leonard D., James Minogue, Shawn y. Holmes, and Meng-Tzu Cheng. 2009. Investigating the impact of video games on high school students’ engagement and learning about genetics. Computers & Education 53(1):74-85. Bonaccorsi, Andrea, and Cristina Rossi. 2003. Why open source software can succeed. Research Policy 32(7):1243-1258. Carnevale, D. 2005. Run a class like a game show: “Clickers” keep students involved. Chronicle of Higher Education 51(42):B3. Chttenden, Maurice. 2006. Comedy of errors hits the world of Wikipedia. The Sunday Times, February 12. Available at: http:// www.timesonline.co.uk/tol/news/uk/article730025.ece. Accessed June 24, 2009. Dunnigan, James F. 2000. Wargames Handbook, Third Edition: How to Play and Design Commercial and Professional Wargames. Lincoln, NE: IUniverse. Geers, kenneth. 2008. Cyberspace and the changing nature of warfare. SC Magazine, August 27. Available at http://www. scmagazineus.com/Cyberspace-and-the-changing-nature-of-warfare/article/115929/. Accessed June 25, 2009. Gilmore, Gerry J. 2002. Army war games provide azimuth for DoD’s future force. News Release, American Forces Press Ser- vice, April 29. Available at http://www.defenselink.mil/news/newsarticle.aspx?id=44118. Accessed June 25, 2009. Hamilton, Samuel N., Wendy L. Miller, Allen Ott, and O. Sami Saydjari. 2002. The role of game theory in information warfare. In Proceedings of Fourth Information Suriability Workshop. Available at http://www.cyberdefenseagency. com/publications/The_Role_of_Game_Theory_in_Information_Warfare.pdf. Accessed June 25, 2009. Jenkins, David. 2008. Report: U.S. Army invests $50M in training games. Gamesutra News, November 24. Available at http:// www.gamasutra.com/news/serious-games/?story=21237. Accessed June 9, 2009. kohler, Chris. 2005. Sir, the gamers are revolting! Wired, October 27. Available at http://www.wired.com/gaming/gaming reviews/news/2005/10/69372. Accessed July 13, 2009. Lardinois, F. 2009. Wolfram|Alpha: Our first impressions. ReadWriteWeb, April 25. Available at http://www.readwriteweb. com/archives/wolframalpha_our_first_impressions.php. Accessed June 25, 2009. Lewis, J. A. 2002. Assessing the risks of cyber terrorism, cyber war, and other cyber threats. Washington, DC: Center for Stra- tegic and International Studies. Available at http://www.steptoe.com/publications/231a.pdf. Accessed June 25, 2009.

OCR for page 55
 THE RISE OF GAMES AND HIGH-PERFORMANCE COMPUTING FOR MODELING AND SIMULATION Lynn, Verne L., Peter Cherry, Edward Brady, Paul Droulihet, Jr., and William Evers. 1991. Army Science Board  Summer Study—Army Simulation Strategy. Available at http://www.dtic.mil/srch/doc?collection=t3&id=ADA250382. Accessed July 13, 2009. McLaughlin, Seth. 2009. U.S. Army expands use of video games for training. World Politics Reiew. Available at http://www. worldpoliticsreview.com/article.aspx?id=3724. Accessed June 25, 2009. Moreno-Ger, Pablo, Daniel Burgos, José Luis Sierra, and Baltasar Fernández-Manjón. 2007. A game-based adaptive unit of learning with IMS learning design and . In Creating New Learning Experiences on a Global Scale: Second European Conference on Technology Enhanced Learning, Erik Duval, Ralf klamma, and Martin Wolpers, Eds. Berlin and Heidelberg: Springer-Verlag, pp. 247-261. Mulvenon, James. 2008. “True is false, false is true, virtual is reality, reality is virtual”: Technology and simulation in the Chinese military training revolution. The “People” in the PLA: Recruitment, Training, and Education in China’s Military. Roy kamphausen, Andrew Scobell, and Travis Tanner, Eds. Carlisle, PA: Strategic Studies Institute of the U.S. Army War College, pp. 49-90. NRC (National Research Council). 2009. Technology, Policy, Law, and Ethics Regarding U.S. Acquisition and Use of Cyberat- tack Capabilities. Washington, DC: The National Academies Press. Oden, J. Tinsley, Ted Belytschko, Jacob Fish, Thomas J. R. Hughes, Chris Johnson, David keyes, Alan Laub, Linda Petzold, David Srolovitz, and Sidney yip. 2006. Reolutionizing Engineering Science Through Simulation: A Report of the Na- tional Science Foundation Blue Ribbon Panel on Simulation-Based Engineering Science. Arlington, VA: National Science Foundation. Available at http://www.nsf.gov/pubs/reports/sbes_final_report.pdf. Accessed July 1, 2009. Ohmae, kenichi. 1995. The End of the Nation State: The Rise of Regional Economies. New york: Simon and Schuster. Orkin, Jeff, and Deb Roy. 2007. The restaurant game: Learning social behavior and language from thousands of players online. Journal of Game Deelopment 3(1):39-60. Ponder, Michael, George Papagiannakis, Tom Molet, Nadia Magnenat-Thalmann, and Daniel Thalmann. 2003. VHD++ development framework: Towards extendible, component based VR/AR simulation engine featuring advanced virtual character technologies. Computer Graphics International Proceedings, pp. 96-104. Available at http://vrlab.epfl.ch/Pub- lications/pdf/Ponder_and_al_CGI_2003.pdf. Accessed January 13, 2010. Schnorr, Peter, and David Perme. 2004. Improving asymmetric threat representations through the use of intelligent agent technology. Presented at the Spring Simulation Interoperability Workshop, Orlando, FL, Simulation Interoperability Stan- dards Organization. Available at http://www.sisostds.org/index.php?tg=fileman&idx=get&id=2&gr=y&path=Simulation+ Interoperability+Workshops%2F2004+Spring+SIW%2F2004+Spring+SIW+Papers+and+Presentations&file=04S-SIW- 063.pdf. Accessed June 25, 2009. Tirole, Jean, and Josh Lerner. 2000. The simple economics of open source. HBS Finance Working Paper No. 00-059. Available at http://ssrn.com/abstract=224008. Accessed June 25, 2009. Turse, Nick. 2003. Bringing the war home: The new military-industrial-entertainment complex at war and play. TomDispatch, October 17. Available at http://www.commondreams.org/views03/1017-09.htm. Accessed July 13, 2009. USAF (United States Air Force). 2003. FY0 Wargaming Assessment Report. Report written by the Air Force Research Laboratory Information Directorate, Rome, Ny. Available at http://www.dtic.mil/cgi-bin/GetTRDoc?AD=ADA420838& Location=U2&doc=GetTRDoc.pdf. Accessed January 13, 2010. WTEC (World Technology Evaluation Center). 2009. Sharon C. Glotzer, kim Sangtae, and Peter T. Cummings, Abhijit Deshmukh, Martin Head-Gordon, George karniadakis, Linda Petzold, Celeste Sagui, and Masanobu Shinozuka, panel members. International Assessment of Simulation-Based Engineering and Science. Baltimore, MD: WTEC. Available at http://www.wtec.org/sbes. Accessed June 25, 2009. unpublished Anderson, Laura. 2009. Computing technology landscape and directions: Impact on simulation, modelling, and gaming. Pre- sentation to the committee, January 12, 2009. Mack, Timothy. 2009. Gaming and Culture: Impact of the Internet. Presentation to the committee, February 26, 2009. Rolleston, Mort. 2009. Future capabilities game 07 results. Presentation to the committee, February 26, 2009.