As the chapters of this book have noted, synthetic environment (SE) systems present a number of highly challenging scientific and technological research problems that, in an ideal and unconstrained funding environment, could be supported solely on the basis of intrinsic scientific interest. However, the funding environment is hardly unconstrained. Policy makers are more concerned with the promise of applications to areas of national need or economic significance than they are with the purely scientific or technical promise or potential in a given area of investigation. Cognizant of this priority, researchers are often tempted to promise a cornucopia of applications even when the state of the art does not permit realization of these applications. Avoiding the hyperbole that leads to a later fall is thus of paramount importance.
Both virtual environments (VE) and augmented reality pose challenging intellectual problems and also evidence considerable potential for a wide variety of applications; however, with few exceptions (one of which is entertainment), serious commercial applications (as opposed to research demonstrations of concept feasibility and promise) are likely to be realizable only in a long-term time frame—perhaps 5 to 10 years. This is not to say that meaningful progress cannot be demonstrated in a shorter time frame—only that according to the metric of commercial viability, major economic and social benefits are not expected to be demonstrated in the near future.
Teleoperation, in contrast, has already been used extensively in a variety of activities, including handling nuclear materials, operating heavy machinery, exploring space, performing underwater inspections, and removing hazardous waste. Furthermore, there are a number of experimental programs in which the use of teleoperation is being explored for surgery, patient monitoring, and delivery of remote treatment. Although teleoperation technology can provide many potential benefits, it has not, to date, been demonstrated to have a high commercial value.
That said, the scientific and technical study of SE is entirely compatible with a tight integration between research and applications development. The history of SE suggests that many interesting research problems in the field have arisen from difficulties faced by applications developers concerning, for example, perception, motion sickness, and software development for real-time interactive systems.
Given that applications-oriented work in areas of national need is appropriate, it is important to choose judiciously which applications should be singled out for near-term attention. An important consideration in this choice is the fact that the entertainment industry has been the primary driver of nonfederal work in VE. For all practical purposes, this industry has supported the development of low-cost, low-performance proprietary hardware and software. This suggests that federal efforts
should focus more appropriately on the development of medium- and high-end technology that can be readily shared. However it does not specify which among the various possible application areas should be most strongly supported. The committee has developed five criteria for determining appropriate application areas. Specifically, the applications should be:
of demonstrable national importance;
realizable in a relatively short time;
an area to which SE technology can make unique contributions; and
an area in which success can be achieved through modest application of additional federal efforts.
Using these criteria, the committee focused on the areas of design, manufacturing, and marketing; medicine and health care; hazardous operations; training; education; information visualization; and telecommunications and teletravel. Our selection of these seven areas should not be construed to mean that these are the only areas worthy of special interest. A different committee with different expertise and interests could well have chosen another set.
Important omissions to this list are entertainment, art, and national defense. Entertainment is not discussed in detail because it is receiving significant commercial support and because the development of technology for this purpose was not deemed a pressing national need. Art is not discussed separately because VE in its current stage of interface technology development provides only marginal opportunities for artistic expression over those offered by more traditional computer-generated art. However, as the interface technology improves, we expect a significant increase in the use of VE for art. National defense is not treated separately because it includes functions and tasks that are represented in many of the other application domains discussed. We do, however, briefly mention certain facets of these areas because of their important roles in technology development and implementation.
Progress in the entertainment industry regarding virtual environments is of interest because its contribution to the field will probably generalize to other application areas. Moreover, entertainment can be expected to be a primary driver and test bed for certain aspects of the technology. According to the popular press, since 1992, many joint ventures have been created among video game companies, computer graphics companies, motion picture studios, and telecommunication conglomerates to use VE technology as a new medium for entertainment, education, and artistic expression. Although several of these ventures are
occurring in the United States, large programs are also under development in Japan and Europe (Fisher, 1993). To date, the entertainment industry's efforts in VE have been proceeding on several fronts, ranging from low-end systems for home use to arcade games, location-based entertainment, and theme parks.
At the low end of the technology, several companies, including Sony, Reflection Technology, Olympus, and Sega, are developing inexpensive VE displays for use in the home with interactive, three-dimensional games. In arcades, VE action games involving one or more players are now appearing. Those currently in existence offer motion platforms and realistic interaction, but the visual quality remains poor. Location-based entertainment differs from arcade games in that it provides several interactive systems on a common theme. Such systems usually involve several players sharing a virtual space over a local area network. In the near future, Paramount Communication will be introducing a location-based Star Trek game in which players enter the bridge of a starship, become one of the characters, and interact with other characters. As the industry moves from passive viewing and interactive two-dimensional environments to the realism of three-dimensional environments in which multiple participants actively play in a fantasy world, serious questions are raised about the effects of the content of these worlds on human behavior. Of particular concern is the use of such worlds to depict violence and sex; both of these areas have been heavily hyped in newspapers and magazines.
VE also offers a new medium for artistic expression that has only begun to be explored (see Loeffler and Anderson, 1994; Leonardo, 1994). A number of individuals are designing highly imaginative video games. A few artists have produced special VE experiences, and some programs are beginning to emerge that encourage artists to create VE art pieces; however, these efforts are in their infancy. Although VE has the potential to support the creation of a wide variety of aesthetic experiences, it is too early in the development process to know how the technology will influence the techniques, effects, genres, and content that will emerge to define the medium. In the case of film, another artistic medium, the process of discovering new techniques (e.g., cuts, pans, fades, slow motion, close-ups and telephoto shots) and using them to create new experiences for the viewing public has taken many decades to evolve.
A few organizations have developed VE art programs or exhibitions over the last few years. The Banff Centre for the Arts in Canada has had an ongoing program allowing artists to create VE art pieces since 1991, and about a dozen pieces have been created so far (Moser, 1991). The annual exhibition Ars Electronica in Linz, Austria, has included VE art pieces in recent years (Hattinger et al., 1990). Since 1991 the annual
SIGGRAPH computer graphics conference has sponsored an exhibition including VE demos and art pieces (Lineham, 1993).
Currently, most efforts are directed toward building the hardware and software that can effectively generate realistic VEs. As adequate equipment becomes more widely available and affordable, we expect to see an increasing number of artists exploring its potential. Although it is difficult to make reliable predictions, it seems fairly obvious that future VE art will make extensive use of observer participation and interactivity of various kinds. Precisely how these features, as well as the other special features of VE, are used to create truly artistic experiences, however, remains to be seen. In order to encourage the use of VE for purely artistic purposes, it will undoubtedly be necessary for the art world to create appropriate supporting attitudes, programs, and funding opportunities.
Finally, as noted above, national defense is not treated as a separate application area because its scope intersects with functions in all the applications discussed in Chapter 12. For example, information visualization and distributed collaboration are critical to strategic and tactical engagement planning; hazardous operations relates to the use of technology in handling unsafe materials or remotely operating vehicles in hostile environments; and telemedicine and the eventual promise of remote surgery are important to the rapid provision of medical support to soldiers on the battlefield or to those located in isolated or inaccessible locations. The two most relevant applications to military concerns may be training and design and manufacturing.
According to a recent draft strategic plan (Thorpe, 1993), the most ambitious application of SE technology in the military is the integration, management, development, and acquisition of new systems such as tanks or aircraft. In this application, the technology will be used to model and test alternative systems in a variety of synthetic exercises. The most promising configuration will then be designed by a computer-aided design (CAD) system, manufactured in a virtual factory, and tested in an SE. Once the appropriate technology is available, the entire system can be selected, designed (with manufacturing specifications, time, and cost included) and tested without the need for physical prototype development. Moreover, individuals and units can receive training on the new system using virtual war games before the system has been produced. The discussion of manufacturing in Chapter 12 uses the design of an aircraft to describe many of the processes involved in the acquisition of a new system.
In the area of training, the Department of Defense (DoD) is currently using VE technology to cover a range of instructional experiences, from those of the individual soldier and small team to theater-level synthetic battlefields in which more than 10,000 participants interact in real time
over distributed networks. For all levels of training, VEs can be used to safely provide exercises in hostile and dangerous environments. Moreover, in the future, computers will have the capability to generate VEs as replacements for and extensions of traditional equipment-bound simulators. A critical issue discussed in the training application section of this chapter is the problem of conducting evaluation studies that demonstrate the impact of one method or another on training success as defined by how well the knowledge and skills acquired in training transfer to performance on the job.
Of particular interest to DoD is the development of networking hardware and software for large-scale distributed training. The first large network simulation of realistic battlefield engagements was SIMNET, developed by DoD's Advanced Research Projects Agency. In SIMNET, as many as 300 soldiers in tanks and aircraft simulators located at different military bases can engage in a realistic battle against an intelligent enemy on a common battlefield. Each participant in the battle views the portion of the terrain and the action that would be visible to him if he were present. As a battle unfolds, the scene changes in real time. Once a battle is completed, it can be replayed as an after-action briefing in which trainees can zoom in on various portions of the engagement or take the perspective of any tank or aircraft.
More recently, DoD has used its newest software—Distributed Interactive Simulation (DIS)—to develop a detailed, true reconstruction of the 73 Eastings battle that occurred during the Persian Gulf war. This fully interactive simulation is based on the events in an actual battle; as a result, it can be used as a benchmark to examine what-if training scenarios when enemy or friendly weapon system capabilities are changed. The projections are that DIS will provide many of the networking capabilities needed by the military for both training and system acquisition.
The final chapter of this report provides a more detailed description of the use of SE technology for each of the application areas selected for review.