Human Factors in the Design of Tactical Display Systems for the Individual Soldier (1995)

This chapter addresses the ergonomic, sensory, perceptual, and cognitive attributes of helmet-mounted (or head-mounted) displays. The discussion includes consideration of how these devices may enhance or detract from the attention, situation awareness, and workload of infantry soldiers. This discussion complements that of the preceding chapters, which considered the military environment in which the devices will be used and the capabilities and limitations of the soldiers who will use them. This chapter reinforces the idea that helmet-mounted display design features must be matched to the requirements of specific soldier tasks and environments.

The working environment of dismounted infantry soldiers is arguably the most dangerous of any current profession. The Land Warrior System aims to increase both the effectiveness and the survivability of infantry soldiers, using technologies that include portable computers, satellite navigation, light amplifying and thermal sensors, and both helmet-mounted and hand-held displays. Although such technologies can certainly enhance performance under certain conditions, they bring costs and risks as well. The pros and cons of each innovation must be considered in balance, to avoid net reductions in safety and capability. Evaluations must be informed by real or simulated field research; they should not be based solely on analyses of human abilities in laboratory situations.

Field research to test the effectiveness of this equipment has only recently begun. To be effective, the research must be directed toward those conditions under which the net benefits from specific sensory enhancements are questionable. Based on an introduction to the relevant literature, this chapter summarizes what planners need to know in order to assess the benefits and costs of the major proposed enhancements. We believe that carefully designed field testing, guided by the kinds of human factors issues that are raised here, together with the concerns expressed by those who use the equipment, will be needed continually as this program evolves. In Phase II we plan to explore the possibility that trade-offs other than design alternatives might mitigate the risks associated with the equipment as it is presently proposed. These trade-offs can include utilization concepts, personnel selection based on individual differences, and training.

In the Land Warrior System (LWS), head-mounted displays are to serve several functions: output and control of the thermal weapon sight; reception of navigation information, such as maps and current location from the global positioning satellite (GPS) system; and various command and control functions, such as messages regarding danger and troop movements. Some soldiers may be equipped with hand-held displays or palm-top computers, but it is the helmet-mounted display, in its several functions, that raises most of the issues of concern in this chapter.

The display in the Land Warrior System is initially to be an opaque cathode ray tube (CRT) screen, with about 40 degrees of visual field (Snow, 1994), positioned about 1 inch from the wearer's eye. The night vision system is the most dramatic enhancement in that it offers some information about a wearer's night environment. In low light level conditions, the recognition of objects and their spatial layout will depend directly on the display parameters. Because the equipment presents an additional encumbrance, and while its opaque CRT screen necessarily restricts day and night vision if it is left in place, daytime use of the display might, in practice, prove more problematic.

Prominent among the concerns that call for further research and testing are both long-and short-term effects of the protracted visual rivalry that results from the use of monocular and biocular displays. It has been suggested that later versions of the display screens may be transparent (Computer Science Corporation, 1993): transparent displays are now used in some combat aircraft as alternative to a head-down instrument panel, in part to eliminate the time a user would otherwise need to perform the oculomotor actions (saccadic glances, vergence movements, accommodation changes) needed to take in multiple sources of information. This is accomplished through the head-mounted display's use of a combiner that integrates the CRT information with the outside view. These displays offer collimated light, so that a user's eyes need not change focus (which can take as long as 2,000 milliseconds, while saccadic glances require only some 250 milliseconds) when switching attention between the outside scene and the display symbology. Where used in an aircraft or other transport, some of the symbology can be made conformal with various aspects of the external world, achieving “proximity compatibility ” (Wickens and Andre, 1990) of the symbology and the real-world visual information. If a display is to be mounted on a soldier's helmet, achieving conformality with the world would require tracking the position of the helmet and slaving the display to a stable world reference.

The Army has been a pioneer in the application of helmet-mounted displays in aviation, with various night vision devices and sensors fed into the Integrated Helmet and Display Sighting System (IHADSS), and it has developed an extensive body of research data on visual performance with sensor displays (Foyle and Kaiser, 1991; Bennett, et al., 1988; O'Donnell et al., 1988), human factors and safety problems (Brickner, 1989; Rush et al., 1990), and field experience with visual illusions (Crowley, 1991). Data from the aviation

experiences demonstrate both the great potential and the risks of using helmet-mounted displays.

The use of head-up displays and helmet-mounted displays in aircraft offers only an introduction to their use by the dismounted infantry soldier. The actual situations are very different. Conformality will be very important for a successful see-through head-up display; the assumption that it facilitates parallel processing of information from instruments and from the world merely through collimation and superimposition does not appear to be supported (Weintraub and Ensing, 1992). This may be because visual attention is allocated to perceptual groups, like objects and surfaces, and not just to locations within the optic array (see Neisser and Becklin, 1975). Because attention to one location in depth (as given by binocular disparity) interferes with the detection of targets at other depths, especially for shapes that share overlapping retinal locations (Hoffman and Meuller, 1994), and because where the symbology is nonconformal it may provide “ visual clutter” that competes with the directly viewed environment on which it is superimposed, presentation of information on the head-up display may degrade performance (Fisher et al., 1980; Wickens, Martin-Emerson, and Larish, 1993; Wickens and Long, 1994). Except in special and relatively simple cases like the output of a weapon sight, it is not clear how it would be possible to achieve such conformality for infantry soldiers. It has not been established, moreover, that an adequately detailed electronic map can be presented in the equipment as it is currently envisioned, or that a soldier can read such a map faster and retain more information with that technology than could be achieved with a paper map.

In the Land Warrior System, the helmet-mounted display is proposed as a monocular device. This supposedly leaves the other eye unobstructed, but visual rivalry will then occur between the two eyes; this rivalry should be particularly intense if the display is opaque. Such rivalry can remove targets from one eye or the other (through monocular suppression) and may produce serious visual disabilities in the user (see below). Moreover, the occlusion of one eye's view of the environment entails the loss of stereoscopic depth perception, which is an especially potent mechanism for “seeing through” camouflage (by correctly assigning the different depths to the camouflaged surface and its surround), for manipulating equipment, for avoiding collision with nearby trees and walls, and for other tasks. Such losses are likely to impose important costs on infantry soldiers in many situations. It should be pointed out, however, that there are very often redundant cues for breaking camouflage.

For example, the Land Warrior System may allow a squad to operate in a more dispersed fashion that reduces the squad's vulnerability. However, if all squad members are wearing and using helmet-mounted displays, the Land Warrior System squad's ability to detect a camouflaged ambush site in some terrains during the daytime probably would be compounded because of the lack of depth perception. Target detection problems would be compounded at night because the squad will be able to move faster than under conditions without the Land Warrior System. Thus, overreliance on the visual system and speed of movement could reduce squad attentiveness to other cues. If an ambush occurs, the squad's speed of execution of the counter ambush drill would be reduced because of the time needed

to orient to the enemy and because of the narrow field of view. Although thermal imagery can defeat visual camouflage for the most part, the problem with depth perception remains.

For the presentation of symbolic data in the Land Warrior System, for some tasks it might be better to place the helmet-mounted display screen off the visual axis, or use a handheld device, which would require the wearer to shift gaze in order to access the information on the screen but might more than balance this cost by reducing clutter and rivalry and by restoring stereopsis. In other cases, presentation on the visual axis might be the best approach.

The introduction of devices that provide remote or local information in the form of enhanced sensory or symbolic displays may in the proper circumstances contribute greatly to the safety and effectiveness of the infantryman. However, the specific means proposed in each case may interfere with the acquisition or use of sensory information, depending on the circumstances, and all such devices are associated with certain general problems. We introduce such problems briefly here and discuss them in more detail in the next section.

First, helmet-mounted displays might degrade or even nullify information about the nearby environment that is normally available through the unaided senses; see report on Soldier Integrated Protective Ensemble (SIPE), (U.S. Department of the Army, 1993). They may distract attention in ways that may have a critical effect on some tasks and they may interfere with a user's situational awareness. Even design factors that might be unimportant under less demanding conditions may seriously contribute to a soldier's workload under combat conditions.

For example, the SIPE squad and team leaders reported to the panel a situation in which they were unable to see an ambush target although the target presented itself on multiple occasions. The squad positioned itself further from the kill zone because they felt secure in their ability to observe the site. The attention distraction phenomenon offers one possible explanation of why the squad was unable to detect the target: the squad reported diligently observing the kill zone, which meant that they were focused on an area. If the target passed outside of the area of narrowed attention, they might never have noticed it, even though it was in the field of view.

Second, through excessive fragility, bulk, and weight, the equipment by which remote or sensor information is displayed may seriously reduce the mobility of the dismounted infantryman, and it may add fatigue and heat stress (U.S. Department of the Army, 1993). In aviation helmet-mounted display equipment, the heavy and off-center optics increase fatigue and headaches, and the close-fitting helmet liners used to retain optics in position may also increase heat stress (see Chapter 1). In the infantry, these physical problems must be an

even greater cause for concern: greater fatigue can be expected because active infantry soldiers do not receive support from a seat, and the equipment may interfere with their ability to move and to take cover rapidly.

The physical effects of the equipment may have perceptual and cognitive consequences as well. Spatial disorientation can be expected; with no external support, such as a seat, a soldier is not provided with tactile information about bodily orientation to help counteract any disequilibrium due to the helmet-mounted display. Because the weight, weight distribution, and configuration of some displays interfere with the free head movements that a soldier would otherwise rely on to obtain the visual information that is intimately tied to normal action and locomotion in the environment, the equipment-based deficits offered to the infantry would seem to be considerably more serious than those offered in aviation.

These two sets of issues, sensory/perceptual and ergonomic, are not only problems in themselves, but may interact in counterproductive ways. Such factors, and others that are beyond this phase of the panel's work, must be kept in mind by both the equipment designers and users. Cost-benefit analyses after appropriate testing--tests whose results apply to the situations in which the equipment is to be used--should precede commitment to any mode of augmentation and to the means by which it is achieved. Appendix A summarizes the major benefits and costs of key factors of helmet-mounted displays and the key research and testing issues. The next section considers various aspects of the human visual system in relation to displays. The subsequent shorter sections consider situation awareness and workload issues.

In connection with night vision in helicopters, Weintraub and Ensing (1992) note that, simply considered, some visual information is obviously better than none. However, that point is less clear-cut if that visual information is misleading; if its correct interpretation requires soldiers with higher mental capability, more training, and better concentration than they have; and if interpretation takes more time than can be spared under the conditions of use. Such potential limitations on information use must be considered in relation to specific tasks.

An example of difficulty in interpretation and perception was the performance of TOW weapon system gunners employing the first thermal night sight. The TOW missile system had been fielded for several years prior to the development of a thermal night sight. When the night sight was added, it clearly enhanced the capabilities and potential of the TOW system; however, there were significant problems training gunners to detect and identify threat targets at ranges of 1,000 meters and beyond. The night sight was capable of detecting the heat difference at extended ranges, but soldiers had great difficulty consistently detecting and then identifying friendly versus enemy targets. Once detected, the probability

of a hit was high, but soldiers could not be easily trained to effectively detect targets and avoid fratricide under varying battlefield conditions. This training problem was not fully resolved until the quality of the thermal image was improved, and soldiers gained confidence in their ability to discriminate images in the sight.

Visual displays provide layout information through the patterning of light and dark (and color, where applicable) on the surfaces that they present to one eye (monocular), to both eyes (biocular), or by different displays to each eye (binocular). For this discussion, a display with high fidelity is one that provides much the same effective patterning to the eye as would the layout or environment itself when viewed under good viewing conditions. Task-independent definitions of fidelity are not now available. When fidelity is too low for a specific task to be performed, the display may be useless or even harmful. Because increases in fidelity will in the short term entail increased monetary cost, fragility, weight, interference with mobility, and other costs, it is important to achieve an understanding of the fidelity needs of different tasks.

Display designs are usually discussed in terms of the displays' sensory properties, that is, in terms of the attributes that the displays can offer to the eye and in terms of which their fidelity limits can be assessed. These sensory properties are readily measured. However, they do not by themselves provide information on whether the display allows the viewer adequate perceptual knowledge of the objects and layout being confronted and adequate situation awareness of the environment in which the actions are to be taken.

Augmentation displays are all vastly impoverished in comparison with the light that reaches the unaided eye from a natural environment under good viewing conditions. The size of the display, or field of view is always far smaller than the natural field of view. Table 3-1 compares the horizontal field of view of the unaided human eye with that of several visual input devices.

A display's resolution (that is, the number of dots or stripes per degree of visual angle that can be discriminated) is virtually always less than the normal eye's highest resolution. The display's contrast (the ratio of its darkest and brightest regions) and the number of intermediate levels (if there are any) between those extremes are far less than the eye normally handles. Approximate guidelines for reading, vernier and stereo acuity, contrast thresholds, and other sensory tasks provide a starting point against which to evaluate the equipment (see Boff and Lincoln, 1988; LWS documents). For reasons considered below, however, those guidelines are not enough. Tests of the features that permit adequate visual performance under actual or simulated field conditions are needed prior to final design decisions.

The significance of these sensory factors depends on the perceptual and cognitive tasks that a viewer is to perform. At the very least, a display must support the visuomotor actions needed to look at it and retrieve information from it. For example, the display must provide enough contrast to focus the eye on the plane of the display (accommodation) and enough coherence (low enough “snow” or noise level) to allow purposeful eye movements aimed at some peripherally distinguishable feature. Although such behaviors are largely elective, their coordinated use has been extremely well practiced in normal environments, as have relationships between the two eyes and between the eyes and the movements head, body, and limbs.

A person's two eyes' lines of sight are normally converged to the angle that matches their accommodation distance, so that each point at that distance in the world falls on corresponding places in the two eyes and is seen as a single point in binocular vision. Each display interferes in some way with the normal process. In monocular displays one eye is augmented, and the other is unoccluded. In biocular displays both eyes receive the same augmented view. In binocular displays the two eye's views are disparate so as to provide binocular parallax or stereoscopic depth information, obtained from sensors of fixed vergence.

Every display system interferes with visual performance. Monocular displays necessarily provide rivalry, an alternating whole-field or piecemeal blindness, which only some viewers can tolerate without sickness and which is not relieved by practice with the device. Largely irremediable low-level binocular rivalry intermittently obscures each eye's

view, either by piecemeal or by total suppression. Brown et al. (1978) found that after 8 days of monocular occlusion, subjects showed large changes in phoria, severe diplopia, and failed all tests of stereopsis; after 4 hours of occlusion, much smaller and more transient effects are found (Sethi, 1986).

Both monocular and biocular displays deprive viewers of stereoscopic depth information; all three displays use collimated light, which remains in focus regardless of accommodation. These conditions tend to keep the human accommodation and vergence system running open-loop, resulting (with sustained use) in eyestrain, fatigue, and possibly disorientation (Ebenholtz, 1988). Although soldiers may be able to adapt to such a system there may be both short- and long-term costs.

This disorientation could be a significant battlefield detractor, rather than a combat multiplier. Infantry School personnel reported to the panel that a number of soldiers had difficulty using the monocular night vision devices. These problems included vomiting, temporary blindness in the unstimulated eye, and temporary total blindness. Visual rivalry is a major contributing factor and can only be reduced by using the system in an intermittent fashion or by designing a fully synthetic environment (virtual reality). Little is known about the distribution of individual susceptibility to visual rivalry or the long-term health hazards that might be associated with prolonged use. Intermittent short-term use may be a solution, but guidelines such as those developed for army aviation are needed.

The range of situations where this visual rivalry can occur is large. Examples include the Apache pilots, who have had to “adjust” when flying at night with night vision devices (IHADSS), and stationary infantrymen in a prepared fighting position. The situation becomes more complex as it becomes more dynamic (e.g., moving targets and moving soldiers). Although the speed at which the infantrymen move is relatively slow, the current helmet is not a stable platform, and the rotary moments of inertia are high; this results in image destabilization. The implications for helmet design are significant.

Even without eyestrain, fatigue, and disorientation, limited display resources means limited information transfer, depending on the task. For example, where field of view, resolution, and gray scale are reduced, fewer different patterns and less information can be displayed, whether the display is an electronically transferred image of the optic array that is faced by a sensor mounted on the helmet or presents maps and computer-generated graphics. This means that a viewer can differentiate and recognize fewer shapes (insignia, equipment, landmarks, etc.) than with normal vision. It means that the static pictorial depth cues seen through the display (the depth cues are basically shapes, revealing interposition, perspective, etc.) are less effective, and so are the behaviors which depend on them. Reduced field of view also means that: (1) there are fewer objects or parts seen from any position of the head when the wearer is surveying some part of the environment and that there is less context to provide meaning for any detail or object that is in fact seen; (2) a wearer receives less of the ambient or peripheral vision that is important for orientation in the environment; (3) there is less scope for surveying the environment by making fast and economical eye movements (see

Table 3-1) while holding a given head position; and (4) there is more need for head movements, which are relatively slow and cumbersome (especially when wearing helmet-mounted displays and associated sensors).

A perfect system, without any of these limitations, is not currently available for the dismounted infantry; an acceptable trade-off must match the available system to the users' demand characteristics, given the tasks the users must perform. Table 3-1 (above) presents a summary of trade-offs for visual displays within different categories of visual tasks.

Because the sensory characteristics of the displays (resolution, gray scale, etc.) are not related in any simple way to the perceptual tasks in which the displays are to be used, we outline the information needed for classes of tasks more generally. We do so first in terms of perceiving what is present in the visual environment and then in terms of situation awareness.

Visual helmet-mounted displays, and visual displays generally, communicate in at least two different ways: they may present two-dimensional patterns that have meaning for the wearer and can guide behavior with no need for the viewer to perceive a three-dimensional world from that pattern; and they may present two-dimensional patterns that have no meaning in themselves but that act as cues to the viewer 's perception of depth, objects, and surfaces in a three-dimensional world. In most actions in the world, it is those three-dimensional perceptions that guide behavior. Although these two channels of information are usually both used in helmet-mounted displays, they are treated differently by virtually all displays, so we consider them separately.

The laser range finder can provide information about the distance of some object in the field, substituting for at least some of the functions served by the visual depth cues, and often surpassing them in accuracy and precision. But the range finder cannot replace the depth cues in providing an integrated perceptual grasp of size, shape, and layout. An attempt to use range to determine some object's size, for example, would require deliberate calculations based on the measured distance and the measured size of the object's image in the display, a time-consuming and error-prone procedure compared with humans' very rapid normal perceptual grasp of size and distance. Similarly, a buddy system that uses radio communication to provide triangulation from two viewpoints can offer distance information that may draw on trigonometric calculations to supplement the visual depth cues where the latter are inadequate (or have been defeated by the helmet-mounted display), but it cannot substitute for the rapid and intuitive grasp of the three-dimensional environment that the

unfettered depth cues afford. The form in which people normally obtain and use the information that underlies human perception of any situation cannot in general be altered without substantial cost in time, error, and situation awareness.

Some tasks require little more visually than that the viewer detect a particular two-dimensional pattern (or classes of two-dimensional patterns) on the display's surface or in the collimated array that it presents to the eye. For example, judging whether an infrared marker in the field of view falls on the pattern projected by a rifle 's target requires only that the user judge whether or not two patches of light coincide on the display. For this task, the equipment used in the SIPE project received favorable testimony from its users (U.S. Department of the Army, 1993). Success on such tasks can probably be closely predicted by existing data on what resolution and contrast are needed to detect contours and points. Similarly, the detection and reading of graphic symbols requires only that the display's contrast, resolution, and signal-to-noise ratio fit what is known about legibility (Helander and Rupp, 1984; Grandjean, 1987; Human Factors Society, 1988). There are probably many tasks that can be reduced to similar visual questions about the display's surface. The substantial psychophysical research literature on visual detection and research on computer and video displays (Helander and Rupp, 1984; Grandjean, 1987; Human Factors Society, 1988) can provide good bases for design trade-offs. The possibility of creating tables based on the human factors literature will be explored in Phase II.

However, most of the visual tasks that are normally required of infantry soldiers do not depend on information about what contours and points can be distinguished on the surface of a display. They depend on recognizing which three-dimensional objects in the environment need action, on perceiving their three-dimensional spatial relationships to each other and to the viewer, and on perceiving the three-dimensional environment in which the actions are to be performed. Those perceptions depend, in turn, on depth cues, which are patterns of light either projected to the eye by the environment or transmitted through the helmet-mounted display. Even where the display shows only maps or diagrams, which can in principle be provided as two-dimensional patterns and evaluated as such, the literature suggests that the information about layout and location is still best conveyed by the same kind of depth cues that are used in pictures of three-dimensional space (Bemis et al., 1988; Burnett and Barfield, 1991).

For visually guided behaviors in three-dimensional space (e.g., advancing, aiming, reconnoitering or scanning the environment, combat, manipulating equipment, etc.) a viewer can normally draw on various kinds of depth information available to unobstructed vision. Chief among these are pictorial depth cues, motion dependent cues, and cues that involve the adjustment of the ocular system.

Pictorial depth cues do not rely on moving pictures. They are essentially patterns in a two-dimensional array of light that the world projects to the eye, patterns that are two-dimensional themselves but usually arise from differences in depth. That is, the various cues are aspects or features of the pattern of light that are likely to be provided both by a three-dimensional scene (layout of objects and surfaces) and by the protective picture of that scene as made by a camera, by an artist, or by a computer. Thus, because the visual angle subtended by an object decreases with its distance, one has linear perspective (within the display, sizes perpendicular to the line of sight decreases as distance increases) and the related cues of relative size (the relative distance of two objects in the world are inversely related to the sizes of their images in the display) and textural density (as distance in the world increases, the density of textural detail in the image of a homogeneous surface increases). Height in field is a powerful depth cue where viewer and target object rest on the same plane (the farther the target, the higher toward the horizon line). Interposition is a strong cue wherever the images of two objects overlap (where two contours form a “t,” the uninterrupted one is probably the nearer).

Other important depth cues result from motion parallax, in which characteristic patterns of motions in the two-dimensional array normally arise from the relative motion between viewer and environment (motion perspective, the optical expansion pattern, etc.). Finally, there are the depth cues that depend on the adjustments of ocular musculature: stereopsis and accommodation. In stereopsis, or binocular disparity, any point that lies exactly at the distance at which the two eyes are (voluntarily) converged falls on corresponding points in the two eyes; any object or point either more or less distant than that plane projects disparate images to the two eyes. Those disparities, taken with the eyes' vergence, are particularly powerful depth cues for relatively nearby distances. Finally, the eye muscles increase the lenses' curvature to focus on nearer objects; such muscular action, or accommodation, is a depth cue within very close distances.

Quite different two-dimensional patterns can thus act as cues to the same three-dimensional situation, providing the same perceived depth. It is normally the perceived three-dimensional situation that determines action and decision, where the viewing conditions permit. Even though they are used to provide information that may be missing from the normal field of view (with dark of night being the extreme case), helmet-mounted displays will degrade or even destroy these depth cues; depending on their design, they will do so to a different extent in different ways.

For example, where helmet-mounted displays transmit light originating in the scene, collimation places all points at infinity as far as accommodation is concerned. Resolution, gray scale, and field of view of the transmitted or reconstructed image are always impaired relative to unaided vision; this degrades or eliminates those cues that depend on detail (like textural gradients), on gradations of shade (like modeling), or on expanse (like linear perspective).

Aside from vergence and accommodation, all of the other cues used for depth perception can be generated by computers. In principle, therefore (although it is not contemplated in the programs considered here), helmet-mounted displays might restore or enhance depth information, might use depth cues to provide simulated environments, or, much more modestly, might use depth cues to enhance the separation of different sets of alphanumeric or graphical data.

The effects of degrading or enhancing depth cues, and the trade-offs involved, can probably be estimated. To do so, one would need to assemble a set of matrices around the four variables: (1) which distances are most important for particular tasks, (2) the distance ranges over which each class of depth cues is effective, (3) which depth cues are offered by particular environments, and (4) the effects of different display properties on the different depth cues. Item (2) has already been devised by Cutting and his colleagues (Cutting and Vishtow, 1955); the others can probably be approximated. Such a matrix may be assembled in Phase II--although it would necessarily be partly speculative, since at present far less is known in an engineering sense about the how the physical properties of a display affect the effectiveness of the depth cues than is known about the display of two-dimensional patterns. It would, however, suggest on a principled basis what research and testing are needed for different tasks and conditions. A discussion of a few examples here illustrate the relationship between these variables and their importance to the design and use of helmet-mounted displays.

First, tasks differ in the ranges of depth information they require. Information about nearby depth (i.e., within 3 meters) is needed in reaching for tools or weapons; for avoiding collision with obstacles (a nearby doorjamb, tree trunk, etc.); and for touching pen to map. Information about whether a moving figure is about to disappear behind, or will remain in front of, a house or wall involves intermediate distance (say, from 5 meters to 3 kilometers); the same question about a column of moving vehicles and a rise in terrain involves far distances (say, above 5 kilometers). Distinguishing a camouflaged object from its background at any of these distances requires depth information appropriate to the object's scale and distance. If a helmet-mounted display design degrades or eliminates the depth cues that normally provide the information for depth perception, as most such devices do, performance of any tasks that need such information will take longer and be less accurate.

Second, depth cues differ in the distances they reveal. Nearby depth is normally provided in unfettered vision by binocular stereopsis, by the visual parallax that arises from head motion, and to a lesser extent by accommodation. When two objects overlap in the field of view, interposition tells which is the nearer, but not by how much (i.e., it gives ordinal information, not interval information as do other cues). Intermediate depth is normally provided by the visual parallax that arises from body motion and head motion, by linear perspective and its related depth cues (e.g., relative size, textural gradients etc.), by height in the field, and by interposition. Far depth relies mostly on height in field (which is readily confused by differences in terrain), by relative size, and by interposition when it

occurs in a detectable way. For a tabulation of these different cues to different depth differences, see Cutting and Vishton (1955).

Third, environments differ in the depth cues they afford. Featureless plains offer no textural information, uneven terrain negates height in field as a depth cue, and tangled occluding jungle growth has both effects (but demands information about nearby depth, which the other two may not). One need not worry that a helmet-mounted display might interfere with normal use of some particular set of depth cues where the environment is not likely to offer them. In fact, if the information provided by those depth cues is needed, one should try to furnish the information in some other way (as by night-vision equipment or by signals from other observers or satellites).

Helmet-mounted displays, like night vision devices, capture optical information about the environment and present it visually to the wearer. If binocular sensors are used, stereopsis may be enhanced (if somewhat distorted) by increasing the separation between them, extending depth information into intermediate distances. If only a single sensor is used, stereopsis is necessarily lost with monocular or biocular head-mounted displays, and accommodation loses all depth discrimination in all displays because the light is collimated. Head-motion parallax is seriously distorted whenever the sensors are at a different optical location than the eyes (as in virtually all head-mounted displays) and is lost when a viewer must remain stationary. Without stereopsis or parallax, a viewer is left only with interposition for nearby depth information, and that information is severely limited; objects' images must be overlapping to provide it, and at most it tells only which is the nearer.

Because a viewer who must remain stationary but who is concerned more with intermediate and far distances than with near distances must depend chiefly on the pictorial depth cues, the loss of stereopsis and the distortion of head-motion parallax that is imposed by most helmet-mounted displays may not represent significant additional costs. That can only be true, however, if the equipment provides the depth cues in a condition that is adequate to the perceptual needs of the task. Such devices are not equal to what the unaided eye receives under good viewing conditions, but more graded assessments are needed for making design decisions, particularly under degraded viewing conditions.

There are data that offer some guidance as to the equipment needed for stereopsis (in the case of binocular displays) and parallax, obtained as functions of luminance, contrast, and resolution, using very simple displays involving wires, dots, and gratings (Schor, 1987; Schor et al., 1984, Foley, 1987). The pictorial depth cues, however, are another matter. At the most basic level--as two-dimensional patterns--there are various data showing that the

exposure time and contrast needed to detect targets vary with their size, background luminance, etc. These might be applied to features of the depth cues that seem amenable to such analysis.

For example, textural gradient and local occlusion may be presented to the sensor by the environment, but they are lost at low resolutions; shading cues and the intersections that provide interposition information are degraded or lost with a sparse gray scale; height-in-field and convergent linear perspective, both of which depend on some extended region of the display, must at some point be degraded as field of view decreases, as aperture-viewing studies confirm (Hart and Brickner, 1987). The effects on these attributes of any display must therefore be assessed before deciding to use the equipment in any task that is likely to call on those cues, and effort should be made to relate, extend, test, and apply the results of such task-oriented analyses.

The same issue arises in the context of computer-generated graphics. One function of the display equipment (including hand-held displays, which do not directly interfere with normal visual perception of the world) is to present maps, diagrams, and charts. Experimental evidence suggests that maps and diagrams that incorporate depth cues may be more effective than traditional displays (see, e.g., Bemis et al., 1988; Burnett and Barfield, 1991), depending on the task and on the parameters and combinations of cues (Ellis et al., 1991; McGreevy and Ellis, 1986)). The pictorial depth cues (and stereopsis and motion-based depth information as well) can in principle be successfully constructed on computer graphics displays, but they can only be used by a viewer to the degree that the sensory qualities of the display permit. The enhancement and simplification techniques that are available to computer-generated images can provide more robust information, but limitations like resolution, contrast, and field of view in field devices must still be evaluated as to their effects on depth cues from these artificial sources.

Most familiar classes of objects can be recognized with extreme rapidity even in the absence of any depth information, cued only by their shapes in the display (Biederman, 1985; Peterson et al., 1991). Performance tasks that do not require any specific depth perception--but that can be carried out by recognizing some object(s) or layout (e.g., the presence or direction of a person or group, of a particular kind of equipment, a particular house, etc.)--is therefore probably not badly degraded by the absence of depth information. Moreover, objects ' familiar sizes can actually act as depth cues, although such distance information takes longer to extract (Predebon, 1992). Like the depth cues, however, the perception of the objects as shapes necessarily depends on the quality of the display, the field of view, and how the situation prepares or primes the viewer.

In a display that is too coarse to resolve the features that characterize a given object, or with gray scale and contrast inadequate to model its forms, shapes may not be recognizable. Such effects cannot be predicted solely from any table of data, however, because objects (and depth cues, as well) are normally highly redundant. That is, a part or feature (or even just an attribute of some object, like its color) may serve instead of the

whole object, given an appropriate past or present context (Hochberg, 1980). With training and familiarity therefore, performance may surpass what would be expected from such tables. Conversely, a given point of view may obscure the relevant features even where display quality is otherwise adequate. As a consequence, trade-off assumptions about any specific equipment need be tested in real or simulated missions for which it is intended.

With a small field of view, some or all of the redundancy in an object or scene is very likely to be lost, because only some portion of either may be included within the display. Moreover, peripheral vision is greatly decreased by the field of view for most displays proposed. This will interfere with object recognition because the information in peripheral vision normally informs a viewer where to look next in order to obtain some needed feature. Peripheral vision also provides landmarks as to where some detail that was previously fixated (i.e., was clearly seen in central vision during a previous glance) lay relative to the feature presently being fixated, and these landmarks are likely to be unavailable within a small field of view. A viewer can compensate to some extent for a small field of view by making more head movements to sample the environment. But successive small glimpses of the environment that are obtained by such movements (which are much slower and more cumbersome than eye movements, especially with heavy helmet-mounted displays in place) can provide information about the entire object, or scene, only if they are effectively stitched together in memory, which is not necessarily possible in cluttered or unfamiliar settings.

There is presently no single accepted cognitive theory from which one can set the bounds of an individual glimpse. For example, are successive views of a display “directly” placed by the visual system into a single perceptual setting, without passing through any memory-like encoding process, just so long as there is enough structural overlap between the views? If so, which seems doubtful (Hochberg and Brooks, in press), how much overlap must there be? Over how much delay? Over how many shifts of view? Regardless of theory, it is known that reduced fields of view reduce a viewer's ability to grasp things and to maneuver within the visual environment.

According to a large body of research (and common sense), objects and scenes can be identified more rapidly and more correctly when a viewer has been previously set or primed by those objects or by the categories to which they belong (Biederman, 1985; Bachman and Alik, 1976). The context in which an object appears, if it is an appropriate context, can serve much the same function (Biederman, 1981). Reduced fields of view eliminate or reduce the context and thereby its facilitating effects.

There may be some minimum field of view below which a wearer will be unable to achieve a coherent grasp of the context by making the successive head movements discussed

above; this is suggested by aperture-viewing studies and by examining motion picture use of “establishing shots” (Hochberg, 1986). At a narrow field of view, the facilitating effects of the context on object recognition may be lost, and the context is often necessary for accurate perception.

When it is important for a viewer to have a ready grasp of where people and things are distributed within the visual environment (which must often be the case with infantry soldiers), the higher cost and weight of displays that go with wider fields of view may be unavoidable. Only controlled research under field (or field-like) conditions can inform that decision. In any case, because the sequence of visual queries (e.g., successive glances and head turnings) is elective--depending on a viewer's task, knowledge, and attention as much as on the information provided by the visual display at each step in the sequence--one must consider the situation that the viewer needs to grasp and the factors that affect such situation awareness.

A helmet-mounted display allows visual and auditory information of various kinds to be offered to a wearer, without the major change in gaze or posture that is needed to look down at a map, to raise a viewing or sighting device, or to position an earphone. However, as noted above, various aspects of the proposed electronic displays interfere with a user's ability to direct attention within the visual field, in particular, and within the inflow of information, more generally. Four aspects stand out as likely to impose high costs in this way:

• Restriction of fields of view interferes with the normal means by which one deploys attention across the visual field (Leibowitz and Dichganz, 1980; Santoro et al., 1994).

• Devices that eliminate stereopsis and that degrade the other depth cues collapse everything into one depth plane and make it difficult to use attention effectively (see above).

• Where additional visual information is added to the display through which a viewer views the world (e.g., a head-up display) by inserting alphanumeric, diagrammatic, or pictorial elements, the information load may become too great, thus reducing the user's ability to attend to the desired information.

• The potential for attentional narrowing (paying exclusive attention to one source of information at the expense of other channels of information) is a significant concern; users may dwell on compelling and complex graphics and text displays at the expense of otherwise available environmental cues.

Computer-generated images, when properly designed and used (as in conformal design and egocentric display), may require less in attentional resources (Wickens and Andre, 1990). It is hard to see how such images, which might be practical in head-up displays that are operated in vehicles like aircraft or ground carriers, could presently be

achieved for the infantry soldiers. In any case, salient and compelling graphics have their own dangers, so far as attention is concerned: they may distract attention through sheer clutter, and they may weaken a wearer's sense of presence and knowledge of information in the real environment.

Beyond the additional visual channels, the helmet-mounted display is supposed to provide auditory, including verbal, information. There is probably some attentional advantage to affording separate channels for separate tasks, if conditions permit adequate time sharing (Wickens, 1992). With such multichannel equipment, a soldier must gain awareness of the environment from many sources, and that overall awareness may be helped or hindered by the design and use of the equipment.

A helmet-mounted display could provide more timely, more detailed, and qualitatively different information to soldiers in remote field settings than is currently available, but it also may shift attention from the immediate environment toward a virtual environment in the display and may overload the soldier with data. Under information overload, soldiers may be unable to use any of the available information efficiently or may be greatly delayed in finding needed information, with a net result of significantly degraded performance.

The testing of helmet-mounted display designs needs to include the helmet-mounted display's impact not only on basic processes like perceptual accuracy, but also on soldiers' global state of knowledge when used in dynamic and complex field settings, where multiple sources of information compete for attention and must be selected, processed, and integrated in light of dynamic goal changes under differing workloads. That is, testing must include the impact of the proposed concept on users' situation awareness.

The situation awareness construct occupies a major role in human factors (Roscoe, 1980; Johnson and Roscoe, 1972) and has direct relevance for the 21st Century Land Warrior System. The reported benefits associated with achieving situation awareness include improved safety, reduced workload, enhanced performance, expanded range of operations, and improved decision making (Regal et al., 1988).

A general definition, applicable here, describes situation awareness as “the perception of the elements in the environment within a volume of time and space, the comprehension of their meaning and the projection of their status in the near future” (Endsley, 1988). The spatial component of situation awareness deals with the three-dimensional geometry of the environment, such as “Where am I?” or “Where are other objects (e.g., terrain, aircraft, threats) in relation to a person's position in three-dimensional space.” The status component of situation awareness deals with such things as the state of equipment, identity and threat level of tactical units, etc. Thus, situation awareness includes perceiving or attending to information, integrating multiple pieces of information and assessing their relevance to the task at hand, and forecasting future situation dynamics to allow for timely and effective decisions. For reviews of definitions, models, and uses of this concept (see Adams et al., in

press; Endsley, 1988, 1994, in press; Fracker, 1988; Smith and Hancock, 1994; Taylor, 1990; Taylor and Selcon, 1994; Tenney et al., 1992).

Situation awareness is restricted mainly by limits of attention and of working memory capacity, which constrain a person's ability to receive and process multiple channels of information (Endsley, in press). Attention is selective: it is directed in part by working memory (current goals, expectations, other situation information), which affects how information is perceived and how it is interpreted. Working memory is a limited resource, and therefore sets limits on situation awareness, and working memory may be further reduced under stress, as in combat (Hockey, 1986; Mandler, 1979). These limits can be circumvented by long-term memory, most likely in the form of mental models or schemata.

Mental models are complex mental representations that describe how systems (such as a combat environment with opponents and other participants) function. Prototypical situations (or states of the model) can be represented by associated schemata that provide rapid comprehension of information and projection of future events (the higher levels of situation awareness). For example, a soldier may recognize a particular combat maneuver or formation of enemy troops based on certain critical features or cues. This allows him to predict what actions the enemy will be taking and enables him to take appropriate proactive measures. These models are of great use in achieving situation awareness as they provide expectations of what (like a gun turret) is to be found where (e.g., in a particular type of terrain), facilitating the deployment of a soldier's limited attention in the environment. Proper use of schemata can significantly relieve working memory and efficiently direct perceptual processes; judicious training in such schemata, and their use in appropriately designed information displays, would obviously be desirable. As a potential cost, however, it should be remembered that strong schemata can also impair situation awareness by biasing the selection and interpretation of information.

More generally, design of helmet-mounted displays should aim to overcome the main factors that currently limit situation awareness in the particular environments being considered. For example, there may be a need for sensory enhancement by improving a wearer's ability to localize targets and self and to navigate in the environment; keeping a soldier and commander up to date on changes and situation factors in the field; sharing information between team members; distributed decision making across teams and between headquarters and teams; and allowing soldiers to look at information in different ways, thus supporting strategic decision making.

A helmet-mounted display may also help situation awareness by presenting information in a format more compatible with the soldiers' needs. Most simply, some information may be better received visually than audibly, or vice versa. As a more technically demanding example, egocentric displays are superior to eccentric displays for self-locomotion because the direction of movement in the display is then compatible with the soldier's motion in the environment (Wickens et al., 1989).

However, helmet-mounted displays may also interfere with situation awareness. By its degradation of perceptual, attentional, and working memory processes, a helmet-mounted display may cause important environmental information to be lost or misperceived. The information on a helmet-mounted display may compete with the outside world for the soldier's attention, resulting in an attentional narrowing and loss of important information in the environment. Their high perceptual salience and the location of the helmet-mounted display right in front of a wearer's eye makes computerized information displays particularly troublesome in this regard. In addition to competing for attention, the new information imposes both extra perceptual processing requirements and a load on working memory. Thus, interface design should attempt to keep to a minimum any needs to memorize commands, syntax, or other information to minimize this effect. High cognitive workload associated with the operation of the 21st Century Land Warrior System may also have a negative impact on users' situation awareness if the requirements exceed users' capacity to process information (Taylor, 1990; Adams et al., in press; Endsley, 1993). Table 3-2 lists some of the situation awareness issues that need to be considered in design trade-off studies.

A wide range of operational factors affect workload. For the present discussion, these factors are limited to those associated with the future operations of an individual infantry soldier and, particularly, the technologies that are being considered for development. It is assumed that much of the information transmitted to soldiers will be visual and graphic, given the present direction of display evolution and the facility with which humans grasp four-dimensional representations. Indeed, one current suggestion is that decision tasks be transformed into this representational medium, in which lucid presentation of information makes solutions immediately and “visually” apparent. Therefore, one major factor influencing workload will be visual demand. Differing forms of design speculation for visual displays involve multiplexing or interleaving real and virtual world representations. Added to this may be vestigial alphanumeric overlays in more standard informational forms. Such representations will present a considerable visual workload burden to soldiers and will have to be carefully managed. The multiple division of visual attention is therefore a topic of research importance, as is the growing information on fixing attention in three-dimensional space.

If displays are used to provide surrogate representations of the actual world that require a high sense of “presence,” there will be an obvious necessity to provide auditory as well as visual cues. Even if Wickens 's (1980, 1987) concept of multiple attentional resources has limits, it still holds much validity as a central design principle for the disbursement of tasks over multiple sensory systems and response modalities. Therefore, auditory input is likely to provide a second workload source, particularly if there is a high noise-to-signal ratio, as might be expected in battlefield operations. In addition, changing from the real world in which sound is matched to the visual display in the to virtual world

viewing, where sound may not be matched to the display, may induce confusion and workload as a soldier seeks to sort not only relevant information from irrelevant, but also real from representational sources. The last thing anyone wants is individuals so overloaded with information that they become essentially incapable of any coherent action. Simplicity of design and the provision of contextual information is critical for the control and mitigation of cognitive workload. While information input and output are typical task-related factors that influence workload, the tasks of an infantry soldier have an added physical component. With most standard subjective measures of workload, physical demand does play a role; physical demand is a nontrivial part of what is required to achieve infantry mission objectives.

Current measures of cognitive workload are presented in Appendix B. In general, it is important to recognize explicitly the concern with physical effort. The overwhelming majority of information on workload has been derived from either laboratory experiments or field evaluations in which physical demand has played a nonsignificant or, at most, a minor role. It must be emphasized that much remains unknown about the effects of cognitive demand on physical activity, or, conversely, the effects of physical activity on cognitive workload. Although there are some comparable conditions (such as fire fighting), relatively little experimental information is available from those endeavors on workload effects (but see Vercruyssen et al., 1989). Furthermore, attention has to be directed to the interaction that occurs when trying to do several cognitive tasks at once. There is some evidence concerning dual-task performance or time-sharing capabilities, but these have not been examined in association with physical effort.

Information provided through a helmet-mounted display cannot, in the current state-of-the-art, be used by a human observer as rapidly and as effectively, with the same situation awareness, and with the same sense of presence as is achieved with normal visual depth cues. Although several sources may be used to provide information (e.g., a range finder, triangulation by radio-linked soldiers) through such displays, these sources may not be equally well used by the human perceptual system. As a result, the engineering for helmet-mounted displays will need to take into account the perceptual requirements of the users, not merely meet local criteria of sensory discrimination. Since those perceptual needs depend heavily on the specific task and the setting in which the task is to be performed, real or simulated field tests will be crucial to the research and development effort. The panel's reviews of perceptual and cognitive aspects of the current technology has revealed some fundamental human factors concerns:

• Sensor and display resolution, field of view, contrast, and chromaticity are all less than optimum.

• Stereo depth information is either absent (in monocular and biocular displays) or anomalous (in binocular systems, with fixed disparity, collimated optics).

• Disparity between the location of sensors and displays tends to produce motion parallax errors, which make the world appear to move in anomalous ways.

• Current optical systems are not free of distortion, resulting in performance decrements and eye strain.

• Better optics (e.g., less distortion, larger fields of view) tend to be more complex and heavier than less capable optics.

• Helmet-mounted display optics weight must largely be placed in front of the eyes, exacerbating the added weight problem with poor center of gravity.

• More capable electronics tend to be more complex and have more adjustments.

• More capable electronic tend to require more processing and more power, therefore increasing battery weight.

Much of what is presently known about visual performance comes from laboratory investigations using standard, highly simplified visual tasks. System-specific research in both the laboratory and the field--taking into account the attributes and demands of specific tasks, environments, and equipment--will be needed to provide the knowledge base for solid engineering design decisions.

When possible, such studies should incorporate the stresses of battlefield conditions--in order to evaluate the potential for increased workload and for loss of situation awareness on the part of the soldier. The research plans should also include procedures for detecting design features that might support situation awareness by directing the soldier's attention to salient aspects of situations that otherwise might be missed. One way such information might be collected is to incorporate data hooks into early prototypes. These data hooks would provide essential information on how individual soldiers use the helmet-mounted displays and under what conditions.

Since the optimal configuration of helmet-mounted displays may depend significantly on task and environment, it is unlikely that a unique set of recommendations can be achieved. What may be achievable is a matrix of trade-off descriptions for the technology and task variations under consideration. Several issues that should be considered in the development of helmet-mounted display technology for use in 21 CLW follow.

• Opaque monocular displays can lead to visual rivalry. By occluding one eye, visual rivalry can remove targets from either eye through monocular suppression, thus causing such problems as temporary blindness in the unrestricted eye, temporary total blindness, and vomiting. These problems may be reduced by using the system in an intermittent fashion or by designing a fully synthetic environment.

• Opaque monocular displays and see-through displays can lead to loss of stereoscopic depth perception. Such loses are likely to impose important costs on infantry soldiers, such as reducing the ability to see through camouflage (to distinguish the

camouflage surface from its surround). Devices that eliminate stereopsis and that degrade other depth cues collapse everything into one depth plane and make it difficult to use attention effectively.

• Current technology offers less than optimum sensor and display resolution, field of view, contrast, and chromaticity. These factors, coupled with absent or anomalous stereo depth information, tend to keep the human accommodation and vergence systems running open-loop. The result can be eyestrain, fatigue, and, possibly, disorientation.

• Since pictorial and motion-based depth cues can be generated by computers, a helmet-mounted display might be used to restore or enhance lost depth information. Tradeoffs associated with enhancing or degrading depth cues can probably be estimated by assembling a series of matrices, based on the following: the distances for particular tasks, the distance ranges over which each class of depth cues are effective, the depth cues offered in each environment, and the effects of different display properties on different depth cues.

• Restriction of field of view interferes with the normal means by which humans deploy attention across the visual field. Although a viewer can compensate to some extent for a small field of view by making more head movements to obtain a series of small glimpses of the environment, there is currently no single accepted cognitive theory from which one can set the bounds of one glimpse. How are successive views of the display directly placed by the visual system into a single perceptual setting? How much structural overlap is required? Over how much delay? Over how many shifts in view?

• Since information can be provided by means of several modalities, it is important to determine the most appropriate modality for a given message in a given situation and evaluating the value added by using multiple modalities to provide redundant information (e.g., an auditory signal to direct the gaze of a soldier or to indicate the onset of an incoming text message).

• The weight of helmet-mounted display optics must be placed largely in front of the eyes, adding poor center of gravity to the weight problem. Better optics tend to be more complex and heavier than less capable optics.

• When a soldier is wearing a helmet-mounted display, the potential for attentional narrowing (paying exclusive attention to one source of information at the expense of other channels of information) is a significant concern: users may dwell on compelling and complex graphics and text displays at the expense of attending to otherwise available environmental cues. In a situation in which information overload occurs, users may be unable to efficiently use any of the available information or may be significantly delayed in locating the necessary information. Yet helmet-mounted displays can aid situation awareness by improving a wearer's ability to localize targets and self, to navigate in the environment, to keep up to date on situational factors, and to share information among team members.

Furthermore, a helmet-mounted display can aid situation awareness by presenting information in a format that is compatible with a soldier 's needs.

• Design of the helmet-mounted display may be able to remedy the main factors that limit situational awareness: for example, there may sensory enhancements that can improve a wearer's ability to localize self and targets and navigate in the environment, improve information sharing among team member, keep soldiers and commanders up to date, and allow soldiers to look at information in different ways to support decision making.

As we move into the second phase of our work, we will continue to examine the features of displays that relate to human perceptual, cognitive, and physiological characteristics. Moreover, we will further explore the impact of various technologies on situation awareness and cognitive workload for soldiers performing infantry tasks in a team context. Finally, we will direct our efforts towards the examination of human factors research and field testing methodologies appropriate for evaluating the benefits and costs of helmet-mounted displays.