7
Research, Development, Testing, and Evaluation

The Army has a well-defined developmental and operational testing program for all new major and minor systems. The complex Land Warrior System, with its range of evolving technologies embedded in the helmet-mounted display, requires even more advanced test programs, supported by the thorough identification of critical issues and criteria and driven by systematically developed measures of effectiveness and measures of performance. In order to achieve a smooth testing process for this complex program, specific performance criteria-which are not presently available-must be identified to allow for unambiguous subsystem evaluation prior to testing the complete, integrated Land Warrior System.

With regard to the design effort for the helmet-mounted display, the lack of user-identified performance priorities makes the development of an effective test and evaluation program more difficult. Key issues in building an effective research program and an effective test and evaluation plan include identification of the technical areas requiring additional research and performance requirements that need discrete, objective evaluation. Historically, human factors research in the Army's programs has been underfunded and understaffed; it has not been a primary driver of revisions to proposed requirements; and it has not played a central role in the planning of creative testing programs. This situation needs to be changed.

In this chapter, we highlight the research that the panel judges to be essential and present a concept for research and testing that effectively incorporates critical human factors considerations for the Land Warrior System's helmet-mounted display.

In order to properly address these issues, a three-tier, highly integrated re-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 164
--> 7 Research, Development, Testing, and Evaluation The Army has a well-defined developmental and operational testing program for all new major and minor systems. The complex Land Warrior System, with its range of evolving technologies embedded in the helmet-mounted display, requires even more advanced test programs, supported by the thorough identification of critical issues and criteria and driven by systematically developed measures of effectiveness and measures of performance. In order to achieve a smooth testing process for this complex program, specific performance criteria-which are not presently available-must be identified to allow for unambiguous subsystem evaluation prior to testing the complete, integrated Land Warrior System. With regard to the design effort for the helmet-mounted display, the lack of user-identified performance priorities makes the development of an effective test and evaluation program more difficult. Key issues in building an effective research program and an effective test and evaluation plan include identification of the technical areas requiring additional research and performance requirements that need discrete, objective evaluation. Historically, human factors research in the Army's programs has been underfunded and understaffed; it has not been a primary driver of revisions to proposed requirements; and it has not played a central role in the planning of creative testing programs. This situation needs to be changed. In this chapter, we highlight the research that the panel judges to be essential and present a concept for research and testing that effectively incorporates critical human factors considerations for the Land Warrior System's helmet-mounted display. In order to properly address these issues, a three-tier, highly integrated re-

OCR for page 164
--> search, testing, and evaluation strategy is proposed. This approach represents a shift from current practice in that it includes an intermediate, semi-controlled set of research and testing experiments between laboratory and bench testing and operational field operations. Equally important, it incorporates the active involvement of users in every stage in the development sequence. First, do the technologies and capabilities embedded in the helmet-mounted display provide a significant improvement in the soldier's local situation awareness under the expected environmental and operational conditions? Second, does the helmet-mounted display significantly improve the visual perception of a soldier and his ability to perform operational tasks under tactical operational conditions? Third, how does the cognitive workload created by the helmet-mounted display affect task performance by soldiers under tactical and operational conditions? And forth, can current infantry soldiers, with existing skill qualifications, perform their expected operational tasks more effectively with the capabilities provided by the helmet-mounted display? These issues are elaborated in a later section of this chapter. ADOPTING A NEW STRATEGY Until the advent of the electronic age, the systems a soldier was required to use were relatively simple mechanical devices, and field testing had only to demonstrate that the equipment would continue to function in mud, rain, and ice. As systems have become more complex, requiring sophisticated training, and electronics-intensive with information transfer aspects, the gap between laboratory or bench testing and operational testing has become much larger. As a result, it is now possible for a system to meet all its engineering specifications in laboratory tests but turn out to be unusable in the field. Furthermore, the reasons for failure in the field may be obscure to the engineering design team, leading to "chase the tumbleweed" redesign efforts. The problem is most likely to be related to the combination of variations in field conditions and variations in the attributes of human users. The failure to recognize and deal with these variables and their interaction can result in incorrect definition or incomplete specifications that lead to limited prospects for success. We propose a methodology that brings the user into the testing and evaluation process earlier, through controlled testing that combines some varied environmental and personnel conditions from operational testing with the structured data collection and controlled conditions characteristic of laboratory testing. It is proposed that testing and evaluation for the helmet-mounted display be developed in three tiers as follows: Controlled laboratory or bench testing of technical performance of the system-both with and without human users.

OCR for page 164
--> Controlled field experiments with a variety of users, from experienced to new entry, and with system experts from the design team involved. Operational test and evaluation exercises employing soldiers from the target population in virtual-type simulations and live simulations in a realistic operational environment. The insertion of the mid-level tier of trials allows for the interaction of potential users and the design team in conditions that combine structured data collection with variability in environmental conditions (e.g., day, dusk, night for visual factors; camouflage for terrain variations) and individual variation in users (e.g., effects of regional accents on the performance of a voice recognition system for acoustics). We list below some guidelines, considerations, reservations, and underlying problems in human performance measurement from the Guide to Human Performance Measurement (American National Standards Institute) that should be kept in mind in designing the testing and evaluation program and user trials: Lack of a general theory to guide performance measurement. At present, only a few basic relationships have been discovered-for example, the inverse relationship between speed of task performance and accuracy, or performance quality. Measurement control and realism. An inverse relationship exists between the control of measurement conditions and operational realism. Behavior is multidimensional. Many factors influence human performance, some of great weight, others of little importance. Moreover, these factors probably change their importance over time and with different measurement contexts, and they may have different weights in different individuals. This results in the need to perform more experiments with more variables, subjects, and test trials. Unclear relationship between objective and subjective data. Objective measures are based on observable behavior (i.e., moving a lever, speaking a word). For complex tasks as well as phenomena that are not directly observable, objective measurements may not be possible or may be too costly to develop. Generalizability. Results may not be generalizable to the real world. Measuring cognitive activity. Cognitive activity is inherently more difficult to measure than physical performance because it takes place within the individual. Cognitive activity cannot be observed directly; it requires analysis of the output consequences of the cognition and, even more important, some kind of self-reporting, which may be tainted by its subjectivity. This means that measurement of behavior continues to be an inherently difficult process that is not becoming easier. In these cases, performance must be inferred from system inputs, outputs, and states, as well as from models of the operator's cognitive processes (Vreuls and Obermayer, 1985).

OCR for page 164
--> Performance criteria. There is a lack of objective performance criteria for most tasks. This lack of criteria makes it difficult to (a) assess performance quality and sufficiency and (b) identify the type of performance measurement techniques required to operationally define significant differences in performance (Vreuls and Obermayer, 1985). The considerations presented above are not intended to reflect an inability to design and evaluate user trials but rather to point out some potential pitfalls. Later in the chapter, we list the test conditions and criteria that may be useful in any testing or evaluation sequence designed to assess the helmet-mounted display and the Land Warrior System. PLANNING THE STRATEGY To be effective, developmental testing of the Land Warrior System and the helmet-mounted display must expand in scope and increase the human and operational variables introduced. The introduction of operational considerations should be executed in a controlled and prioritized manner with a deliberate research and testing sequence using controlled field experiments. To be effective and cost effective, subsequent operational testing should be implemented at the small unit level, minimizing larger-scale testing scenarios. The testing should have increased scope, with longer durations or cycles of task performance than have been used in the past. Soldier in-the-loop testing with increased emphasis on the number of operational considerations should be an early objective. In recent years, the Army has made major strides in developing, refining, and integrating constructive models, virtual simulations, and live simulations in the training arena. However, these advances have not been effectively applied in the area of test and evaluation, especially in the early stages of the development cycle. Rapidly growing modeling and simulation technologies appear to present a significant opportunity for enhancing the integration of human factors considerations into the development of the Land Warrior program. Simulation technology now presents a real possibility of assessing human performance. The U.S. Army Training and Doctrine Command is developing a series of virtual simulations and simulators that may be very useful in assessing human performance. However, there is insufficient attention to the measurement of human performance in these activities at the present time. Although the virtual simulations in their current form do not meet the design consideration needs for the helmet-mounted display per se, it is an example of simulation approaches that should be considered in planning and structuring the research and testing program for the helmet-mounted devices. As described by the user (U.S. Army Infantry School), the Land Warrior System is not being employed. This void must be filled prior to the development of a detailed test and evaluation program. Modeling and simulation can play useful role in specifying the parameters for use of the Land Warrior System use in the field.

OCR for page 164
--> By simulating a variety of threat environments, ranges of employment options, and different organizational structures, it should be possible to assess detection ranges, friendly and enemy engagement ranges, and the impact of a host of environmental and electronic warfare issues on the battlefield. The ability to quickly and relatively inexpensively vary potential battlefield conditions would allow the investigation of difficult performance specifications. As suggested above, the design, testing, and evaluation processes of the Land Warrior program and the helmet-mounted display can benefit from expanding the capabilities and application of constructive and virtual simulations. Such approaches could reduce the amount of operational testing, but they will not entirely eliminate the need for live simulations and tactical exercises. Executing experiments and employment exercises with real soldiers and their leaders in the loop has always been a primary part of the Army's test and evaluation process. If, however, live simulations are to have an impact on the design of the helmet-mounted display, a more stringent scientific methodology must be adopted for early operational testing. The large number of integrated subsystems that are part of the Land Warrior System make it difficult to measure the effects on human performance of a specific subsystem. The ability to measure the speed and to automatically locate and estimate the range of a potential enemy target, coupled with an automatic request for indirect fires, does not measure the soldier's concurrent potential to detect or failure to detect other, more significant threats that might mitigate the sheer speed of engaging one target. In order to properly evaluate the task-loading effects of the helmet-mounted display and the interfaces within the Land Warrior System, the discrete subsystems, sequences of events, and associated human task loads must be tested in a controlled environment. Such an approach will allow measurement of the addressed factors with a sampling of typical soldiers. User-Centered Design In the early stages of the research, development, test, and evaluation process, critical design decisions are often made by scientists and engineers who are knowledgeable concerning the technology but less expert in the operational employment of the new system. Without consistent and thorough collaboration with knowledgeable user representatives, the evolving design is driven by the engineer's and technician's view of what is important or what is possible, with less focus on what is most needed by the user. The Land Warrior System and the helmet-mounted display represent a major step forward in the integration of technology. To be effective, the research, development and design process must have more than user inputs; it must have user involvement and commitment. User-centered design is a significant aspect of the three-tier process described earlier. User-centered design is an outcome-oriented approach to system develop-

OCR for page 164
--> ment. This phrase means that user needs, capabilities, limitations, and preferences are not only taken into account when design decisions are being made, but also can actually drive such decisions. It also means that the user is the focus of attention throughout the development, and that the developing system is tested and evaluated in a realistic operational environment early in the development process. The objective is to build a high level of user acceptance and confidence in the system. The Land Warrior System will only be effective if infantry soldiers can achieve enthusiastic confidence in their abilities to perform their mission using it. The Army development process is well defined. The role of the Training and Doctrine Command as the user representative is well known and accepted within the development community. However, the level of knowledgeable and focused user interaction in the design process is, all too often, lacking. Using the philosophy of user-centered design can aid the accelerated development cycle that has been outlined for the Land Warrior System and the helmet-mounted display. Critical Test Issues In order to make human factors a vital part of the test and evaluation process for the helmet-mounted display, it is necessary to identify the critical human factors research questions that must be answered during the test and evaluation process. After careful analysis, the panel identified the following test issues in four broad areas as critical to the design of displays for the individual soldier: Local situation awareness, Visual perception, Cognitive workload, and Soldier qualifications. Local Situation Awareness Do the technologies and capabilities embedded in the helmet-mounted display provide a significant improvement in a soldier's local situation awareness under the expected environmental and operational conditions? An operational way to phrase this would be, "Does a dismounted infantry soldier equipped with a helmet-mounted display have a significantly improved ability to detect, identify, and effectively engage enemy targets over a currently equipped soldier?" The discussion on situation awareness in Chapter 3 addressed both global and local situation awareness. From the soldier's point of view, both his survival and his performance enhancement involve local situation awareness. Under what set of environmental or tactical employment conditions does the helmet-mounted display enhance soldier response time? Clearly, on the basis of what we know from current human factors research and technology development, enhanced

OCR for page 164
--> performance does not result simply by adding a helmet-mounted display capability. The criteria associated with this test issue deal with the employment conditions under which enhanced performance is sought. Performance comparisons of the currently equipped soldier with a Land Warrior-equipped soldier should be integral to the test plan. Visual Perception Does the helmet-mounted display significantly improve the visual perception of a soldier and his ability to perform operational tasks under tactical and operational conditions? This report has addressed some of the major visual perception issues that the design of the helmet-mounted display raises. More research is required to define operational parameters for task performance under likely operational conditions to establish acceptable trade-offs to maximize and enhance human visual capabilities. Cognitive Workload How does the cognitive workload created by the helmet-mounted display affect task performance by soldiers under tactical and operational conditions? Measuring cognitive workload is the most significant problem area that affects the design of the helmet-mounted display and the one that is most difficult to objectively quantify in testing. The outcomes of workload measurement are dependent on environmental conditions, individual human cognitive capabilities, and a large number of other variables. The research and testing program must first be focused on isolating the contribution of key variables under controlled conditions. Soldier Qualifications Can current infantry soldiers, with existing skill qualifications, perform their expected operational tasks more effectively with the helmet-mounted display and its designed capabilities? A basic assumption of the Land Warrior program is that the system will be used by basic infantry soldiers. This implies that no special selection criteria different from the current standards will be needed. Based on current research in human performance, it is unclear whether soldiers who meet the current Army criteria can operate the Land Warrior System and the complex helmet-mounted display and achieve enhanced performance under realistic battlefield conditions. The testing program for the helmet-mounted display should be designed to investigate the boundaries of effective performance for soldiers who meet the current standards and with soldiers from controlled groups who meet higher and more selective entrance requirements.

OCR for page 164
--> Cross-Cutting Issues A number of topics relate to more than one test issue. One is the effects of stress on performance; another is the organization of information and the methods selected to display that information. Stress may be caused by physical or cognitive workload or by poorly fitting equipment. Such stresses can influence situation awareness and the ability of the soldier to process and use information. Specifically, although there is some knowledge of separate physical and cognitive workload issues, there is almost no research information on their combined effects. As stress increases, attention narrows to a few immediately obvious informational cues. Battle places enormous stress on the soldier, so one should expect such narrowing to occur. Likewise, there are many unresolved questions related to the stress imposed by poor and ill-fitting equipment. As the ergonomic portions of the program continue, developers need to be informed by the stress reactions of users; otherwise, apparent gains may well be wiped out under actual combat operations. What is needed is much more basic research on this issue, tied to applied research questions associated with stressful performance using the helmet-mounted display. Questions of display design relate to all four test issues. One area in need of research attention is the hierarchical ordering of information from immediate threat to minor operational concerns and evaluating alternative presentation sequences and formats. Another research area concerns the allocation of information to visual versus auditory channels and the applicability of advanced technology, such as three-dimensional audio, to making these allocation decisions. Also, research is needed on the way in which graphic displays are structured and how these displays are formatted into standard iconic symbols for action. As discussed earlier, it is possible that new information-processing and display capabilities could be used to reduce stress by providing a global help function (e.g., location of nearest friendly force) at all times. A critical area of software development is the provision of this information in a secure manner. One could even imagine that an adaptive interface system could be used to on-load the soldier during periods of boredom and off-load the soldier during periods of high workload. IMPLEMENTING THE STRATEGY The panel proposes a three-tier concept of research and testing. The ideas of overlaying developmental and operational testing and expanding the use of simulations are not new to the Army, but embedding a level of controlled field testing with real users is. The Army test community has long recognized the potential of combining developmental and operational testing and evaluation as a cost- and time-saving approach. However, the relative difficulty of assessing technical performance to meet specific technical contract specifications, versus evaluating

OCR for page 164
--> system performance in typical user's hands under operational conditions, has never been adequately addressed. The latter requires integration of operational performance criteria and user personnel into technical and developmental tests as well as controlled field tests. In doing so, the experimental design features used in behavioral research should be incorporated, including the procedures associated with sampling human subjects that specify the sample characteristics and size requirements needed to draw robust conclusions. Such testing must be focused on critical performance criteria that can be specified in operational terms. This has been a stated objective within the Army's acquisition guidelines and the test and evaluation community for a number of years. The Land Warrior System Specification (Specification A3246133G) dated February 22, 1995, documents the requirements for the system and includes many operational criteria and parameters. However, the specifications are not at a level of operational specificity to provide insight into the human performance issues associated with the helmet-mounted display. Greater focus is needed on the types of human performance expected under realistic combat conditions. Table 7-1 highlights existing requirements for helmet-mounted display subsystems and the critical human-factors related test issues that must be resolved to permit the necessary specification of design requirements that address human operational performance criteria. In the panel's view, these issues, which relate directly to the broader test issues discussed previously, must drive the helmet-mounted display design and research and testing process. Although cognitive overload is addressed for each component in Table 7-1, it is important to keep in mind that questions of workload cannot be answered in a piecemeal fashion but require the integration of components. A system's performance requirements should define the environmental conditions under which the system must operate. Human performance is influenced by the single or combined effects of numerous factors that enhance or degrade and/or facilitate or interfere with that performance, depending on the particular situation. These factors, referred to as performance-shaping factors, are aspects of the human-machine system that influence behavior and affect the time or accuracy of the human response (Swain and Guttman, 1980). To be relevant, performance-shaping factors must have the potential for significantly affecting the magnitude of, frequency of, or variability in human performance. Identifying performance-shaping factors and establishing their effect on human performance is an important aspect of the research and testing necessary for designing the helmet-mounted display. Performance-shaping factors are grouped as operational, equipment, task, personnel, and environmental factors. Operational factors include doctrine and tactics, length of time of use, and system objective. Equipment performance-shaping factors for the helmet-mounted display include the physical parameters, operating characteristics, display layout, and reliability and maintainability. Personnel factors relate to items such as training, experience level of the user, moti-

OCR for page 164
--> vation, skill level of the user, fatigue, monotony, attitude, and workload. Task factors such as complexity, duration, repetitiveness and environmental factors such as distracting stimuli, vibration, motion, visibility, temperature, and noise must all be addressed in the design of specific test iterations to build a better understanding of the impacts on human performance of use of the helmet-mounted display. Research and Testing Approaches Table 7-2 lists the key research issues identified by the panel and proposes a general approach to addressing them. Included are the subsystems of the HMD and the subsystems that interface with the HMD. The test conditions and test criteria listed are representative of the conditions and measures that are most significant. The intent of the table is to list the kinds of conditions that must be addressed to assess human performance differences and the types of standards or criteria that should be considered. As we've said, the intent of the proposed three-tier test approach is (1) to use infantry soldiers in the research during controlled laboratory and bench testing whenever practical as well as under controlled and operational field conditions and (2) to strictly control test variables in an attempt to isolate the human performance differences. Identifying Measures of Effectiveness A measure of effectiveness is a measure with a standard or criterion that allows testers or researchers to evaluate performance. A measure of performance is a quantitative indicator without a standard that describes performance but does not evaluate it. The thrust of the proposed research and testing effort for the helmet-mounted display should be directed toward assessing the human performance of soldiers against operational measures of effectiveness early in the development cycle. The intent is to use controlled experiments in a much more operational way and in smaller structured test scenarios. Although it would be desirable to build simulations that allow the conduct of many iterations using actual soldiers, controlled testing can be initiated now and does not have to be executed on a large scale to obtain significant levels of human performance data. Tables 7-3 and 7-4 illustrate links among the Land Warrior System's mission tasks, potential measures of effectiveness and the proposed measures of effectiveness. The individual soldier tasks using the Land Warrior System are listed in Table 7-3 for a reconnaissance mission are not an exhaustive set. However, they represent tasks that we would specifically expect the individual soldier equipped with the Land Warrior System to perform in an enhanced manner over a currently equipped infantry soldier. Table 7-4 presents a similar set of tasks for a squad leader equipped with the Land Warrior System linked to a series of potential measures of effectiveness.

OCR for page 164
--> TABLE 7-1 Land Warrior/Helmet-Mounted Display Capability and Critical Issues Subsystem Description/Requirement Critical Helmet-Mounted Display Test/Research Issue GPS/computer radio subsystem -   Provides position location data -   Integrates information via software -   Operates under all operational conditions Does the GPS/computer radio subsystem cognitively overload the IIB Land Warrior soldier? (#3) Weapon subsystem -   Integrates target acquisition, location, and marking -   Increases speed of engagement -   Operates under all operational conditions Can Land Warrior soldiers more effectively engage targets with the integrated helmet-mounted display sight systems? (#1) Computer processor in the computer radio subsystem -   Executes software programs and computations -   Processes sensor inputs -   Operable by IIB soldiers (ASVAB scores) Do the functions of the computer processor cognitively overload the IIB Land Warrior soldier? (#3) Can IIBs who meet current qualification standards effectively employ the CPS? (#4) Sensor/display assembly (night sensor display) -   Displays clear video images -   Operates under all operational conditions -   Controls permit tactile identification -   Adjustments while wearing gloves -   Operable by IIB soldiers (ASVAB scores) Is local situational awareness of the Land Warrior soldier increased with the night sensor display? (#1) Does night sensor display enhance Land Warrior visual perception? (#2) Can IIBs who meet current qualification standards effectively employ the night sensor display? (#4) Do the functions of the night sensor display cognitively overload the IIB Land Warrior soldier? (#3) Sensor/display assembly (day sensor display) -   Displays clear video images -   Operates under all operational conditions -   Compatible with laser eye protection optical -   Operable by IIB soldiers (ASVAB scores) Is local situational awareness of the Land Warrior soldier increased with the day sensor display? (#1) Does day sensor display enhance Land Warrior visual perception? (#2) Do the functions of the day sensor display cognitively overload the IIB Land Warrior soldier? (#3) Remote input pointing device -   Operates under all operational conditions -   Operable by IIB soldiers (ASVAB scores) -   Tactile feedback with gloves Does the remote input pointing device support the workload of the Land Warrior soldier effectively? (#3) Can IIBs who meet current qualification standards effectively employ the remote input pointing device? (#4)

OCR for page 164
--> Subsystem Description/Requirement Critical Helmet-Mounted Display Test/Research Issue Audio amplifier of computer radio system -   Allows mixing of computer and radio outputs -   Controls overall audio volume level -   Provides audio emphasis control Is local situational awareness of the Land Warrior soldier increased by use of the audio amplifier? (#1) Audio headset (microphone/speaker) -   Receives/transmits radio and computer generated tones and outputs -   Useable in airborne and air assault operations -   Noise cancellation microphone Is local situational awareness of the Land Warrior soldier increased by use of the audio headset? (#1) Do the functions of the audio headset cognitively overload the IIB Land Warrior soldier? (#3) Ballistic helmet assembly (shell and suspension) -   Fits 5-95 percent male with improved stability -   Standard PSGT tooling (40 percent improve protection) Does the ballistic helmet assembly provide the required level of stability to support Land Warrior missions? Hand-held display and mini keyboard -   Operable while wearing gloves -   Tactile feedback with gloves and NBC Do the functions of the night sensor display cognitively overload the IIB Land Warrior soldier? (#3) The last column lists candidate measures of performance that would be used to quantify performance differences. Both tables are representative of the types of objective measures that need to be constructed for the individual soldier, squad leader, and platoon leader for all tactical infantry Land Warrior missions. The potential measures of effectiveness and measures of performance illustrated in Tables 7-3 and 7-4 are basic operational measures that could be applied in all three tiers of the proposed research and testing process. The intent is to conduct controlled experiments with test and control groups using representative soldiers under operational conditions starting early in the research and testing cycle. Such an approach would facilitate human performance assessment, and at the same time, minimize the amount of follow on operational testing required to support the Army's acquisition process. The key is establishing a consistent set of agreed upon operational conditions that can be applied in all testing phases.

OCR for page 164
--> TABLE 7-2 Research and Testing Approaches for the Helmet-Mounted Display Equipment Design Research Test Approach Test Conditions Test Criteria Helmet-mounted display and weapon sight Distance-offset of sensor/display Controlled field experiments Operational test and evaluation In buildings and with obstacles Time to detect targets Helmet-mounted display/night vision goggle and TWS Resolution Field of view Chromaticity Controlled laboratory/bench tests Controlled field experiments Operational test and evaluation ± Levels of visibility, distance, background, contrast, target-nontarget similarity Time to acquire Time to identify targets Navigation time and error rate Obstacle error rate (in lab and operational field conditions) Helmet-mounted display/night vision goggle and TWS Monocular versus biocular versus binocular Controlled laboratory/bench tests Controlled field experiments Operational test and evaluation ± Levels of employment, duration and use, light condition Time to acquire Time to identify targets Navigation error rate Physiological effects Obstacle error rate Time to detect unaided after aided

OCR for page 164
--> Helmet-mounted display and hand held device Information to be communicated Display format Input format Controlled field experiments Operational test and evaluation ± Levels of duration, fatigue, stressors, experience Time to perform tasks Task error rate Level of local situational awareness Helmet-mounted display/audio modality (monaural) Information cues in display: Placement, format, types: attentions, warnings Controlled field experiments ± Levels of visibility, distance, contrast, sensitivity, noise, task loading Percent cues detected Time to detect cues Time to identify information in environment Helmet-mounted display/3-D audio displays (biaural) Resolution needed Types need: Way-point navigational Localized communication Enhanced hearing Head tracker Stereo audio Controlled laboratory/bench tests Controlled field experiments LOS radio acoustically controlled; noise at defined volume/ frequency content; visual displays with control target Outdoors, ± ambient noise; LOS, masked at min/max ranges; visual targets at given distances; day, dusk, night Percent cues detected Time to detect cues Time to identify information in environment Percent correct Time to detect target information Time to identify target

OCR for page 164
--> Equipment Design Research Test Approach Test Conditions Test Criteria     Operational field testing Field exercise, noise, fatigue, masking, varied ranges and Percent correct recognition of standard numbers and letters Percent correct recognition of standard message format content Speech recognition (N.B.: Assessing system recognition of user/command voice input, converting to text) Vocabulary design Speaker display versus speaker independent Controlled laboratory/bench tests LOS radio acoustically controlled environment; insertion of noise at defined volume/frequency content Percent correct words on modified rhyme test   Discrete versus continuous     Percent correct of standard numbers and letters     Controlled field experiments Outdoors, ± ambient noise; LOS, masked at minimum/maximum ranges; visual targets at controlled distances Percent correct and time for target information detection and identification     Operational field testing Field exercise, noise, fatigue, masking, varied ranges-day, dusk, night.  

OCR for page 164
-->         Percent correct recognition of standard message format contents Recognition and application of message report contents Soldier/radio Intelligibility of speech Controlled laboratory/bench tests LOS radio acoustically controlled environment; insertion of noise at defined volume/ frequency content Percent correct words on modified rhyme test Percent correct of standard numbers and letters     Controlled field experiments Outdoors, ± ambient noise; LOS, masked at minimum/maximum ranges; visual targets at controlled distances Percent correct and time for target information detection and identification     Operational test and evaluation Field exercise, noise, fatigue, masking, varied ranges-day, dusk, night Percent correct recognition of standard message format contents Recognition and application of message report contents

OCR for page 164
--> TABLE 7-3 Soldier Battle Tasks and Potential Measures of Effectiveness (MOE)/Measures of Performance (MOP) Land Warrior Task Potential MOE Potential MOP Receive squad order Land Warrior soldiers have better knowledge Time required to issue order Percent of soldiers able to state mission and location Move tactically Moving Land Warrior soldiers are less detectable Time required to execute movement Number enemy Land Warrior detections Detection range Identify ORPs Land Warrior soldiers correctly identify ORPs Percent ORPs correctly identified Observe/listen for enemy Land Warrior soldiers detect enemy activity more effectively Percent enemy activity detected Number of correct detections Conduct reconnaissance Land Warrior soldiers are able to conduct accurate reconnaissance in less time Time to conduct reconnaissance Ratio of enemy Land Warrior detections to Land Warrior enemy detections Locate objective area Land Warrior soldiers locate objects more effectively Time to locate objectives Percent correct identification

OCR for page 164
--> Land Warrior Task Potential MOE Potential MOP Observe enemy activity Land Warrior soldiers detect enemy activity more effectively Percent enemy activity detected Number of correct detections Record information Reconnaissance results are more complete Time required to record results Percent observations recorded Report enemy information Land Warrior reports are more timely and accurate. Time required to deliver report Number of reports received Percent complete reports CONCLUSION The complex human performance issues associated with the Land Warrior System require a rigorous program of research integrated with effective user oriented field experiments and operational tests. Applying advanced technologies to enhance the performance of infantry soldiers cannot be achieved without also enhancing individual cognitive processing and individual local situation awareness under expected future battlefield conditions. To successfully achieve the optimistic capabilities envisioned in the Land Warrior program, there are significant elements of developmental risk that must be minimized by further research and controlled testing. The research must be completed in time to help drive the design of the helmet-mounted display.

OCR for page 164
--> TABLE 7-4 Squad Leader Battle Tasks and Potential Measures of Effectiveness (MOE)/Measures of Performance (MOP) Reconnaissance Reconnaissance Task Potential MOE Potential MOP Receive platoon order Land Warrior squad leaders have more order complete mission knowledge Time to receive platoon Percent squad leaders able to correctly state 5 Para Order details. Evaluate move route Land Warrior squad route effective in route evaluations Time to complete route evaluation Percent identification of critical features Move tactically Moving Land Warrior squads are less detectable Time to execute movement Number enemy Land Warrior detections Detection range Control quad movement Squad movement is more effective Time to execute movement Average squad dispersion Land navigate Squad leaders navigate more accurately Time to move and identify ORPs Percent correct location reports Identify ORPs Squads correctly identify ORPs Percent ORPs correctly identified Determine location Squad locations are accurately known Percent time squad leader correctly locates elements Report location Squad locations are accurately reported Percent time higher command levels know the correct squad location Observe/listen for enemy Squads detect enemy Percent enemy activity more effectively detected Number correct detections

OCR for page 164
--> Reconnaissance Task Potential MOE Potential MOP Conduct reconnaissance Squads conduct effective reconnaissances Time to conduct reconnaissance Percent enemy detections to Land Warrior detections Locate objective area Squads locate objectives effectively Time to locate objectives Percent correct identifications Observe enemy activity Squads detect enemy more effectively Percent enemy activity detected Number correct detections Record information Squad reconnaissance results are more complete Time required to record results Percent observations recorded Receive enemy information reports Squad reports of enemy are accurate and timely Percent enemy activity reported Number correct reports Time required to record results Percent observations recorded Report enemy information Squad reports of enemy are more timely and accurate Percent enemy activity reported Number correct reports Time required to record results Percent observations recorded Report situation Squad reports of enemy are accurate and timely Time required to deliver report Number enemy reports received Percent complete reports