topics already identified by the committee. Some topics were dropped, and some new papers were generated. Others were combined into coherent units; still others were deferred for further study or initiative.

The material in this report is the result of that process. Each chapter is designed to be a self-contained report of an important area in which research is needed. All the topics discussed here meet the following criteria: (1) each topic is germane to our military and NASA sponsors; (2) the topics are within the expertise of the committee; (3) each topic has been, in the opinion of the committee, incompletely addressed by previous or current military and civilian research efforts; and (4) the potential results of the recommended research will be important contributions to the scientific basis and practice of human factors. And the work of the committee is ongoing. In addition to the research areas presented in this report, work on a number of topics is in various stages of development: (1) organizational context in relation to design; (2) team performance; (3) simulation; (4) human performance modeling; (5) multicolor displays; (6) human factors education, and (7) accident reporting systems. We expect to address many of these as well as other topics in subsequent reports.

In the paragraphs that follow, the areas of research suggested by the committee are summarized together with some of our major recommendations. The chapters themselves provide a detailed elaboration of these topics.

HUMAN DECISION MAKING

A central issue in the understanding of human performance is human decision making. It has become even more important with the increased role of automation in complex modern systems ranging from military command, control, and communication systems to aircraft and process control systems. There has been much support for research on decision making over the last 15 years, particularly by the Defense Advanced Research Projects Agency and the Office for Naval Research. This research has tended to focus on formal decision theoretic constructions, which, while analytically powerful, have proved to be insufficiently robust to reflect the strengths and weaknesses of human decision-making capabilities. The committee recommends further research, with an emphasis on moving into uncharted areas.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 4
Research Needs for Human Factors topics already identified by the committee. Some topics were dropped, and some new papers were generated. Others were combined into coherent units; still others were deferred for further study or initiative. The material in this report is the result of that process. Each chapter is designed to be a self-contained report of an important area in which research is needed. All the topics discussed here meet the following criteria: (1) each topic is germane to our military and NASA sponsors; (2) the topics are within the expertise of the committee; (3) each topic has been, in the opinion of the committee, incompletely addressed by previous or current military and civilian research efforts; and (4) the potential results of the recommended research will be important contributions to the scientific basis and practice of human factors. And the work of the committee is ongoing. In addition to the research areas presented in this report, work on a number of topics is in various stages of development: (1) organizational context in relation to design; (2) team performance; (3) simulation; (4) human performance modeling; (5) multicolor displays; (6) human factors education, and (7) accident reporting systems. We expect to address many of these as well as other topics in subsequent reports. In the paragraphs that follow, the areas of research suggested by the committee are summarized together with some of our major recommendations. The chapters themselves provide a detailed elaboration of these topics. HUMAN DECISION MAKING A central issue in the understanding of human performance is human decision making. It has become even more important with the increased role of automation in complex modern systems ranging from military command, control, and communication systems to aircraft and process control systems. There has been much support for research on decision making over the last 15 years, particularly by the Defense Advanced Research Projects Agency and the Office for Naval Research. This research has tended to focus on formal decision theoretic constructions, which, while analytically powerful, have proved to be insufficiently robust to reflect the strengths and weaknesses of human decision-making capabilities. The committee recommends further research, with an emphasis on moving into uncharted areas.

OCR for page 4
Research Needs for Human Factors Surprisingly, despite the effort devoted to decision-making research in general, there is still a need for research on how to structure practical decision problems and on improving the realism of models that claim to relate to decision-making performance. We do not know how to represent decision situations that evolve dynamically, nor do we have a systematic framework from which to consider decision aiding. Furthermore, we are coming to realize that many planning activities actually involve decision making that cannot be modeled by enumerating the possible states of the world and courses of action in a unitary decision matrix. They often evolve over time in bits and pieces with limited central direction. We need a deeper understanding of such diffuse decision processes in order to provide effective computer aids for this kind of decision. While previous work has led to many decision-making aids and models, no criteria or methodologies have been suggested for evaluating their relative merits. Until such comparisons are made, practitioners will continue to advocate their own products without a basis for choice among them. Finally, there is a persistent need for development of innovative ways of soliciting preference and relative value judgments from people, a problem that leads us directly to the second topic. ELICITING EXPERT JUDGMENT The application of expert judgment covers everything from medical evaluations to accident investigations. Although the subject matter ranges widely, it is our belief that there are generic, substantive research issues that should be addressed in a coherent program. These problems recur in diverse contexts for which elicitation methods either do not exist or are inadequately standardized across applications to yield consistent results. The research issues include (1) creating a common frame of reference from which to assess judgments among a group of experts; (2) formulating questions for experts in a way that is compatible with their mental structures or cognitive representations of a problem; (3) eliciting judgments about the quality of information; (4) detecting and identifying reporting bias in judgments; and (5) minimizing the effects of memory loss and distortion on the reporting of past events.

OCR for page 4
Research Needs for Human Factors SUPERVISORY CONTROL SYSTEMS Supervisory control is a relatively new conceptualization of system function that is playing an increasingly important role in automated systems. In such systems, operators supervise the semiautomatic control of a dynamic process, such as a chemical plant or railway system. Typically the operators work in teams and control computers, which in turn mediate information flow among various automatic components. Other examples of supervisory control systems are modern aircraft, medical intensive care units, power plants, and distributed command and control systems such as may be found in military operations or in manufacturing by robots. Such systems deemphasize the importance of human sensory and motor capabilities and emphasize complex perceptual and cognitive skills. This perspective is relatively new to practicing system designers; work is beginning to be sponsored in these areas, but much further development is needed. Supervisory control may be thought of as a generalization from earlier work on monitoring and controlling complex systems; in that sense the foundations for modeling and theory are established. The theory must be greatly elaborated and extended, however, to meet the analysis requirements of current and future systems. As the human skills of thinking, reasoning, planning, and decision making become key, the models must be able to accommodate these human capacities and limitations. This is a choice opportunity to bring together work on control theory models and cognitive science representations. Cognitive psychology is also advancing our understanding of the way in which resources are shared among various processes within the brain. This work has unexplored implications for understanding how to modify system design to change perceived workload, particularly in the complex tasks typical of supervisory control. Each of the military services has research programs focused on human workload analysis. In our opinion many of them are too application-oriented; they need a stronger focus on research to advance the knowledge base from which new application techniques will emerge. Another key concern in supervisory control is prediction and the control of human error. Our understanding of this topic is in its infancy. We have no general theory of human error, although theories abound for human response time. Human reliability analysis has been in

OCR for page 4
Research Needs for Human Factors vogue for several years, but, as currently practiced, it simply uses the numerical aggregation of historical data on recorded human failure rates. It is weakest in just the situations in which it is most needed—when the activity involves complex diagnosis, situation assessment, and interaction with computers. At the level of design, there are three major questions: how to design supervisory control tasks to accommodate human capabilities and limitations; how to organize and display the information needed to carry out these tasks; and how much control to delegate to the human versus the automatic parts of the system. USER-COMPUTER INTERACTION Since computers are already playing a major role in most new system developments, including supervisory control systems, issues of facilitating the learning and use by both computer professionals and novices has been accorded a chapter of its own. At a March 1982 conference on user-computer interaction, more than 100 papers addressed a variety of topics related to hardware and software design. More than half of the 1,000 participants were system design specialists from industry and government. The committee believes that this level of interest foretells a heavy demand for scientific knowledge that has yet to be created. Although a number of industrial laboratories are supporting proprietary work, there is only one major funded collaborative effort between computer science and human factors specialists, that at Virginia Polytechnic Institute and State University (funded by the Office for Naval Research). Most human factors research has been done in the area of computer hardware. Information is available on which to base design decisions concerning information display hardware and keyboards. Many alternative input devices, such as joy sticks, track balls, and light pens, have been studied in the context of specific applications. There is a need for further work on input devices that focuses on comparison among the full range of devices across a broad set of uses, including instruction, text processing, and graphics. Automatic speech recognition and production have attracted much interest as the technology improves. Speech as an alternative to manual and visual modes of

OCR for page 4
Research Needs for Human Factors input and output needs systematic investigation. Fundamental work is necessary on the design of interactive speech dialogs that involve inherently sequential communication and potentially heavy memory demands on the listener. As computer terminals are becoming pervasive in the white-collar workplace, concern is growing about the adverse effects on people from long-term use of terminals with cathode ray tubes (CRTs). A recent study by the National Institute for Occupational Safety and Health found no radiation hazards from CRTs but did find a substantial increase in worker complaints of fatigue and other health problems from sustained daily use. This study was not able to distinguish CRT design-based complaints from those relating to the task or other features of the workplace—and this is an urgent research need. In Europe, governments are now mandating standards for workplace designs. It will not be long before similar actions are taken in the United States and the research must begin now to anticipate them. In the area of software design, research needs are only beginning to be filled. Effective design of sophisticated software implies understanding of human knowledge sytems and the ability to represent not only what a user knows but also how a user makes inferences from that information. There is a need for models of users’ understanding of the system with which they are interacting, a problem that is important for supervisory control applications as well. Perhaps the most neglected research area in computer system development is how to produce effective materials and reference information. While design principles developed for printed materials are useful for computer system documentation, there are documentation opportunities unique to interactive systems that we do not yet understand how to exploit effectively. Finally, there is a need to understand in more detail the characteristics of the user population that make a difference in computer system design. We need research that suggests, in parametric terms, how changes in user characteristics should be reflected in system design changes. The committee regards user-computer interaction as one of the most urgent topics on which to undertake research initiatives.

OCR for page 4
Research Needs for Human Factors POPULATION GROUP DIFFERENCES Through public sentiment as well as government legislation, our society has mandated the elimination of discrimination among population groups in the design of jobs and workplaces. In addition to racial discrimination, there is growing concern about discrimination on the basis of sex, age, and disability. We lack the research necessary to describe the nature and extent of performance differences among the various population groups about which discrimination is a concern. The committee believes it is in the national interest to undertake the research necessary to accommodate this relationship between population group differences and design. It is not enough to consider population group differences per se. In some cases the effect of a group characteristic such as age on performance may depend on the value of some other variables, such as amount of training or level of interpersonal skills. It may be misleading to discover simply that performance deteriorates with age, when in fact training or experience may reverse that trend. Such interactions remain largely unexplored. There is also a need to understand the way in which these differences in performance should influence workplace design or training procedures. We know how to write equipment specifications designed to fit 95 percent of a particular user population insofar as body dimensions are concerned, but for most other human performance characteristics we lack this knowledge. APPLIED METHODS Much human factors work is performed under constraints of money, time, and opportunity that preclude the use of the kind of experimental methods used in laboratory research. From necessity, human factors practitioners have adopted or developed a variety of applied methods for acquiring or organizing information related to human characteristics that arise in the context of system design, development, and evaluation. Examples of these methods are task analysis, information flow analysis, collection and analysis of survey data, evaluation of physical mock-ups, and the structured walk through. In contrast to the methods of scientific research, which are maintained and disseminated in university curricula and textbooks, and by specialists who devote careers to improving and

OCR for page 4
Research Needs for Human Factors inventing experimental design procedures, applied methods in human factors work are described only briefly in technical project reports, which are difficult to access, and efforts to improve or invent methods occur largely in connection with a particular project. There is a clear need to develop a compendium of standard descriptions of the most important applied methods. This compendium would be valuable for use in human factors curricula in colleges and universities and for continuing education tutorials for human factors practitioners. Currently most knowledge of applied methods is gained through on-the-job experience. Documenting existing applied methods, however, will not fulfill the methodological needs for all current and future system design purposes. Advances in computer technology applied to automation and supervisory control systems and computer systems themselves all have profound methodological implications for the analysis and description of the roles people play in these sytems. Existing methods such as workload analysis, protocol analysis, and function allocation require research to modify and extend their use in new applications in which the emphasis is on cognitive functions of operators rather than on the perceptual-motor functions prominent in old systems. Similarly, there is a need to develop new methods to provide information of the type and form necessary to resolve such issues as translating task requirements into personnel selection criteria, deriving training requirements from functional requirements, and describing or evaluating the effects of task or system functions on the affective responses of personnel. All the basic research needs addressed in this report require experimental investigations to provide the theory, principles, and data to support human factors work in the design and evaluation of systems. The application of the knowledge derived from basic research, however, will occur largely through the use of applied methods. Documentation of existing methods and research to extend and initiate methods to meet future needs are as essential as the substantive research to improve both the scientific basis and the practical effectiveness of human factors work. CONCLUSION System design and the world of work are undergoing profound changes. In a period when automation is replacing

OCR for page 4
Research Needs for Human Factors the need for finely tuned perceptual-motor activities by skilled operators, human productivity is no longer easily assessed in terms of unit output. New systems place increased demands on the cognitive and decision-making aspects of human performance. The role of people in systems is shifting to those of monitoring and directing otherwise automatic processes in industrial production, transportation, military operations, and office work. These changes in human-machine relations both offer new opportunities and present new problems for system design. It is therefore timely and appropriate that the committee’s first report of research needs in human factors emphasizes the importance of understanding fundamental cognitive processes and their role in interactive and supervisory control systems.

OCR for page 4
Research Needs for Human Factors 2 HUMAN DECISION MAKING Work organizations, and those who staff them, rise and fall by their ability to make decisions. These may be major strategic decisions, such as the deployment of forces or inventories, or local tactical decisions, such as how to promote, motivate, and understand particular subordinates. To list the kinds of decisions that need to be made and the stakes that sometimes ride on them would be to repeat the obvious. Decisions are made explicitly whenever one consciously combines beliefs and values in order to choose a course of action. They are made implicitly whenever one relies on a ritualized response (habit, tradition) to cope with a choice between options. Repetition of past decisions may result in suboptimal choices; however, it may also provide a ready escape from the difficulties and expense of explicit decision making. The reasons decision making often seems (and is) so difficult are quite varied, as are the opportunities for interventions and the needs for human factors research to buttress those interventions. One problem is information overload: More things need to be considered than can be held and manipulated in one’s head simultaneously. Coping with such computational problems is an ideal task for computers, and there are a variety of software packages available that in one way or another combine decision makers’ beliefs and values in order to produce a recommendation. Choosing between and using these decision aids forces one to face a second inherent difficulty of decision making: not knowing how to define (or structure) the decision problem and to assess one’s own values, that is, how to make trade-offs The principal author of this chapter is Baruch Fischhoff.

OCR for page 4
Research Needs for Human Factors between competing objectives. Because analytic decision-making methods cannot operate without guidance on these issues, judgment is an inevitable part of the decision-making process, as is the need for judgment elicitation methods to complement the decision aid (see Chapter 3). A third difficulty is knowing when to stop analyzing and start acting. Taking that step requires one to assess the quality of the decision-making process and reconcile any remaining conflicts between the recommendation it produces and that produced by one’s own intuitions. To help one through this step, a decision aid must reveal its own limits in ways that are psychologically meaningful. A fourth difficulty is that in many interesting decisions one knows too little to act confidently. When uncertainty is a fact of life, the role of good design is to ensure that the best use is made of all that is known. The existence of these four problems is common knowledge. Their resolution is complicated by a fifth difficulty whose identification requires research: People’s commonsense judgments are subject to robust-and systematic biases. These biases make it difficult to rely on intuition as a criterion for the adequacy of decisions and the methods that produce them. Decision aids must accommodate these biases and may require supplementary training exercises lest their recommendations be adopted only when they affirm intuitions that are known to be faulty. Given the multitude of decisions that are made, any research or design effort that made even a minute contribution to the quality of a minute proportion of all decisions would bear a large benefit in absolute terms. Proving that such a benefit had been derived would be as difficult as it is in most areas of human factors work. Whenever uncertainty is involved, better decisions will produce outcomes only over the long run. That makes it difficult to establish the validity of bona fide improvements and easy to fall prey to highly touted methods with good face validity, but little else. A sound research base is needed not only to develop better decision-making methods, but also to give users a fighting chance at being able to identify which methods are indeed better for their purposes. BACKGROUND Ad hoc advice to decision makers can be traced from antiquity to the Sunday supplements. Scientific study of