decision making probably begins with the development of statistical or Bayesian decision theory by Borel, Ramsey, de Finetti, von Neumann, Morgenstern, Venn, Wald, and others. They showed how to characterize and interrelate the primitives of a general model of decision-making situations, highlighting its subjective elements. The development of scientific decision aids could be traced in the work of Edwards, Raiffa, Schlaifer, and others, who showed how complex real-world decision situations could be interpreted in terms of the general model. Essential to this model is the notion that decision-making problems can be decomposed into components that can be assessed individually, then combined into a general recommendation that reflects the decision makers’ best interest. Those components are typically described as options, beliefs, and values or alternatives, opinions, and preferences, or some equivalent triplet of terms. They are interrelated by an integration scheme called a decision rule or problem structure (e.g., Fischhoff, et al., 1981; Sage, 1981).

More generally, decision-making models typically envision four interrelated steps.

  1. Identify all relevant courses of action among which the decision maker may choose. This choice among options (or alternatives) constitutes the act of decision; the deliberations that precede it are considered to be part of the decision-making process.

  2. Identify the consequences (advantages) that may arise as a result of choosing each option; assess their relative attractiveness. In this act the decision maker’s values find their expression. Although these values are essentially personal, they may be clarified by techniques such as multiattribute utility analysis and informed by economic techniques that attempt to establish the market value of consequences.

  3. Assess the likelihood of these consequences’ being realized. These probabilities may be elicited by straightforward judgmental methods or with the aid of more sophisticated techniques, such as fault tree and event tree analysis. If the decision maker knows exactly what will happen given each course of action, it then becomes a case of decision making under conditions of certainty and this stage drops out.

  4. Integrate all these considerations in order to identify what appears to be the best option. Making the best of what is or could be known at the time of the



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 14
Research Needs for Human Factors decision making probably begins with the development of statistical or Bayesian decision theory by Borel, Ramsey, de Finetti, von Neumann, Morgenstern, Venn, Wald, and others. They showed how to characterize and interrelate the primitives of a general model of decision-making situations, highlighting its subjective elements. The development of scientific decision aids could be traced in the work of Edwards, Raiffa, Schlaifer, and others, who showed how complex real-world decision situations could be interpreted in terms of the general model. Essential to this model is the notion that decision-making problems can be decomposed into components that can be assessed individually, then combined into a general recommendation that reflects the decision makers’ best interest. Those components are typically described as options, beliefs, and values or alternatives, opinions, and preferences, or some equivalent triplet of terms. They are interrelated by an integration scheme called a decision rule or problem structure (e.g., Fischhoff, et al., 1981; Sage, 1981). More generally, decision-making models typically envision four interrelated steps. Identify all relevant courses of action among which the decision maker may choose. This choice among options (or alternatives) constitutes the act of decision; the deliberations that precede it are considered to be part of the decision-making process. Identify the consequences (advantages) that may arise as a result of choosing each option; assess their relative attractiveness. In this act the decision maker’s values find their expression. Although these values are essentially personal, they may be clarified by techniques such as multiattribute utility analysis and informed by economic techniques that attempt to establish the market value of consequences. Assess the likelihood of these consequences’ being realized. These probabilities may be elicited by straightforward judgmental methods or with the aid of more sophisticated techniques, such as fault tree and event tree analysis. If the decision maker knows exactly what will happen given each course of action, it then becomes a case of decision making under conditions of certainty and this stage drops out. Integrate all these considerations in order to identify what appears to be the best option. Making the best of what is or could be known at the time of the

OCR for page 14
Research Needs for Human Factors decision is the hallmark of good decision making. The decision maker is not to be held responsible if this action meets with misfortune and an undesired option is obtained. These steps are both demanding and vague. Fulfilling them requires considerable attention to detail and may be accomplished in a variety of ways. Moreover, they may not even be followed sequentially, if insights gained at one step lead the decision maker to revise the analysis performed at a different step. This flexibility has produced a variety of models and methods of decision making whose interrelations are not always clearly specified. The opportunity for routinizing and merchandising these decision-making procedures led to one of the academic and consulting growth industries of the 1970s. A wide variety of software packages and firms can now bring the fruits of these theoretical advances to practicing decision makers. Decision analysis, the most common name for these procedures, is part of the curriculum of most business schools. Although it has met considerable initial resistance from decision makers because of its novelty and because of the explicitness about values and beliefs that it requires, decision analysis seems to be gaining considerable acceptance (e.g., Bonczek, et al., 1981; Brown, et al., 1974; Raiffa, 1968). This acceptance seems, even now, to go beyond what could be justified on the basis of any empirical evidence of its efficacy. Figure 2–1 gives some examples of the contexts within which decision-aiding schemes relying on interactive computer systems have been operating and have been reported in the professional literature. Figure 2–2 is similar to the summary printout of one such scheme, which offers physicians on-line diagnoses of the causes of dyspepsia. Behavioral decision theory (e.g., Einhorn and Hogarth, 1981; Slovic, et al., 1979; Wallsten, 1980) has taken decision aiding out of the realm of mathematics and merchandising into the realm of behavioral research by recognizing the role of judgment in structuring problems and in eliciting their components. Researchers in this field have studied, in varying degrees of detail, the psychological processes underlying these judgments and the ways in which they can be improved through training, task restructuring, and decision-aid design. A particular focus has been on the identification and eradication of judgmental biases. The research described below is that which seems to be needed to help behavioral decision research fulfill this role.

OCR for page 14
Research Needs for Human Factors FIGURE 2–1 Examples of Operating Decision-Aiding Systems An important development in this research over the last decade has been its liberation from the mechanistic models of behavior inherited from economics and philosophy. The result has been more process-oriented theories, attempting to capture how people do make and would like to make decisions (e.g., Svenson, 1979). This change was prompted in part by the realization that mechanistic models offer little insight into central questions of applications, such as how action options are generated and when people are satisfied with the quality of their decisions. These developments are reflected in the research described below. There may seem to be a natural enmity between those purveying techniques of decision analysis and those studying their behavioral underpinnings, with the latter revealing the limits of the procedures that the former are trying to sell. In general, however, there has been rather good cooperation between the two camps. Basic researchers have often chosen to study the problems that practitioners find most troublesome, and practitioners have often adopted basic researchers’ suggestions for how to improve their craft. For example, in both commercial and government use, one can find software packages and decision-making procedures that have been redesigned in

OCR for page 14
Research Needs for Human Factors FIGURE 2–2 Summary Printout of a Medical Decision-Aiding Scheme Source: D.C.Barber and J.Fox (1981).

OCR for page 14
Research Needs for Human Factors response to basic research. Established channels (e.g., conferences, paper distribution lists) exist for members of this community to communicate with one another. Many of the leading practitioners have doctoral-level training, usually in psychology, management science, operations research, or systems engineering, and maintain academic contacts. Indeed, the quantity of basic research has been reduced by the diversion of potential researchers to applied work, although its quality may have benefited from being better focused. Although problems remain, research in this area has a fairly good chance of being useful and of being used. In addition, none of the research issues discussed in the following sections appears to pose any serious methodological difficulties. The conventional experimental methods of the behavioral sciences are suitable for performing the recommended investigations. RESEARCH ON DECISION MAKING Given the relatively good communication between decision-making researchers and practitioners, the primary focus of the recommendations that follow is the production of new research, as opposed to its dissemination. It seems reasonable to hope that the same communication networks that brought these applied problems to the attention of academics will carry their partial solutions back to the field. Research on decision making per se assumes that there are general lessons to be learned from studying the sorts of issues that recur in many decision problems and the responses typically made to them. In fact, the complexity of real decision problems is often so great as to prevent some lessons from being learned from direct study. These recommendations are cast in terms of research needed to improve the use of computerized decision aids, referred to generically as decision analysis. These aids work in an interactive fashion, asking people to provide critical inputs (e.g., the set of actions that they are considering, the probability of those actions achieving various goals), combining those inputs into a recommendation of what action to take, and repeating the process until users feel that they have exhausted its possibilities. In order to be useful, an aid must: (a) deal with those aspects of decision making for which people require assistance, (b) ask for inputs in a language compatible with how people think intuitively about

OCR for page 14
Research Needs for Human Factors decision making, and (c) display its recommendations in a way that properly captures their implications and definitiveness. Achieving these goals requires understanding of (a) how people assess the quality of human performance in decision-making tasks, (b) the nature of decision-making processes, and (c) how people assess the quality of decision-making processes, both those they perform and those performed for them. The research described below is intended to contribute to all three of these aspects of systems design. It is also intended to facilitate the development of supplementary components of decision-support systems, such as exercises for improving judgment or for more creative option generation. In this light, research that contributes to hardware or software design should also be a useful adjunct to any formal or semiformal decision-making process in which judgment plays a role. Even the devotee of decision analysis often lacks the time or resources to do anything but an informal analysis. Decision Structuring Decision making is commonly characterized as involving the four interrelated steps described earlier. The first three of these give the problem its structure, by specifying the options, facts, and value issues to be considered as well as their interrelations. Prescriptive models of decision making elaborate on the way these steps should be taken. Most descriptive theories hypothesize some deviation of people’s practice from a prescriptive model (Fischhoff, Goitein, and Shapira, 1981). These deviations should, in principle, guide the development of the prescriptive model. That is, they show how the prescriptive models fail to consider issues that people want to incorporate in their decisions. In practice, however, the flow of information is typically asymmetrical, with prescriptive models disproportionately setting the tone for descriptive research. As a result, decision structuring is probably the least developed aspect of research into both prescriptive and descriptive aspects of decision making (von Winterfeldt, 1980). Prescriptive models are typically developed from the pronouncements of economists and others regarding how people should (want to) run their lives or from ad hoc lists of relevant considerations. Descriptive models tend more or less to assume that these prescriptions are

OCR for page 14
Research Needs for Human Factors correct. Neither seems to have explored fully the range of possible problem representations that people use when left to their own devices. Paying more attention to the diverse ways in which people do make decisions would enable decision aiders to offer their clients a more diverse set of alternative ways in which they might make decisions, along with some elaboration on the typical strengths and weaknesses of each method. Some research projects that might serve this end follow. Studies of dynamic structuring, allowing for iterations in the decision-making process, with each round responding to the insights gained from its predecessors (Humphreys and McFadden, 1980). Can people use such opportunities, or do they tend to stick to an initial representation? Are there initial structures that are less confining, which should be offered by the aids? Studies of goals other than narrow optimization. In economic models, the goal of decision making is assumed to be maximizing the utility of the immediate decision. Recently attention has turned to other goals, such as reducing the transaction costs from the act of making a decision, improving trust between the individuals involved in a decision, making do with limited decision-making expertise, imposing consistency over a set of decisions, or facilitating learning from experience. Theoretical studies are needed to clarify the consequences of adopting these goals (e.g., how badly do they sacrifice optimization); empirical studies are needed to see how often people actually want to accept them (particularly after they have been informed of the results of the theoretical studies). Option-generation studies. Decision makers can only choose between the options they can think of. Each decision need not be a new test of their imaginations, particularly because research indicates that imagination often fails. Research can suggest better formulation procedures and generic options that can be built into decision analysis schemes (Gettys and Fisher, 1979). Many decision analysis schemes are sold as standalone systems, to be used by decision makers without the help of a professional decision analyst. The validity of these claims should be tested, particularly with regard to decision structuring, the area in which the largest errors can occur (Pitz, et al., 1980). Research could also show ways to improve the stand-alone capability (e.g., with better introductory training packets).

OCR for page 14
Research Needs for Human Factors Measuring Preferences Unless one is fortunate enough to find a dominating alternative, one that is better than all competitors in all respects, making decisions means making trade-offs. When one cannot have everything, it is necessary to determine the relative importance of different goals. Such balancing acts may be particularly difficult when the question is new and the goals that stand in conflict seem incommensurable (Fischhoff, et al., 1980). Dealing with hazardous technologies, for example, leads us daily to face questions such as whether the benefits of dyeing one’s hair are worth a vague, minute increase in the chances of cancer many years hence. Decision analysis schemes seem to complicate life by making these inherent conflicts apparent (McNeil, et al., 1978). They actually complicate it when they pose these questions in cumbersome, unfamiliar ways in order to elicit the information needed by their models—e.g., how great an increase in your probability of being alive in five years’ time would exactly compensate for the .20 probability that you will not recover from the proposed surgery—and does this trade-off depend on other factors? Such questions are difficult in part because their format is dictated by a formal theory or the programmer’s convenience, rather than by the decision maker’s way of thinking. They are also difficult because of the lack of research guiding their formulation. Research on the elicitation of values has lagged behind research on the elicitation of judgments of fact (Johnson and Huber, 1977). Although there are many highly sophisticated axiomatic schemes for posing value questions, few have been empirically validated for difficult, real-life issues. In practice, perhaps the most common assumption is that decision makers are able to articulate responses to any question that is stated in good English. The projects described below may help solve problems that currently are (or should be) worrying practitioners. Some similar needs have been identified by the National Research Council’s Panel on Survey-Based Measures of Subjective Phenomena (Turner and Martin, in press). No opinion. In most behavorial decision research, as in most survey research, economics, and preference theory, people are typically assumed to know what they want. Careful questioning is all that is needed to reveal the decision maker’s implicit trade-offs between whatever

OCR for page 14
Research Needs for Human Factors goals are being compared. The need for some response is often necessary for the analysis to continue. Knowing how to discover when decision makers have no opinions and how to cope with that situation would be of great value. Studies of “no opinion” in survey research (Schumann and Presser, 1979) would provide a useful base to draw on, although they often show that people have a disturbing ability to manufacture opinions on diverse (and even fictitious) topics. Interactive value measurement. One possible response to situations in which decision makers’ values are poorly articulated (or nonexistent) is for the decision aider to engage in a dialogue with the client, suggesting alternative ways of thinking about the problem and the implications of various possible resolutions. Although there are obvious opportunities for manipulating responses in such situations, research may show how they could be minimized; at any rate they may be rendered no worse than the manipulation inherent in not confronting the ambiguity in respondents’ values. Of particular interest is the question of whether people are more frank about their values and less susceptible to outside pressures when interacting with a machine than with another human being. Again, some good leads could be found in the survey research literature, particularly in work dealing with the power and prevalence of interviewer effect. Specific topics. In order to interact constructively with their clients, should decision aiders be able to offer a comprehensive, balanced description of the perspectives that one could have on a problem? The provision of such perspectives may be enhanced by a combination of theoretical and empirical work on how people could and do think about particular issues (Jungermann, 1980). For example, to aid decision problems that involve extended time horizons, one would study how people think about good and bad outcomes that are distributed over time. One might discover that people have difficulty conceptualizing distant consequences and therefore tend to discount them unduly; such a tendency could be countered by the use of scenarios that reify hypothetical future experiences. Medical counseling and the setting of safety standards are two other areas with specific problems that reduce the usefulness of decision technologies (e.g., the difficulty of imagining what it would be like to be paralyzed or on dialysis, unwillingness to place a value on human life).

OCR for page 14
Research Needs for Human Factors Simulating values. One obvious advantage of computerized systems is to work quickly through calculations using alternative values of different parameters. A possible didactic use would be to help people clarify what they want, by simulating the implications of different sets of preferences (“If those were your trade-offs, these would be your choices”), both on the problem in question and on sample problems. Work along this line was done at one time in the context of social judgment theory (Hammond, 1971). Completing it and making it accessible to the users of other decision aids would be useful. Framing. Recent research has demonstrated that formally equivalent ways of representing decision problems can elicit highly inconsistent preferences (Kahneman and Tversky, 1979; Tversky and Kahneman, 1981). Because most decision-aiding schemes have a typical manner of formulating preference questions, they may inadvertently be biasing the results they produce. This work should be continued, with an eye to characterizing and studying the ways in which decision analysis schemes habitually frame questions. Evaluation The decision maker looking for help may be swamped by offers. The range of available options may run from computerized decision analysis routines to super-soft decision therapies. Few of these schemes are supported by empirical validation studies; most are offered by individuals with a vested interest in their acceptance (Fischhoff, 1980). A comprehensive evaluation program would help decision makers sort out the contenders for their attention and to use those selected judiciously, with a full understanding of their strengths and limitations (Wardle and Wardle, 1978). Such a program might involve the following elements: Collecting and characterizing the set of existing decision aids with an eye to discerning common behavorial assumptions (e.g., regarding the real difficulties people have in making decisions, the ways in which they want to have problems structured, or the quality of the judgment inputs they can provide to decision-making models). Examining the assumptions identified above. This might include questions like: Can people separate judgments of fact from judgments of value? When decision

OCR for page 14
Research Needs for Human Factors makers are set to act in the name of an institution, can they assess its preferences, unencumbered by their own? Can people introspect usefully about beliefs that have guided their past decisions, free from the biasing effects of hindsight? Developing methods for evaluating the quality of decisions (such as are produced by different methods). For example, what weights should be placed on the quality of the decision process and on the quality of the outcome that arises? What level of successful outcomes should be expected in situations of varying difficulty? This work would be primarily theoretical (Fischer, 1976). Clarifying the method’s degree of determinacy. To what extent do arbitrary changes (i.e., ones regarding which the method is silent) in mode of application affect the decisions that arise (Hogarth and Makridakis, 1981)? Similarly, one would like some general guidance on the sensitivity of the procedure to changes in various aspects of the decision-making process, in order to concentrate efforts on the most important areas (e.g., problem structuring or value elicitation). Conversely, one wants to know how sensitive the method is to the particulars of each problem and user. That is, does it tend to render the same advice in all circumstances? Assessing the impact of different methods on “process” variables, such as the decision maker’s alertness to new information that threatens the validity of the decision analysis or the degree of acceptance that a procedure generates for the recommendation it produces (Watson and Brown, 1978). Such questioning of assumptions has been the goal of much existing research, which should provide a firm data base for new work (although many questions, such as the first two of the three raised, have yet to be studied). Improving Realism The simplified models of the world that decision analysis software packages use to represent decision problems are in at least one key respect very similar to the models generated by flight or weapons simulators. Their usefulness is constrained by the fidelity of their representations to the critical features of the world they hope to model. Although there is much speculation about process effects, it points in inconsistent directions and is seldom substantiated by empirical studies (either in the

OCR for page 14
Research Needs for Human Factors would like to automate human operators out of their systems. But they know they must depend on them to plan, program, monitor, step in when failures occur with the automation, and generalize on system experience. They are also terrified of human error. Both the commercial aviation and the nuclear power industries are actively collecting data on human error and trying to use it analytically in conjunction with data on failures in physical components and subsystems to predict the reliability of overall systems. The public and the Congress, in a sense, are demanding it, on the assumption that it is clear what human error is, how to measure it, and even how to stop it. Human error is commonly thought of as a mistake of action or judgment that could have been avoided had the individual been more alert, attentive, or conscientious. That is, the source of error is considered to be internal and therefore within the control of the individual and not induced by external factors such as the design of the equipment, the task requirements, or lack of adaquate training. Some behavioral scientists may claim that people err because they are operating “open loop”—without adequate feedback to tell them when they are in error. They would have supervisory control systems designers provide feedback at every potential misstep. Product liability litigants sometimes take a more extreme stance—that equipment should be designed so that it is error proof, without the opportunity for people to (begin to) err, get feedback, then correct themselves. The concept of human error needs to be examined. The assertion that an error has been committed implies a sharp and agreed-upon dividing line between right and wrong, a simple binary classification that is obviously an over-simplification. Human decision and action involve a multidimensional continuum of perceiving, remembering, planning, even socially interacting. Clearly the fraction of errors in any set of human response data is a function of where the boundry is drawn. How does one decide where to draw the line dividing right from wrong across the many dimensions of behavior? In addition, is an error of commission, (e.g., actuating a switch when it is not expected), equivalent to an error of omission, (e.g., failing to actuate a switch when it is expected)? Is it useful to say, in both these instances, an error has been committed? What then exactly do we mean by human error?

OCR for page 14
Research Needs for Human Factors People tend to differ from machines in that people are more inclined to make “common-mode errors,” in which one failure leads to another, presumably because of concurrency of stimuli or responses in space or time. Furthermore, as suggested earlier, if a person is well practiced in a procedure ABC, and must occassionally do DBE, he or she is quite likely in the latter case to find himself or herself doing DBC. This type of error is well documented in process control, in which many and varied procedures are followed. In addition, when people are under stress of emergency they tend more often to err (sometimes, however, analysts may assume that operators are aware of an emergency when they are not). People are also able to discover and correct their own errors, which they surely do in many large-scale systems to avert costly accidents. Presumably the rationale for defining human error is to develop means for predicting when they are likely to occur and for reducing their frequency (Swain and Gutman, 1980). Various taxonomies of human error have been devised. There are errors of omission and errors of comission. Errors may be associated with sensing, memory, decision making, or motor skill. Norman (1981) distinguishes mistakes (wrong intention) from slips (correct intention but wrong action). But at present there is no accepted taxonomy on which to base the definition of human error, nor is there agreement on the dimensions of behavior that should be invoked in such a taxonomy. There is usefulness in both a case study approach to human error and in the accumulation of statistics on errors that lead to accidents. Both these approaches, however, require that the investigator have a theory or model of human error or accident causation and the framework from which to approach the analysis. In addition there is a need to understand the causal chain between human error and accident. One has only to examine a sampling of currently used accident reporting forms to realize the importance of the need for a framework for analyzing human error. They range from medical history forms to equipment failure reports. None that we have examined deals satisfactorily with the role of human behavior in contributing to the accident circumstances. Furthermore, for accident reports to be useful, their aim needs to be specified. There is an inherent conflict between the goals of understanding what happened and attempting to fix blame for it. The former requires candor, whereas the latter discourages it. Other poten-

OCR for page 14
Research Needs for Human Factors tial biases in these reports include: (a) exaggerating in hindsight what could have been anticipated in foresight; (b) being unable to reconstruct or retrieve hypotheses about what was happening that no longer makes sense in retrospect; (c) telescoping the sequence of events (making their temporal course seem shorter and more direct); (d) exaggerating one’s own role in events; (e) failing to see the internal logic of others’ actions (from their own perspective). Variants of these reporting biases have been observed elsewhere (Nisbett and Ross, 1980). Their presence and virulence in accident reports on supervisory control systems merits attention. In addition to these fundamental research needs, there is a variety of related issues particularly relevant to supervisory control systems that should be addressed. In supervisory control systems it is becoming more and more difficult to establish blame, for the information exchange between operators and computers is complex, and the “error,” if there ever was any, could be in hardware or software design, maintenance, or management. Most of us think we observe that people are better at some kinds of tasks than computers, and computers are better at some others. Therefore, it seems that it would be quite clear how roles should be allocated between people and computers. But the interactions are often so subtle as to elude understanding. It is also conventional wisdom to say that people should have the ultimate authority over machines. But again, in actual operating systems we usually find ourselves ill prepared to assert which should have authority under what circumstances and for how long. Operators in such systems usually receive fairly elaborate training in both theory and operating skills. The latter is or should be done on simulators, since in actual systems the most important (critical) events for which the operator needs training seldom occur. Unfortunately there has been a tendency to standardize the emergencies (classic stall or engine fire in aircraft, large-break loss-of-cooling accident in nuclear plants) and repeat them on the simulator until they become fixed patterns of response. There seldom is emphasis on responding to new, unusual emergencies, failures in combination, etc., which the rule book never anticipated. Simulators would be especially good for such training. A frustrating, and perhaps paradoxical, feature of “emergency” intervention is that supervisors must still

OCR for page 14
Research Needs for Human Factors rely on and work with systems that they do not entirely trust. The nature and success of their intervention is likely to depend on their appraisal of which aspects of the system are still reliable. Research might help predict what doubts about related malfunctions are and are not aroused by a particular malfunction. Does the spread of suspicion follow the operator’s mental model (e.g., lead to other mechanically connected subsystems) or along a more associative line (e.g., mistrust all dials)? A related problem is how experience with one malfunction of a complex system cues the interpretation of subsequent malfunctions. Is the threshold of mistrust lowered? Is there an unjustified assumption that the same problem is repeating itself, or that the same information-searching procedures are needed? How is the expectation of successful coping affected? Do operators assume that they will have the same amount of time to diagnose and act? Finally, how does that experience generalize to other technical systems? Do bad experiences lead to a general resistance to innovation? A key to answering these questions is understanding the operators’ own attribution processes. Do they subscribe to the same definition of human error as do those who evaluate their performance? What gives them a feeling of control? How do they assign responsibility for successful and unsuccessful experiences? Although their mental models should provide some answers to these questions, others may be sought in general principles of causal attribution and misattribution (Harvey, et al., 1976). CONCLUSIONS AND RECOMMENDATIONS Supervisory control of large, complex, capital-intensive, high-risk systems is a general trend, driven both by new technology and by the belief that this mode of control will provide greater efficiency and reliability. The human factors aspects of supervisory control have been neglected. Without further research they may well become the bottleneck and most vulnerable or most sensitive aspect of these systems. Reseach is needed on: How to display integrated dynamic system relationships in a way that is understandable and accessible. This includes how best to allow the computer to tell the operator what it knows, assumes, and intends.

OCR for page 14
Research Needs for Human Factors How best to allow the operator to tell the computer what he or she wants and why, in a flexible and natural way. How to discover the internal cognitive model of the environmental process that the operator is controlling and improve that cognitive representation if it is inappropriate. How to aid the cognitive process by computer-based knowledge structures and planning models. Why people make errors in system operation, how to minimize these errors, and how to factor human errors into analyses of system reliability. How mental workload affects human error making in systems operation and refinement and standardization of definitions and measures of mental workload. Whether human operator or computer should have authority under what circumstances. How to coordinate the efforts of the different humans involved in supervisory control of the same system. How best to learn from experience with such large, complex, interactive systems. How to improve communication between the designers and operators of technical systems. Research is needed to improve our understanding of human-computer collaboration in such systems and on how to characterize it in models. The validation of such models is also a key problem, not unlike the problem of validating socioeconomic or other large-scale system models. In view of the scale of supervisory control systems, closer collaboration between researchers and systems designers in the development of such systems may be the best way for such research, modeling, and validation to occur. And perhaps data collection should be built in to the normal—and abnormal—operation of such systems. REFERENCES Bainbridge, L. 1974 Analysis of verbal protocols from a process control task. In E.Edwards and F.Lees, eds., The Human Operator in Process Control. London: Taylor and Francis.

OCR for page 14
Research Needs for Human Factors Baron, S., Zacharias, G., Muralidharan, R., and Lancraft, R. 1981 PROCRU: A model for analyzing flight crew procedures in approach to landing. In Proceedings of the Eighth IFAC World Congress, Tokyo. Baron, S., and Kleinman, D. 1969 The human as an optimal controller and information processor. IEEE Trans. Man-Machine Systems MSS-10(11):9–17. Bower, G. 1981 Mood and memory. American Psychologist 36:129–148. Cavanaugh, J.C., and Borkowski, J.G. 1980 Searching for meta-memory-memory connections. Developmental Psychology 16:441–453. Chase, W.G., and Simon, H.A. 1973 The mind’s eye in chess. In W.G.Chase, ed., Visual Information Processing. New York: Academic Press. Chi, M.T.H., Chase, W.G., and Eastman, R. 1980 Spatial Representation of Taxi Drivers. Paper presented to the Psychonomics Society, St. Louis, November. Collis, A.M., and Loftus, E.M. 1975 A spreading activation theory of semantic processing. Psychological Review 82:407–428. Cowey, A. 1979 Cortical maps and visual perception. Quarterly Journal of Experimental Psychology 31:1–17. Ericsson, A., and Simon, H. 1980 Verbal reports as data. Psychological Review 87:215–251. Feldman, J. A., and Ballard, D.H. in Connectionist models and their properties. In press J.Beck and A.Rosenfeld, eds., Human and Computer Vision. New York: Academic Press. Goldberg, L.R. 1968 Simple models or simple processes? Some research on clinical judgments. American Psychologist 23:483–496. Green, D.M., and Swets, J.A. 1966 Signal Detection Theory and Psychophysics. New York: John Wiley. Harvey, J.H., Ickes, W.J., and Kidd, R.F., eds. 1976 New Directions in Attribution Research. Hillsdale, N.J.: Lawrence Erlbaum.

OCR for page 14
Research Needs for Human Factors Kelly, C. 1968 Manual and Automatic Control. New York: John Wiley. Kinsbourne, M., and Hicks, R.L. 1978 Functional cerebral space: a model for overflow transfer and interference effects in human performance. In J.Requin, ed., Attention and Performance VIII. Hillsdale, N.J.: Lawrence Erlbaum. Kok, J.J., and Stassen, H.G. 1980 Human operator control of slowly responding systems: supervisory control. Journal of Cybernetics and Information Sciences 3:124–174. Landman, M., and Hunt, E.B. 1982 Individual differences in secondary task performance. Memory and Cognition 10:10–25. Levison, W.H., and Tanner, R.B. 1971 A Control-Theory Model for Human Decision Making. National Aeronautics and Space Administration CR-1953, December. Navon, D., and Gopher, D. 1979 On the economy of the human-processing system. Psychological Review 86:214–230. Newell, A., and Simon, H.A. 1972 Human Problem Solving. Englewood Cliffs, N. J.: Prentice-Hall. Nisbett, R., and Ross, L. 1980 Human Inference: Strategies and Shortcomings of Social Judgment. Englewood Cliffs, N. J.: Prentice-Hall. Norman, D.A. 1981 Categorization of action slips. Psychological Review 88:1–15. Posner, M.I. 1978 Chronometric Explorations of Mind. Hillsdale, N. J.: Lawrence Erlbaum. 1980 Orienting of attention. Quarterly Journal of Experimental Psychology 32:3–25. Posner, M.I., and Rothbart, M.K. 1980 Development of attentional mechanisms. In J. Flowers, ed., Nebraska Symposium. Lincoln: University of Nebraska Press. Rasmussen, J. 1979 On the Structure of Knowledge—A Morphology of Mental Models in a Man-Machine System Context. RISO National Laboratory Report M-2192. Roskilde, Denmark.

OCR for page 14
Research Needs for Human Factors Sheridan, T. 1981 Understanding human error and aiding human diagnostic behavior in nuclear power plants. In J.Rasmussen and W.Rouse, eds., Human Detection and Diagnosis of System Failures. New York: Plenum Press. 1982 Supervisory Control: Problems, Theory and Experiment in Application to Undersea Remote Control Systems. MIT Man-Machine Systems Laboratory Report. February. Sheridan, T., and Ferrell, W.R. 1967 Supervisory control of manipulation. Pp. 315–323 in Proceedings of the 3rd Annual Conference on Manual Control. NASA SP-144. Sheridan, T.B, and Young, L.R. 1982 Human Factors in aerospace. In R.Dehart, ed., Fundamentals of Aerospace Medicine. Philadelphia: Lea and Febiger. Swain, A.D., and Guttman, H.E. 1980 Handbook of Human Reliability Analysis with Emphasis on Nuclear Power Plant Applications NUREG/CR 1278. Washington, D.C.: Nuclear Regulatory Commission. Tsach, U., Sheridan, T.B., and Tzelgov, J. in A New Method for Failure Detection and press Location in Complex Systems. Proceedings of the 1982 American Control Conference. New York: Institute of Electrical and Electronics Engineers. U.S. Army Research Institute 1979 Annual Report on Research, 1979. Alexandria, Va.: Army Research Institute for the Behavioral and Social Sciences. Young, R.M. 1981 The machine inside the machine: users’ models of pocket calculators. International Journal of Man-Machine Studies 15:51–85. Zajonc, R.B. 1980 Feeling and knowledge: preferences need no inferences. American Psychologist 35:151–175.

OCR for page 14
Research Needs for Human Factors 5 USER-COMPUTER INTERACTION INTRODUCTION Electronic computers have probably had a more profound effect on our society, on our ways of living, and on our ways of doing business than any other technological creation of this century. Computers help manage our finances, checking accounts, and charge accounts. They help schedule rail and air travel, book theatre tickets, check out groceries, diagnose illnesses, teach our children, and amuse us with sophisticated games. Computers make it possible to erase time and distance through telecommunications, thereby giving us the freedom to choose the times and places at which we work. They help guide planes, direct missiles, guard our shores, and plan battle strategies. Computers have created new industries and have spawned new forms of crime. In reality, computers have become so intricately woven into the fabric of daily life that without them our civilization could not function as it does today. Small wonder that all these effects have been described as the results of a computer revolution. Gantz and Peacock (1981) estimate that the total computer power available to U.S. businesses increased tenfold in the last decade, and that it is expected to double every two to four years. According to the most recently available estimates (U.S. Bureau of the Census, 1979), there are currently about 15 million computers, terminals, and electronic office machines in the United States. That number is expected to grow to about 30–35 million by 1985, The principal authors of this chapter are Alphonse Chapanis, Nancy S.Anderson, and J.C.R.Licklider.

OCR for page 14
Research Needs for Human Factors at which time there will be roughly one computer-based machine for every three persons employed in the white-collar work force. Spectacular advances in computer technology have made this growth possible, decreasing the cost of computer hardware at the rate of about 30 percent a year during the past few decades (Dertouzos and Moses, 1980). Computers are still not as widely accepted as they might be. In a study by Zoltan and Chapanis (1982) on what professionals think about computers, over 500 certified public accountants, lawyers, pharmacists, and physicians in the Baltimore area filled out a 64-item questionnaire on their experiences with and attitudes toward electronic computers. Six factors emerged from a factor analysis of the data. Factor I, the largest in terms of the variance accounted for, is a highly positive grouping of adjectives attesting to the competence and productivity of computers, such as efficient, precise, reliable, dependable, effective, and fast. Factor II, the second largest in terms of the variance accounted for, is made up of highly negative adjectives: dehumanizing, depersonalizing, impersonal, cold, and unforgiving. Still another factor in the Zoltan-Chapanis study indicates discontent with computers in terms of their ease of use. The respondents thought that computers are difficult and complicated and that computing languages are not simple to understand. These views are apparent in their responses to such statements as: “I would like a computer to accept ordinary English statements” and “I would like a computer to accept the jargon of my profession,” both of which they agreed with strongly. The findings of that study are generally in agreement with more informal reports in the popular press and other media about difficulties people have with computers and their use. Indeed, concerns about making computers easy to use can have serious economic consequences that may have to be faced by more and more computer manufacturers. For example, a small company in California was recently awarded a verdict for substantial monetary damages because of the inadequate performance of a computer that the company had purchased (Bigelow, 1981). In rendering his opinion substantiating the award, the presiding judge said, “It’s a particularly serious problem, it seems to me, in the computer industry, particularly in that part of the industry which makes computers for first-time users, and seeks to expand the use of computers by…

OCR for page 14
Research Needs for Human Factors targeting as purchasers businesses that have never used computers before, who don’t have any experience in them, and who don’t know what the consequences are of a defect and a failure” (Bigelow, 1981:94). In Europe resistance to computerization has taken a somewhat different form than that in the United States. Television programs roughly equivalent to the American program 60 Minutes have been broadcast about the real and imagined evils of computers. Several countries—Austria, England, France, Germany, and Sweden among them—have prepared strict standards for the design of computer systems and have enacted federal laws restricting hours of work at computer terminals. Similar regulations may soon be in effect in this country. One difficulty is that current standards and regulations about computers are sometimes based on skimpy and unreliable data and sometimes on no data at all (Rupp, 1981). Whatever their origins, these events and trends are symptoms of fairly widespread uneasiness and malaise about computers, their usefulness, and usability. No one denies that computers are here to stay. The important question is: “How can we best design them for effective human use?” This chapter describes some of the research needed to answer that question. Research needs are identified throughout the chapter. However desirable it might appear to assign specific priorities to each, we feel that it is difficult and risky to do so for at least three reasons. First, computer hardware, software, and interface design features are changing very rapidly (for a summary of the trends and progress in computer development see Branscomb, 1982). So, for example, the increased availability of modularly arranged components for microcomputers for personal use, in the office and at school as well as new networking and communications features allow design improvements to be made quickly by trial and error. As Nickerson (1969) has pointed out, such trial-and-error design improvements can be made more quickly than they could be by careful laboratory research studies. Second, practical considerations are likely to be significant determinants of what research can be performed. Operational computer systems rarely can be disrupted for research purposes, and up-to-date hardware and software as well as appropriate groups of users are not always available. Under these circumstances it takes great ingenuity to conduct human factors research on user-computer interactions that can produce useful, generaliz-