Emerging Needs and Opportunities for Human Factors Research
This summary report identifies the areas that the Committee on Human Factors believes represent new needs and opportunities for human factors research during the next few decades. It is organized as follows: after a background discussion, we describe the process by which the committee determined which topics to cover. Each topic is then considered in turn, and the committee's consensus regarding needed research is presented.
A set of papers addressing these topics appears as Part II of this volume. These papers were written by committee members, sometimes with assistance from colleagues, as part of the process of informing the committee's discussions of the topics addressed. The conclusions and recommendations articulated in this report draw heavily on these papers.
The National Research Council established the Committee on Human Factors in 1980. The committee's original sponsors were the Office of Naval Research, the Air Force Office of Scientific Research, and the Army Research Institute. The National Aeronautics and Space Administration became a sponsor in 1981. The committee's charter was to identify basic research needs of the military services as they relate to human factors issues and to make recommendations for basic research that would improve the foundations of the discipline.
The committee's first report, Research Needs for Human Factors, published in 1983, focused on six topics: human decision making, eliciting expert judgment, supervisory control systems, user-computer interaction, population group differences, and applied methods. These topics were selected after committee discussions of research needs, tours of military laboratories, and solicitation of suggestions from the human factors community through an article in the Human Factors Society Bulletin. Topics were selected because they were germane to the sponsors' interests, within the expertise of the committee members to address, incompletely addressed by previous or ongoing research, and important vis-à-vis the committee's charter.
Since the publication of Research Needs for Human Factors, the committee has been responsible for numerous panels and workshops, many of them about problem areas discussed in that report. Figure 1 gives a complete list of the reports produced by these panels and workshops.
Since the committee's establishment, its sponsorship has broadened considerably,
Organizational Linkages: Understanding the Productivity Paradox
Workload Transition: Implications for Individual and Team Performance
Human Factors Specialists' Education and Utilization: Results of a Survey
Application Principles for Multicolored Displays: A Workshop Report Quantitative Modeling of Human Performance in Complex, Dynamic Systems
Distributed Decision Making: Report of a Workshop
Human Factors Research Needs for an Aging Population
Human Performance Models for Computer-Aided Engineering
Fundamental Issues in Human-Computer Interaction
Human Factors Research and Nuclear Safety
Ergonomic Models of Anthropometry, Human Biomechanics, and Operator-Equipment Interfaces: Proceedings of a Workshop
Human Factors in Automated and Robotic Space Systems: Proceedings of a Symposium
Mental Models in Human-Computer Interaction: Research Issues About What the User of Software Knows
Human Factors Aspects of Simulation
Methods for Designing Software to Fit Human Needs and Capabilities: Proceedings of the Workshop on Software Human Factors
Research Needs on the Interaction Between Information Systems and Their Users: Report of a Workshop
Research Issues in Simulator Sickness: Proceedings of a Workshop
Research and Modeling of Supervisory Control Behavior: Report of a Workshop
and new sponsors have come from both military and civilian sectors. The committee's composition has also broadened to include not only researchers who represent traditional human factors interests in equipment design and use but also some with more cognitive and social orientations.
Concurrently, technology has been advancing rapidly, and both the nation and the world have changed in significant and remarkable ways. Some of these changes offer new challenges and opportunities for human factors research.
In view of these developments, the committee found it appropriate to again address the general topic of research needs and opportunities for human factors. Committee members agreed that the report should be:
reflective of the committee's views and opinions, and also informed by inputs from several sources;
forward looking: short on reviewing old and current work and long on identifying problems and opportunities for the future;
problem/opportunity-oriented: the subject matter being determined more by what the committee perceives the needs and opportunities to be than by its understanding of what the existing research activities of the human factors community are; and
selectively focused—with no attempt to be comprehensive—on a few major topics that the committee believes to be among the more important problem/opportunity areas for the near future.
The process of selecting topics for emphasis was lengthy and deliberative. The decision was made early to cast a wide net for ideas and opinions and then, through committee discussion and debate, to attempt to reach a consensus regarding a subset on which to focus. Suggestions of major research needs were solicited from current and former members of the committee, from committee sponsors, and from numerous other members of the human factors research community, including several human factors leaders in Europe. The committee also reviewed the topics generated as candidates for inclusion in the original Research Needs report, the list of technical groups of the Human Factors and Ergonomics Society, the list of titles of Committee on Human Factors reports published or in process, and recent project suggestions from sponsors. This yielded a long list of suggested research topics.
As the list grew, the committee discussed it at several meetings. At one such meeting, an analysis of responses to a written request for suggestions from past and current committee members was presented and discussed. A complete list of all suggestions from all sources was then distributed to committee members, and at a later meeting, the members engaged in a structured exercise designed (1) to permit every member to identify all the suggestions he or she considered worthy of further consideration, (2) to facilitate grouping the suggestions to eliminate redundancies and replace similar suggestions with a more general category, and (3) to prioritize the resulting list through a quasi-formal, iterative voting procedure.
At all of these meetings, the discussions were long and spirited. All members saw the goal of identifying human factors research needs and opportunities as very important. However, the task of selecting a few problems or problem areas for special attention was difficult because selecting some areas for inclusion meant excluding others.
The committee's deliberations eventually led to a consensus on the topics that should be covered in its report. Continuing discussion resulted in some minor changes in terminology in the interest of clarity and the merging of some topics because of content overlap. Some of the selected topics are well within the mainstream of traditional human factors research; others are not. The committee intentionally took a relatively broad view of human factors and did not exclude a problem area simply because it has not traditionally been a major focus of the discipline. As it happens, the topics are fairly easily grouped under three major headings—national or global problems, technology issues, and human performance—as follows:
National or global problems
Productivity in organizations
Training and education
Employment and disabilities
Communication technology and telenetworking
Information access and usability
Emerging technologies in work design
Cognitive performance under stress
Aiding intellectual work
Two topics the committee considers to be priority research areas—human performance modeling and human error—are not covered in this
volume, because committee panels were established on both. A report on human performance modeling was recently published (Baron et al., 1990) and one on human error management in high-hazard systems is currently in preparation. With these exceptions, the topics on which the committee finally settled constitute the titles of sections of this summary report and of the papers that make up Part II.
The committee believes that these areas are, without exception, very important. However, failure of a topic to appear as a heading should not be taken as evidence that the committee considers the topic unimportant. The committee recognizes that many more human factors problems and problem areas are deserving of research than can be covered in a report of this sort.
This report focuses on needs and opportunities for human factors research; however, the committee also wishes to note that one of the most pressing needs at present, and probably during the near future, is to get the results of human factors research applied to equipment, procedures, systems, and situations. It continues to be easy to find equipment and operating procedures being designed in ways that are inconsistent with well-documented human factors research results. How to improve the dissemination of the results of human factors research, how to make this information readily available to users in a helpful form, and how to make potential users aware of its existence, are continuing major challenges to human factors researchers and to everyone who recognizes the potential usefulness of the results of human factors research.
This report is intended for people who identify research funding priorities and sponsor research programs and projects and for those who perform human factors research. People responsible for identifying research funding priorities and sponsoring research efforts will find here the collective opinions of the committee on how needs and opportunities for human factors research relate to several problems of major national concern. Researchers will find specific questions that the committee believes constitute challenges to research that could contribute significantly to the solution or amelioration of these problems.
In keeping with the committee's decision to be forward-looking, little effort was made to review research extensively in problem areas that are or
have been major foci. A very modest amount of reviewing is done in this summary report and somewhat more in Part II, but, in both cases, only to help provide an appropriate frame of reference for thinking about future needs.
Some of the committee's recommendations are relatively general; others, quite specific. Some follow traditional lines of research; others relate to problems that have not received much attention from the human factors community in the past. In the latter cases, the recommendations are offered primarily as points of departure for further discussion and planning rather than as items for a research agenda. The justification for including them is the perceived seriousness, from either a national or a global perspective, of the problem areas involved.
The opinions expressed in Part II are those of the authors of the papers, but their shaping has been strongly influenced by inputs from other members of the Committee on Human Factors and the other sources of ideas mentioned above. The recommendations that appear in this summary report are made by the committee as a whole.
The remainder of this summary report is organized by topic. A section is devoted to each of the topics selected by the committee for emphasis. In each case, the section describes the problem, briefly discusses representative previous work that relates to the problem, and gives a set of recommendations for research.
PRODUCTIVITY IN ORGANIZATIONS
Productivity is a major national and international concern. Economists see productivity as a primary determinant of competitiveness both among companies within an industry and among national economies. Because it is also believed to be causally linked to standard of living, increasing productivity globally is seen to be the best hope of improving living conditions worldwide.
Although the United States is the most productive country in the world, its annual rate of increase in productivity has been considerably smaller than that of several other industrialized countries during the last few decades (Bureau of Labor Statistics, 1988a, 1988b). As a consequence, our national competitiveness has decreased significantly in the automobile, steel, shipbuilding, and textile industries and appears to be slipping in electronics, computers, robotics, and biotechnology as well. The U.S. share of the total
world economy decreased from 35 percent in 1965 to 28 percent in 1985 (Johnston and Packer, 1987).
This trend can be attributed to many factors, such as the diffusion of technology, the globalization of work, and the development of new work processes and new work structures throughout the world. Many economists, however, are concerned that productivity in the United States is not what it should be and that the nation's competitiveness will continue to decline if ways are not found to accelerate the rate of productivity increase.
Human factors researchers have given considerable attention to the question of how to improve human performance in the workplace and thereby increase individual productivity. Yet they have made relatively little effort to determine how individual productivity relates to the productivity of the groups, organizations, or industries within which the individuals' work is done. Our focus is on organizational productivity and how the human factors community can contribute to improving it.
The prevailing assumption appears to be that increases in individual productivity automatically translate into increases in productivity at higher levels of organization, but this assumption gets little support from research. There is little evidence that increases in productivity at one level, say the level of the individual worker, automatically translate into increases in productivity at a higher level, say that of a work team, a corporation, or an industry. Similarly, there is very little theory or research that helps us understand how observed changes in organizational productivity relate to changes at the individual or group level.
The challenge is to better understand organizational productivity. We want human factors researchers to think about their work in terms of improvement in organizational productivity.
Most of the research on productivity has dealt with productivity at a particular level—individual, group, corporation. The research that has been done by applied psychologists and human factors specialists has dealt primarily with the individual. Training has been identified as an important determinant of individual productivity (Guzzo, 1988), along with goal setting (Locke and Latham, 1990) and the details of the design of specific tasks (Guzzo, 1988).
At the level of work teams or small groups, self-management has received some attention as a determinant of productivity (Goodman et al., 1988; Hackman, 1990). Several studies of larger systems have focused on the effects of the introduction of automation technologies in the workplace and, interestingly, have not found that greater automation invariably means higher productivity at the system level. In particular, technological change
often has had little or no positive effect on system productivity unless appropriate organizational change has accompanied it (Goodman, 1979; MacDuffie and Krafcik, 1990).
Although the previous research has provided some knowledge of the determinants of productivity at specific levels of organization, very little of it has been directed at determining how improvements at one level do or do not effect changes at higher levels of operation. In particular, we know something about how to increase the productivity of individuals, but we are unable to specify the conditions under which improvements at this level will translate into improvements in the productivity of an organization as a whole.
Workplace interventions that appear to have the potential to increase productivity not only at the level of the individual worker but also at that of the organization as a whole are sometimes not implemented successfully (Goodman and Griffith, 1991). There is a need for better understanding of the implementation process and, in particular, of how to ensure that innovations that would increase individual and organization productivity will be adopted and used.
One opportunity for research is the exploration of how changes in organizations or in technology are related to improved productivity. Billions of dollars have been spent on new forms of technological and organizational change, many of which have clear or inherent benefits. There is, however, growing research evidence that many innovations are not successfully implemented or are implemented on a temporary basis. As a result, the objective benefits of the proposed technological and organizational changes are canceled out. If we cannot successfully implement improvements at the individual, group, or organizational level, we cannot expect to see productivity changes at the organizational level.
The challenge, then, is to find new research approaches to successful implementation and institutionalization. Much of the work over the past decade is on how different variables (e.g., top management support) facilitate the change process. This work suggests that new developments will come from identifying the critical processes that drive the implementation. Furthermore, a change in methodology is required—for example, there are very few multivariate and/or longitudinal studies capturing the implementation of new technologies or organizational intervention.
Another opportunity for research is the linkage between individual and organizational productivity. Focusing on single levels of analysis (e.g., individual, group) will not increase our understanding of organizational productivity. We assume that successful implementations create productivity
increases at the individual level and that these changes will eventually appear at the organizational level. This assumption is incorrect—changes at one level may have no effect on other levels of analysis. There is neither good theory nor research about this linkage issue.
The challenge for the human factor researcher is first to acknowledge and understand the issues associated with these linkages. What are the factors that prevent productivity changes at the individual level from increasing productivity at the individual level? What are some of the enablers? Concepts such as organizational slack and different forms of organizational interdependence are starting points of our analysis. Increasing productivity on some jobs can simply create slack time that does not translate into increased organizational interdependence. The degree and form of interdependence between jobs and between organizational units may to a large degree determine whether changes in one unit will lead to positive, negative, or no changes in other units. Horizontal and vertical linkages are critical in determining whether changes in productivity at one level get translated to another level.
A third research opportunity is the congruency among technology, people, and organizational factors. It seems inevitable that organizational productivity must depend, in part, on the ways these factors interrelate. The empirical evidence supports the view that focusing solely on one factor (e.g., productivity) will not increase organizational productivity; rather it is the simultaneous restructuring of organizational, technological, and people factors that is key to increases in organizational productivity.
Although we know that congruency among these three factors is important, much of our research evidence is after the fact. We do not have any ex ante models about how combinations of organizational, technological, and people factors lead to increased organizational productivity. This is an important issue because the workplace is changing rapidly. All indications are that rapid change in technology, organizations, and characteristics of the workforce will continue. The opportunity for human factors researchers is to provide new insights into how combinations of changes in technology, organization, and people factors contribute to organizational productivity.
A fourth opportunity for research concerns internal and external integration. New innovations in design processes and linking production processes in manufacturing and retail/distribution organizations signal the importance of internal integrations. At the same time, there have been dramatic changes in relationships with suppliers and customers. ''Arm's-length" transactions are a thing of the past. Boundaries between customers, suppliers, and the focal organization have become very permeable.
Although these forms of integration are initiated with the expectations of improving organizational productivity, little research is available. Consider the following: in our analysis of organizational productivity, we generally
focus on linkages among levels (e.g., individual and group) within the organization. But let us assume that greater levels of integration between the organization and its customer increase productivity for the customer. The interesting question is: How do changes in productivity for the customer feed back and affect productivity of the focal organization?
TRAINING AND EDUCATION
The critical importance of worker training to the national economy and to the general well-being of the nation is widely recognized. It is also recognized that the educational level of prospective trainees strongly influences the efficiency and effectiveness of training. Unfortunately, neither our education nor our training systems, as currently structured, appear to be meeting the country's needs. A key symptom of the problem is the growing number of programs in which employers attempt to provide remedial education for incoming employees as a way to build a base upon which specific, job-oriented training can proceed.
The seriousness of the education and training problem has been documented by numerous published studies, including the widely cited A Nation at Risk (National Commission on Excellence in Education, 1983) and subsequent reports by the Hudson Institute (Johnston and Packer, 1987), the William T. Grant Foundation (1988), the Commission on Workforce Quality and Labor Market Efficiency (1989), the Commission on the Skills of the American Workforce (1990), the Office of Technology Assessment (1990), and the National Council on Education Standards and Testing (1992).
The problem could become worse in the future because the workforce will need to be even more proficient—especially more versatile and adaptive—than it is now. This expectation is rooted in the rapidity of technological change and the increasingly tight interconnectedness of the economies of the world into a single global ensemble. Technological change means changes in job requirements. The ability to satisfy changing, and not entirely predictable, job requirements in a complex, culturally diverse, and constantly evolving environment will require a literate workforce that has good problem-solving and learning skills. The challenge to education and training will be amplified by expected changes in the demographics of the workforce. In the aggregate, these changes mean that an increasing percentage of the labor needs will be met by nontraditional sources; in some cases, special training will be required to tap these sources effectively. Furthermore, because advances in technology tend not only to create new jobs but also to increase the speed with which many existing jobs become obsolete, the need for worker retraining will become increasingly common.
A great deal of research on education and training has been done by psychologists and educational researchers. Inasmuch as our focus is primarily on postsecondary and work-based education and training, it suffices to mention here the following publications that, taken together, provide extensive coverage of research on the design, delivery, and evaluation of training methods and systems: Druckman and Bjork (1991), Druckman and Swets (1988), Goldstein (1988, 1992), Wexley (1984), Latham (1988), and Tannenbaum and Yukl (1991).
Unfortunately, many of the basic research findings on learning and skill development do not cross the boundary between the laboratory and the workplace. As a discipline that historically bridges psychology and engineering, human factors should be able to help transfer what has been discovered in the laboratory to the design of education and training systems. How best to do this is itself a question that will require intensive study.
Training and human factors are sometimes characterized as complementary ways of getting human-machine systems to function smoothly, the idea being that the objective of human factors research should be to determine how to design systems so that little, if any, training is required for the system to function effectively. Here we take the broader view that human factors encompasses whatever enhances the performance of human-machine systems, including training. However, even within the traditional, narrower characterization of human factors, many questions arise regarding the design and evaluation of training materials, methods, and systems. The research challenges that are identified in the following section arise from both modes of characterizing the field.
Research Needs and Opportunities
Training systems tend to be complex both functionally and structurally; they have to work under numerous constraints to achieve multiple objectives. Many variables are involved in the operation of such systems. Understanding the interactions among these variables requires using a systems-analytic approach that human factors researchers have effectively applied to the study of many other kinds of complex systems. The challenge is to apply such methods to the study of training when outcome requirements arise from interactions among changing technologies, changing workforce demographics, and changing organizational structures.
Technology has the potential to change the methods of education and training drastically. Computer and communication technologies, in particular, provide the possibility of new approaches involving interactive graphics, process simulation, individualized adaptive training, embedded training,
and a host of other innovations. Each of these methods, however, is valuable only to the extent that it contributes to accomplishing the teaching/learning objectives of the training system into which it has been incorporated. There is a need for evaluations of technologically innovative approaches to education and training that reveal not only how effective the approaches are but also how they might be improved. Human factors researchers can perform evaluations of the type required.
An example of a type of evaluation that is much needed is a study to determine how much (and what type of) physical fidelity is required in a system simulation generated by virtual reality. Inasmuch as small increases in physical fidelity can mean large increases in costs, it is important to know how much fidelity is enough in specific cases.
The human factors community has promoted user-centered design—designing systems to match the capabilities and limitations of their intended users—and has widely applied it to the design of many kinds of systems, especially, but not exclusively, in military contexts. User-centered design principles that have proved to be effective should be applied to the design of education and training systems as well. The challenge is to devise, evaluate, and perfect ways to transfer the methods, data, and principles of human-centered design to the development of training systems and to ensure that feedback from applications closes the loop between research and practice.
Technology has had two particularly striking effects on jobs during the last few decades: (1) a decrease in the manual component of many jobs accompanied by an increase in the cognitive component and (2) acceleration of the pace of change in the workplace. The probability that these trends will continue establishes a need for a greater research focus on how individuals and organizations can best learn to adapt to rapid changes in work procedures and organizational structures while continuing to be engaged in production processes. There is need for the development of theory, tools, and techniques that will support lifelong learning in many occupations.
Because of the increasing rapidity with which jobs and the skill requirements of jobs for them are changing, it is also becoming increasingly important to have effective methods for anticipating future job requirements so that education and training programs can be designed to provide workers with the requisite skills in a timely way. Formal procedures used by human factors researchers to identify or analyze the skills, knowledge, and abilities that are requisite to the adequate performance of specific tasks should be useful in establishing education and training goals, but there is also a need for the development of more effective methods of identifying cognitive, as opposed to psychomotor, task demands.
Despite the widely acknowledged imperfection of current education and training systems and programs and the considerable amount of money and
effort that continues to be spent on developing them, the technology for evaluating their effectiveness is still primitive and not very sensitive. A major practical need is for significant improvement in the means for performance assessment. Human factors researchers have experience in evaluating the effectiveness of complex systems of many types and thus should be in a position to help meet this need.
EMPLOYMENT AND DISABILITIES
The population of people with one or another type of physical or cognitive disability is very large. According to the U.S. Census Bureau, there are about 75 million people with some sort of disability in the United States, about 30 percent of the entire population (Bureau of the Census, 1987, 1989). Not all of these disabilities are sufficiently severe to interfere with work or other activities, but, if we count only those that are, the number is still large, probably as many as 30 million in the United States (Elkind, 1990).
The rate of unemployment among people with disabilities who would like to work is several times higher than the unemployment average nationwide (Kraus and Stoddard, 1989). And the percentage of people who live below the poverty line is between two and three times greater for people with a disability that interferes with their ability to work than for the total working population (Vachon, 1990).
Failure to find more effective ways to make employment available to people with disabilities is costly to the national economy in two ways. The cost of maintenance support programs is approximately $100 billion per year and is increasing rapidly (Vanderheiden, 1990), and the cost of the country's lost opportunity to utilize the knowledge and skills of people with disabilities, although difficult to quantify, is surely also great.
Most of the previous research that directly relates to enhancing employment opportunities for people with disabilities has aimed at finding ways to use technology to mitigate the limiting effects of disabilities. The objective in many cases was the development of a device of some sort to help people compensate for the loss of sight, hearing, mobility, or other specific functions.
For at least two reasons, much of this work has focused on computer and communication technologies. First, the versatility of the computer provides opportunities for creative approaches to the augmentation of abilities
that have been diminished by physical or cognitive disorders, disease, or accident; this has been amply demonstrated by the innovative ways that people with disabilities have adapted these technologies for their own use (Bowe, 1984). Second, because more than half the workforce is now engaged in information-oriented jobs (Strassmann, 1985) and the percentage is expected to increase in the future (Kraut, 1987), it is also natural to look to computer and communication technologies as a source of job opportunities for people with disabilities.
Some emphasis on computer and communication technologies to enhance work opportunities for people with disabilities in the future is warranted and for the same reasons. We believe that these technologies offer great, and largely unrealized, potential to help remove barriers that have kept people with disabilities from jobs. We also believe that many, if not most, of the new job opportunities that could be readily made available to people with disabilities are likely to be in areas that make heavy use of these technologies.
Unfortunately, most of the research on how the requirements of specific jobs relate to the capabilities and limitations of prospective jobholders has excluded people with disabilities from consideration. Data have almost invariably been collected only on able-bodied people; people with disabilities are not even statistically represented in the results. As a consequence, little is known of how the capabilities and limitations of people with specific disabilities compare with the requirements of specific jobs. A requirement in this context is something that is essential to getting a job done; job requirements are to be distinguished from the architectural, social, and other barriers that hinder a person from getting to a job location or situation but are not essential aspects of the job itself.
Research Needs and Opportunities
Most disabilities are impediments only to certain activities, and typically many fewer activities are affected than are not. Specifically, any given type of disability usually impedes only some of the activities required by some jobs, and often even those impediments can be removed or overcome by restructuring the job or using assistive technology. A major need is for a much better understanding of precisely how the capabilities and limitations associated with specific disabilities relate to the functional requirements of specific jobs. This need suggests the following two objectives:
development of a database of information on the performance capabilities of people with the more prevalent types of disabilities, with a view
to incorporating such information in design references, design texts, and in human performance models, thereby enhancing the ability to design for people with disabilities; and
systematic analysis of the skill requirements of jobs, especially jobs in the information sector, to match requirements with the performance capabilities of people with disabilities. A clear distinction should be made between essential job functions and nonessential aspects of job situations that may have evolved as conveniences to able-bodied individuals who typically perform them.
Some jobs cannot be done by people with certain types of disabilities in the same way in which they are standardly performed by able-bodied people. It is important to know whether those jobs could be performed just as effectively in alternative ways that would be manageable by people with disabilities. A useful first step would be an extensive study of previous efforts to redesign jobs to accommodate people with specific disabilities; such a study should identify differences between efforts that have worked and those that have not.
As was noted already, the development of devices to assist people with disabilities in overcoming specific functional limitations has received considerable attention, especially from scientists and engineers who may be keenly aware of the kind of assistance that is needed because they are disabled in some way or because someone close to them is. Human factors researchers have been somewhat involved in these efforts, but this has not been a major focus of the discipline as a whole. The discipline has much to contribute to this design challenge and much to learn from involvement in this type of work.
Unfortunately, knowledge of much of what has been done to apply technology to benefit people with disabilities is not readily accessible, because the work was not all done by people who are members of the same professional community for which the channels of communication are well established and known. Much of the information exists in relatively inaccessible documents as opposed to mainstream technical journals. It can be very hard to obtain detailed information on a particular assistive device or type of device—or even to discover whether a device to serve a specified purpose exists. A very useful objective would be to develop a set of computer-based information resources on assistive devices, training programs, job opportunities, and job requirements for people with disabilities. Such resources could also serve the research community, making information on past and ongoing research much more readily accessible than it currently is.
Health risks stem from many sources. Prominent among them are behavioral and occupational factors. Cancer and heart disease, for example, are often attributable to smoking, diet, or other lifestyle variables (Newell and Vogel, 1988; Williams, 1991). Long-term occupational exposure to radiation or toxic substances is known to be a significant cause of certain types of illness, as is chronic job-related stress (Levi, 1990; Smith, 1987; Swanson, 1988).
Paradoxically, significant threats to health are to be found even within the technology for the delivery of health care. Many of these threats arise from the possibility of human error in using health care equipment and carrying out health care procedures. Such errors can have tragic consequences. For example, Bogner (1991) has cited the substantial number of potentially preventable incidents in which anesthesia has resulted in brain damage or death.
Much of the equipment in modern hospitals and trauma centers is complex, and even highly trained people can inadvertently use it incorrectly with injurious effects. In addition, more and more commonly, people without medical training are using complicated equipment, including ventilators, infusion pumps, and kidney dialysis machines, in their homes; these devices can extend and enhance the quality of life when used correctly, but they can cause irreparable harm when used erroneously.
Injurious errors can also occur in the administration of drugs, either by administration of the wrong substances or incorrect doses of the right ones. This problem too is compounded by the fact that patients are often responsible for their own drug regimens and are sometimes confused as a consequence of the effects of illness or age. An important challenge to research is developing ways to reduce the incidence of human error in the delivery of health care; this issue is likely to grow in importance as the technology that is available for health care delivery becomes ever more complex.
Although there have been attempts to call attention to the many challenges that medicine and health care delivery provide for human factors researchers (e.g., Pickett and Triggs, 1974), health care has not traditionally been a major focus of human factors research.
The human factors community has done considerable research both to determine the causes of human error in a variety of situations in which people interact with machines and to identify ways to decrease the probability
of such errors or reduce harm when they do occur (Reason, 1990; Senders and Moray, 1991). Relatively little of this work has been done in medical or health care contexts.
Innovative work has been done on evaluating the relative effectiveness of medical imaging devices (Swets et al., 1979; Swets, 1988) and on human factors aspects of the design of imaging workstations (O'Malley and Ricca, 1990). Some human factors evaluations have been done of glucose measuring devices of the type used by people with diabetes (Kelly et al., 1990; McDonald, 1984; Moss and Delawter, 1986). Such studies have revealed a number of problems with the use of even these devices, which are relatively simple compared with many that are likely to become common for home use in coming years.
Research Needs and Opportunities
Accident prevention and wellness maintenance are important aspects of health care. As the population continues to age and as more and more elderly and very elderly people are able to live independently, an important objective of human factors research will be to determine how to design living environments that limit the risk of accidents without sacrificing the functionality that residents need or want. An aspect of this challenge that deserves increasing attention is designing living spaces for the elderly that can accommodate the types of health care devices that are more and more commonly utilized in the home.
Research in several other areas would also enhance the opportunities of people to live relatively independently, despite various types of sensory, psychomotor, or cognitive limitations. These include developing and evaluating memory aids to help people take medicines in proper dosages on schedule; improving the interpretability of labels, warnings, and instructions on medicines and on health care devices that are intended to be used by nonprofessionals in the home; and developing and evaluating practical approaches to training in the use of such devices.
Important questions for research are how to convey to nonspecialists accurate and understandable representations of (1) the health risks (or benefits) associated with specific behaviors and (2) the prospects for the various possible outcomes from specific types of treatments of their medical problems. More generally, the challenge is to find effective ways both to convey the information that people need in order to make truly informed decisions about their health care and to ensure that the information conveyed has been understood.
The need for group programs to increase positive health practices offers an opportunity for human factors involvement in health-related work. Programs to reduce smoking, decrease fat intake and increase fiber, reduce
recreational sun exposure, and induce participation in screening programs for such diseases as breast and colon cancer should have beneficial results. Human factors, broadly defined, could help both in designing such programs and in evaluating their effectiveness.
The designs of medical devices need to be evaluated for usability and safety, just as do the designs of other devices. Such evaluations are especially important for medical devices because a patient's well-being often depends on the device's being used effectively.
Information systems—medical databases, diagnostic and decision aids, expert systems, and computer-based advisers—are becoming increasingly available for medical practitioners of all types. These tools differ greatly in how well they meet adequate design standards from a human factors point of view, and the amount of use they get varies greatly among potential users. Inclusion of the design features that have traditionally been considered essential to usability for a computer-based device or system appears not to be sufficient to ensure acceptance or use by health care providers. Research is needed to learn why potentially helpful computer-based systems are or are not used.
Among the many spectacular developments in medical technology during the recent past is the wealth of new imaging techniques (e.g., computed axial tomography, positron emission tomography, magnetic resonance imaging) that permit noninvasive visual exploration of the body's internal organs and structures. In part because these techniques are very expensive, it is important to understand exactly what advantages they offer and how useful they are for diagnosis, compared with one another and with more traditional diagnostic techniques. The appropriate cost-benefit comparisons require the use of psychophysical and statistical approaches of the type that have been widely used by human factors researchers and applied psychologists on similar comparison problems and on this problem as well (Swets et al., 1979; Swets, 1988).
Another challenge to the human factors community is helping find ways to reduce the frequency and severity of the injuries suffered by health care professionals in the performance of their jobs. Back injury, for example, appears to be an occupational hazard for nurses and nursing aides, who frequently put unsafe levels of stress on the spine in performing their work (Gagnon et al., 1986; Stubbs et al., 1983). Both training and the development of more effective devices to assist in lifting tasks could help remediate this problem.
Human factors, especially that part of the field that focuses on biomechanics, has much to offer not only in preventing the chronic pain and disabling injuries that can result from inappropriate physical stresses in work situations, but also in rehabilitating people after an injury and preventing
its recurrence (Khalil et al., 1990; Rosomoff et al., 1981). Developing better biomechanical models could be an important part of this effort.
Although the problems of harmful environmental change are not usually associated with human factors research, the topic is included in this report because the committee considers it a serious national and international problem and believes that human factors research could contribute to a better understanding of, and perhaps solutions to, aspects of it.
Concern about the threat that certain changes in the environment pose for the future has been growing among scientists, policy makers, and the public. Aspects of this threat include the possible impact on the world's climate of an increasing concentration of carbon dioxide and other ''greenhouse gases" in the atmosphere (Houghton and Woodwell, 1989; National Research Council, 1983); the acidification of precipitation and its effects on lakes and streams, forests, and materials (Baker et al., 1991; Mohnen, 1988; Schwartz, 1989); air pollution and urban smog (Gray and Alson, 1989; National Research Council, 1991; Office of Technology Assessment, 1988); thinning of ozone in the stratosphere (Stolarski, 1988; Stolarski et al., 1992); contamination and depletion of fresh water supplies (la Riviere, 1989; National Research Council, 1977; Postel, 1985); depletion of the world's forests (Myers, 1989; Repetto, 1990) and wetlands (Steinhart, 1990; Wallace, 1985); and an accompanying decrease in biodiversity (Soule, 1991; Wilson, 1989) and worldwide loss of arable land (Crossen and Rosenberg, 1989; National Research Council, 1990; Schlesinger et al., 1990).
Human activities are believed to be among the major causes of these threats. The burning of fossil fuels contributes to the atmospheric accumulation of greenhouse gases, air pollution, and the production of acid rain. The use of chlorofluorocarbons as coolants and aerosols is believed to be a cause of the thinning of stratospheric ozone. Water contamination results from runoff from agricultural use of fertilizers and pesticides, from improper disposal of toxic wastes, and from salt used for roadway deicing. The list of ways in which human activities affect the environment is easily extended (Stern et al., 1992).
Because these activities are aimed at satisfying human needs and desires, it is reasonable to assume that they will increase as world population grows. Many of these activities have been much more common in the industrialized world than in underdeveloped countries, because they are associated with industrialization and the products of technology. These activities are expected to grow even more rapidly as countries in many of
the underdeveloped regions of the world try to accelerate their rate of technological development.
Psychologists have done much research on how to get people to modify environmentally harmful behavior to make it more environmentally benign (Holahan, 1986; Russell and Ward, 1982; Saegert and Winkel, 1990; Stern, 1992). This work includes studies of the use of incentives, rewards, education and information campaigns, persuasion, and other techniques to motivate people to conserve energy or water, participate in recycling programs, generate less waste and decrease littering, and make other changes in their behavior that would be desirable for environmental preservation (Baum and Singer, 1981; Coach et al., 1979; Cone and Hayes, 1980; Geller, 1986; Geller et al., 1982).
These studies have demonstrated that the desired types of behavior change can be effected through the techniques that have been used. Unfortunately, studies that have checked for the persistence of the new behavior much beyond the end of the intervention period have generally not reported positive results (Geller et al., 1982).
A complementary approach is to try to change technology so as to maintain its effectiveness in meeting human needs while decreasing the opportunities it affords for harming the environment. Except for work aimed at identifying causes of industrial accidents, which often have substantial environmental impact (Reason, 1990; Senders and Moray, 1991), this approach has not received much attention from researchers. It is, however, one the human factors community should be well suited to take. We believe there is a need both for continuation of research on ways to motivate people to behave in environmentally benign ways and for much greater attention to the possibility of shaping technology so that the natural consequences of its use will be less environmentally damaging.
Research Needs and Opportunities
Inasmuch as many of the most significant threats to the environment are direct consequences of the production and use of energy, high-priority objectives must include improving the efficiency of energy use and substituting forms of energy that are not harmful to the environment for those that are. Demands for energy can be decreased whenever goals traditionally served by transporting people and material can be served equally well by transmitting information. Computer and telecommunications technologies have the potential, through such media as electronic mail, teleconferencing, and computer-supported cooperative work facilities, to reduce the need for
travel for certain purposes considerably. The extent to which the potential of these technologies is realized in this regard will depend on their acceptability to industry and potential users. A challenge to human factors researchers is to help ensure that such facilities are well designed from the users' point of view; there is, however, also a need for research that will lead to a better understanding of what factors, in addition to interface design, make these systems more acceptable to potential users and of how to facilitate the transition to their use.
Making mass public transportation more attractive, so that it is more often used as an alternative to private automobiles in urban areas, is also in the interest of environmental protection because it will reduce the energy expenditure per passenger mile traveled. This could also help reduce noise pollution, traffic congestion, traffic accidents, and other problems that attend excessive traffic in urban areas. Greater use of carpooling could have similar beneficial effects.
Paper and paper products account for about one-third of all the solid waste produced in the United States. Much of the paper that becomes a waste disposal problem is used to store and distribute information via newspapers, magazines, books, and other print media. The technology to store and distribute much of the information electronically exists and is rapidly becoming available to the public. A challenge to research is to determine how to design the display and input devices that provide people with access to electronic information so that they are acceptable, if not preferred, substitutes for paper.
"Virtual reality" technology—which has been receiving a lot of attention in the press recently, in part because of its potential use in entertainment and education—could have implications for environmental issues. For example, to the extent that virtual systems can be effectively substituted for the sometimes material-and energy-heavy systems that have been required in the past for training purposes, the environment should benefit. Determining how real a virtual reality must be to be effective for specific training purposes is largely a human factors question.
A variety of other challenges for human factors research relate to the problems of waste reduction and waste management. One need is for the development of design criteria for consumer goods that include—in addition to functionality, usability, and user safety—such considerations as longevity, maintainability, and recyclability or disposability. Another is for improvement, from a human factors perspective, of recycling and waste management. The handling of radioactive and toxic wastes poses some especially difficult problems that human factors researchers should be able to help solve.
Other human factors challenges that would help the environment include making the information in environmental databases more accessible
to users and potential users who would use that information for research on, and the monitoring and predicting of, environmental change; continued study of the problems of risk assessment, management, and communication; and continued emphasis on designing industrial systems to reduce the probability of human error, especially when it can significantly harm the environment.
The problems of environmental change has not received a lot of attention from the human factors research community in the past, and there has been relatively little debate or discussion about what the community has to offer toward solutions in this area. We believe that human factors research could have an impact on various aspects of these problems. The suggestions made here are but a few of the possibilities; they are offered in the hope of stimulating more thought and discussion by which others would be identified.
COMMUNICATION TECHNOLOGY AND TELENETWORKING
Since the experimental establishment of the first networks linking computers from different geographical locations in the mid-1960s, network technology has advanced very rapidly. The ARPANET, which became the largest operational network in the world and remained so for many years, was started as a four-node system by the Advanced Research Projects Agency of the U.S. Department of Defense in 1969 (Heart, 1975; Heart et al., 1978). According to a recent Science report, its successor, the Internet, now connects about 1.7 million host computers and between 5 and 15 million users, and the numbers are doubling annually (Pool, 1993). It now appears that this technology will continue to advance rapidly and that its applications will become increasingly pervasive over the foreseeable future.
The establishment and proliferation of computer networks have been accompanied by—indeed made possible by—an ever-increasing blurring of the distinction between computer and communication technologies. For instance, computing resources are heavily used in the operation of communication networks, and the capabilities and services to which these networks provide access include electronic mail and bulletin boards, computer-mediated teleconferencing, and information utilities of many types.
Networks are almost certain to continue to increase in number, in complexity, and in the rate at which they can move information from place to place. If their bandwidth, or carrying capacity, continues to increase at anything like its recent rate, the transmission of enormous amounts of data (including digital voice and high-quality video) from almost anywhere to almost anywhere will be possible at relatively low cost. Concurrently with
advances in the technology, the uses to which it is being put are multiplying as well.
The evolving macrosystem of interlinked networks can be thought of as one enormous global nexus that has the potential to increase by many orders of magnitude the extent to which individuals and information resources all over the world are interconnected and therefore accessible to each other. Such capabilities should give people unprecedented access to both information (in libraries, museums, news databases, and other repositories) and people. They have the potential to greatly facilitate not only the distribution and exchange of information in the conventional sense, but also long-distance collaboration involving a real-time sharing of workspaces, tools, and resources (National Research Council, 1993; Wulf, 1993).
A great deal of human factors research has focused on certain aspects of computer and communication technology and especially how the interfaces through which people use this technology for various purposes should be designed. Indeed the general area of human-computer interaction has perhaps been the fastest-growing area of research done by human factors researchers and others in closely allied fields over the last few decades. This work has been reported in several journals that focus on the subject, most of which have come into existence during the last 20 years.
Much less work has focused more directly on the human factors of telecommunications and the resources that are accessible through telecommunications systems. Teleconferencing (Carlisle, 1975; Kerr and Hiltz, 1982) was perhaps the first service provided by telecommunications networks that received some human factors research attention. More recently, cooperative work supported by telecommunication systems has been the focus of some research (special 1992 issues of Human-Computer Interaction and Interacting with Computers), as have electronic mail and electronic bulletin boards (Sproull and Kiesler, 1991).
Computer-based communication should become a high priority for human factors research for many reasons, not the least of which is the expectation that eventually nearly everyone is likely to be a user of this technology. A growing percentage of the workforce will find it essential in the performance of their jobs; many people will use it for personal reasons—information acquisition and exchange, personal business transactions, entertainment, interpersonal communication. The user community will be as diverse as the general population, and the range of uses will be great. As this technology continues to evolve, challenges to human factors research that we cannot now anticipate are bound to emerge; here we mention only a few that are already apparent.
Research Needs and Opportunities
Human-computer interaction, the focus of much research in the past, deserves continued attention from human factors researchers. Issues of interface design, information finding and utilization, personal information management, and users' comprehension of the systems with which they interact continue to be important research challenges. Most terminals today depend primarily on two-dimensional visual displays and on typewriter-like keyboards, typically complemented with a pointing and drawing instrument such as a mouse or trackball. Speech will become an increasingly feasible option for both input and output (Makhoul et al., 1990; Weischedel et al., 1990), as will three-dimensional, "virtual-reality" representations of objects one can "walk around" and environments with which users can interact (Durlach and Mavor, 1995). Refining these and other innovative input-output techniques requires that a variety of human factors issues be addressed.
Realization of the potential benefits of computer networks requires the development of a variety of tools that facilitate interaction with complex databases by both specialists and the public. Tools are also needed to help people apply information technology effectively to manage their personal data stores and to cope with the information overload that a greatly increased connectivity to information resources and to people can create.
With computer-based systems playing increasingly important roles in people's lives, there is a need to learn more about people's attitudes and beliefs about these systems. We also need to understand how to increase the likelihood that people's conceptualizations of what computer-based systems can do and how they do it are reasonably accurate—or at least not inaccurate in counterproductive ways.
Many important challenges to human factors research arise because computer-based communication technology has the potential to greatly increase the ways in which people can communicate with each other. More and more people are using electronic mail, electronic bulletin boards, electronic forums and discussion groups, and computer-based teleconferencing facilities, but these provide only a hint of what is likely to be possible in the not-distant future. Much human factors research needs to be done on the design and utilization of such facilities to ensure that they develop in directions consistent with the needs and preferences of their prospective users and in ways that really enhance and enrich interpersonal communication. Questions involve how interpersonal communication through computer-based systems compares, favorably or unfavorably, with communication through more traditional media and how the emerging media might be shaped so as to increase the opportunities for interpersonal communication as well as to enhance the quality of the communication that occurs.
Communication technology and telenetworking should have important implications for the problem of increasing the employment of people with disabilities. This technology should be able to greatly increase the access of people with mobility problems to many resources that once were available only to people who could travel to them. The full realization of this potential will require that questions of a human factors nature be addressed regarding how best to design the interfaces and operating procedures that will ensure the usability of the technology by people with various types of disabilities.
Even though the use of network technology to hold conferences among "attendees" located at different places has been of interest since the mid-1960s, teleconferencing has not yet become a widely used alternative to face-to-face meetings, even when the meetings involve the expense and inconvenience of considerable travel. Some of the technical limitations of teleconferencing systems are being eased as networks acquire sufficient bandwidth to support the real-time transmission of sufficiently high-fidelity video representations of participants' images to create a realistic impression of an actual gathering. This should help make teleconferencing more attractive, but there is still much to be learned about the human factors aspects of this technology before it becomes the communication asset its originators intended it to be.
The term telecommuting captures the idea of using telenetworks to provide people with the resources to enable them to work at home or in locations other than their traditional workplaces. Substituting the transmission of information via computer networks for the transportation of people to and from places of work is attractive for several reasons, not the least of which is the possibility of conserving energy and easing some of the load on urban areas (traffic congestion, parking problems, air pollution) to which commuting traffic contributes. A significant fraction of the workforce is already doing telecommuting, and increasing this fraction manyfold is believed to be technically feasible. Impediments to greater use of telecommuting include the need for additional user-oriented tools to ensure that workers can perform the desired tasks; other impediments involve issues of worker satisfaction and related psychological variables. Human factors studies are needed to address both types of issues.
With the help of computer networks, colleagues can cooperate at a distance in ways that were impossible until fairly recently. Teams of experts, all in different places but drawing on the same resources and sharing a common "virtual" work space, can collaborate on problems that require their collective knowledge and skills. The success with which this scenario can be played out in specific instances will depend, to no small extent, on how well the many human factors issues relating to the design of the underlying systems are resolved.
As the bandwidth of computer networks continues to increase, it will be possible to transmit an increasingly detailed and veridical representation of a physical situation to a remotely located individual. It seems unlikely, however, that it will be possible, at least anytime soon, to represent most nontrivial situations in sufficient detail that one could not tell the virtual reality from the real thing. Fortunately, such verisimilitude is not necessary for most applications, but the question of how real (in appearance) is real enough is open and probably must be answered on a case-by-case basis. Representations of reality that would be more than adequate for some applications might be inadequate for others. There is a need for work on how to determine the degree of fidelity required in specific instances.
INFORMATION ACCESS AND USABILITY
Among the more striking characteristics of modern society is the dependence of its governmental, economic, and social institutions on constant accessibility to accurate, up-to-date information of many types. The primacy of information and knowledge as resources in today's world is reflected in the growing tendency to refer to our society as the information society (e.g., Salvaggio, 1989) or the postindustrial society (Bell, 1973).
An associated development is the greatly increased quantity of information available to people in all walks of life. The material published for professionals in specific fields has long exceeded the capacity of individuals to stay current except in very narrow subspecialties. The amount of information that is available to the public is also enormous and growing. Being available, however, does not necessarily mean being readily accessible, and users, or potential users, of information that is available often experience frustration in locating, accessing, and interpreting the information they want.
Technological developments—including ever more powerful information-manipulation and display facilities, "intelligent" interfaces, and information-finding and utilization aids—may help to address this problem. An essential aspect of any effort to make information more accessible to people who need or want to use it, however, must be attention to a variety of human factors issues relating to the ways in which people might interface with information repositories and tools that are intended to facilitate finding, using, and conveying information. Moreover, this attention will need to extend beyond the technical user of information systems because the proportion of the labor force that engages in information handling has been increasing rapidly and is expected to continue to do so (Koenig, 1990); many people tap into information resources for purposes other than the
requirements of their jobs. Given the unlimited diversity of the community of users of information systems, designers cannot rely on user perseverance or high technical skills to overcome poor interface design.
Considerable research has been done on the design of interfaces for databases representing restricted domains. Menu organization, for example, has been the focus of numerous studies (Card, 1982; Giroux and Belleau, 1986; Kiger, 1984; Miller, 1981; Norman, 1991; Shneiderman, 1987; Snowberry et al., 1983). Most of these studies have used relatively small, homogeneous, and often abstract databases (Fisher et al., 1990) unlike those typically found in operational settings, and search questions have generally been goal-directed, which means that the results may not help us understand browsing behavior.
Work has also been done on the design of query languages for providing database access (Belkin and Croft, 1987; Shneiderman, 1987), and studies have addressed the effectiveness of various possible search techniques (Blair, 1984; Muckler, 1987; Vigil, 1985). Investigators have also studied the strategies that people use spontaneously to search for information in a large database (Brooks et al., 1986; Chen and Dhar, 1991; Harter and Peters, 1985). Attempts have been made to enhance database search through artificial intelligence techniques (Croft and Thompson, 1987; Hawkins, 1988).
The versatility of the computer for generating graphical representations has sparked some interest in how databases might be represented so as to make the information in them easier to find. Among the approaches being explored are some that represent data sets as three-dimensional structures that users can inspect from different angles and at adjustable viewing distances, thus permitting views of the data from relatively global as well as narrowly focused perspectives (Card et al., 1990; Clarkson, 1991; Newby, 1992). Little is known, however, about the relative effectiveness of different forms of data representation as aids to understanding complex data sets (Jensen and Anderson, 1987; Liu and Wickens, 1992; Merwin and Wickens, 1991).
Research Needs and Opportunities
A continuing human factors challenge relating to complex information systems is interface design. This applies even to systems that represent highly restricted areas of knowledge, for example, an on-line help system for a word-processing program or an airline flight reservation system. An effective interface should allow users to readily locate and access items of information as they are needed and to make a rapid transition from one set
of related items to another. These capabilities relate to the organizational structure of the database (e.g., hierarchical, matrix, network; Durding et al., 1977) and the navigational tools for moving from entry to entry (Seidler and Wickens, 1992). The organizational structure may or may not be independent of the navigational tools. For example, a database may be organized in a strict hierarchical fashion, but navigational tools may allow the user to access any node in the database from any other node with a single command, obviating the need to travel up and down the paths in the hierarchy. Whatever the relationship between the database structure and the navigation aids, the important requirement from the user's point of view is that it facilitate and not impede access.
There is some evidence that the effectiveness of a menu design for an information system may partly depend on how closely the structure represented by the menu corresponds to the user's mental model of the structure of the database (Roske-Hofstrand and Paap, 1986). In some cases, however, there are many "natural" ways to organize a database, and there is little or no guidance as to whether one way is preferable to another, whether one corresponds more closely than another to a mental model, how homogeneous these mental models are across different users, or whether the organizational structure that is appropriate for one task (e.g., normal operation) differs from what is needed for another (e.g., fault diagnosis and troubleshooting). Many human factors issues must be addressed if interface designs are to be optimized for particular combinations of information systems and the users they are intended to serve. The use of electronic database "maps" has proved to be a promising method of making the organization of a database visible to the user (Vincente and Williges, 1988), but major design challenges confront the human factors community if such maps are to represent the vast number of nodes and entries in very large databases in a truly useful way (Mackinlay et al., 1991; Shneiderman, 1987).
Some databases are fluid, in that the information in them is continually changing and they may not have a permanent structure. The database of human factors research is a case in point; not only is new information being added to it all the time, but people do not agree on where its boundaries should lie. Menu-based interfaces have serious limitations when used with such systems. Query languages that rely on keyword searches have some advantages, but also their own limitations.
A relatively neglected, but important, question pertaining to evaluating the effectiveness of any information system or information search procedure is how to determine the value of information to its prospective users. This issue will have to be considered in the design of automated search and retrieval techniques. Not all the information that is relevant to a given topic of interest is of equal value to a user, but little is known about how to
convey to a search procedure the distinction between what would be considered important and what would not. More generally, evaluating the effectiveness of information systems is a continuing challenge. Simply counting ''hits" or computing the ratio of hits to false alarms does not suffice, because it begs the question of what should be considered a hit, and it does not distinguish among hits and misses of different degrees of importance. Taking into account the use to which information is put would probably give a more accurate picture of system effectiveness, but often this may not be practical. We also do not know how to assess effectiveness for systems that are intended to facilitate browsing as opposed to structured search, and this will be a difficult problem.
A special challenge for human factors research relates to the unprecedented capability of computer-based systems to represent information in graphical or pictorial form. The rapid development of capabilities for dynamic three-dimensional displays and of tools that permit users to manipulate and interact with such displays has outdistanced our ability to exploit such capabilities to full advantage. There is a need both to analyze the tasks for which sophisticated visualization tools could be especially helpful and to find better ways to evaluate how well visualization-aiding systems help users perform their tasks. The considerable current interest in better understanding the use of visualization in science and the application of computer-based displays to the development of more powerful techniques for visual data representation emphasize the importance of human-factors research in this area.
For systems that are intended to have some built-in intelligence, designers have often stated that one of their goals is to give the system the ability to adapt flexibly to the needs of individual users, and even perhaps to adapt to changes in those needs over time. Too much adaptability, however, could prove to be counterproductive. The research done to date provides little guidance as to the conditions under which intelligent adaptability may stop being beneficial and may actually be harmful. In general, there is a need to explore the trade-off between flexibility (adaptability to users' needs or preferences) and consistency in the characteristics and operational features of information systems.
People already have extensive experience navigating in space; this creates a strong argument for attempting to exploit spatial metaphors in the design of tools for helping people interact effectively with large complex databases and information networks. But effective use of spatial metaphors will require addressing a number of unresolved issues: determining how best to represent data that have a dimensionality greater than two or three, developing effective methods for helping users maintain their orientation (keep from getting lost) when navigating a database, providing easy means
of conveying to the system desired control actions (zooming, panning, relocating), and numerous issues involving other aspects of user-information system interaction.
EMERGING TECHNOLOGIES IN WORK DESIGN
Remaining competitive in the global marketplace has been an increasingly difficult challenge to many U.S. industries, especially in the manufacturing sector of the economy. The U.S. lead in several industrial areas has been eroded in recent years, and the prospects for competing effectively in these and other areas will depend on the country's ability to use material and human resources more effectively.
It appears that a critical aspect of industrial competitiveness will be the ability to adapt quickly to rapid technological developments and constantly changing market conditions. This means, among other things, being sufficiently flexible to shift production quickly from one item to another and being able to produce items economically in relatively small quantities (Piore and Sabel, 1984). Production flexibility requires a very different job design from that of conventional twentieth-century assembly lines. Other innovative ways to increase competitiveness, such as just-in-time manufacturing, which is intended to save costs by minimizing the need for large inventories, also have implications for work design (National Research Council, 1986).
Emerging technologies—many forms of automation, robotics, and information technologies—will figure prominently in shaping the workplaces of the future. In some cases, workers will be displaced as technological advances make their tasks obsolete or performable by machine. Other jobs will be transformed by the availability of new tools that workers will have to learn how to use. And jobs that do not now exist will be created. Change will be the norm.
Human factors research has something useful to contribute to maintaining the industrial competitiveness of the United States as these changes take place, but the goal of the research must be more than the design of better displays and controls (which is not to discount the importance of this objective). Improving manufacturing, for example, will require attention to human factors issues conceived sufficiently broadly to include the social psychology of the functioning of work groups as well as classical ergonomics. The realization of the need for a relatively broad perspective has led to the current interest in macro-ergonomics.
Job design has been a focus of human factors research from the earliest days of the discipline. Originally attention was focused primarily on the individual worker and the demands of specific tasks. "Taylorism," as a theory of job design, was concerned with the optimization of physical effort in order to increase speed of production. Jobs were subdivided into repetitive tasks to minimize the amount of learning required by the worker and to maximize the speed with which a task could be performed (Knights et al., 1985). "Fordism" introduced the assembly line, which organized Taylorized tasks into patterns that would yield automobiles and other complex consumer products.
The assembly line approach was very effective in mass-producing consumer goods that could be sold at affordable prices, but it left much to be desired from the worker's perspective and often led to stress, low worker motivation, absenteeism, and labor conflict (Kornhauser, 1965; Walker and Guest, 1952). Researchers interested in job design began to see worker satisfaction and quality of life in the work setting as important complements to worker efficiency. In time, the importance of taking into account the interactions among individuals in work settings was recognized, as was the need to apply a systems perspective to the study of complexes involving both people and machines.
Despite the attention that job design has received from human factors researchers, most jobs still are not designed in any rigorous sense, but they evolve—often being forced to change in order to accommodate the introduction of new technology, but not necessarily in optimal or even satisfactory ways. What was already a complex problem has been exacerbated in some instances by the extraordinary rapidity with which information-based technologies have permeated the workplace.
The need for more attention from human factors researchers to the introduction and use of information-based technology in the workplace is seen in such disparate problems as the biochemical and physiological difficulties that are sometimes associated with the use of information-technology devices (e.g., eyestrain and carpal tunnel syndrome) and the claims of some investigators that these technologies do not increase productivity as they were expected to do. There have also been calls for a better understanding of the ways in which the new technologies are changing the skills required of workers (National Commission on Employment Policy, 1986) and of why these technologies appear to be underused or inefficiently used in some contexts, especially manufacturing (National Academy of Engineering/National Research Council, 1991).
Research Needs and Opportunities
If U.S. industry is to compete effectively in world markets, it must give a high priority to product quality. The importance of quality increases with the amount of competition that an industry faces, and many of the industries that are important to the economic health of the nation have much competition on the world scene. Quality flaws in products typically derive either from worker error or from suboptimal work processes and procedures. There are many opportunities for human factors research on both decreasing the probability of human error in the workplace and enhancing product quality by improving the designs of work processes and procedures. The study of human error in certain types of information work—programming or software design, for example—poses a special challenge because errors may be very difficult to identify and it may be almost impossible to trace back to them from their eventual effects.
Workplace health and safety will continue to be major foci of human factors research. Lighting, air quality, potentially dangerous machinery, noise, fatigue, biomechanical stress, and other traditionally important human factors issues relating to work and the workplace will still require attention, but new problems will also surface as jobs and workplace technology continue to change. Certain types of musculoskeletal disorders, sometimes referred to as repetitive strain injury, appear to have been becoming increasingly common among people who make constant use of computer terminals or similar keyboard devices. So also have certain ailments that are probably attributable to psychosocial properties of some jobs that seem to create undue psychological stress. Effectively addressing workplace health and safety will require attention both to the traditional ergonomic concerns of proper equipment design and to psychosocial factors that can directly and indirectly affect workers' health and safety.
The integration of human and machine labor in semiautomated industrial operations continues to be a challenge, and so far it has not been met effectively. The idea that machine intelligence might substitute for human intelligence in many manufacturing processes appears to have been giving way to the idea that what is needed, for the foreseeable future at least, is a better understanding of how to combine human and machine capabilities in truly symbiotic ways (Brodner, 1986; De Greene, 1991; Kellso, 1989; Kuo and Hsu, 1990). If such systems are to be realized, many human factors issues will have to be addressed; they range from the traditional problems of interface design to planning, scheduling, and coordinating activities to top-level policy setting (Sanderson, 1989).
Many industrial jobs of the future will involve people as supervisors of highly automated operations. When things are operating smoothly as intended, "supervisory controllers" may have relatively little to do; however,
when systems fail or malfunction, they may have to take quick and decisive action to avoid serious consequences. Such situations pose problems of boredom and skill maintenance. How are supervisory controllers to be kept attentive and interested in their jobs when everything is proceeding as intended without their involvement, and how are their skills to be kept honed so they can do what is required in moments when they are needed to avert a crisis? More generally, there is a need for systematic and detailed descriptive research about the skills, especially the higher-level cognitive skills, that workers need in the various job situations that high technology is creating and that will be even more common in the future.
One effect of the new information technologies on many jobs has been to put some distance, real or symbolic, between workers and whatever they are producing. This is illustrated most strikingly in the use of tele-operators and other remotely controlled devices in some production processes, and it is apparent in other contexts as well. Many machinists no longer do any machining by hand with traditional machine tools; instead they feed their specifications to a programmable machine tool. Even people who generate paper products (reports, drawings, blueprints) typically do so via a computer-based system that yields the paper only after the work has been done. There has been some speculation about the effects of this change on workers and on the skill requirements of their jobs (Zuboff, 1988), but there has been little empirical investigation into the matter.
The emphasis in the foregoing has been on the need to find ways to ensure the competitiveness of U.S. industry in a rapidly changing world; this is indeed a serious challenge for the nation. We believe human factors researchers can significantly contribute to meeting this challenge. It is important, however, that in doing so, this community maintain the complementary objective of ensuring that the work climate, culture, and environment are humane and fulfilling to the human beings who work within it. Job satisfaction will require attention as long as there are jobs, but the answer to the question of how to promote it is likely to change as the characteristics of jobs change, and not always in predictable ways.
Most people interact with transportation systems as operators and as passengers. Millions of us operate motor vehicles, and millions more are passengers on airplanes and railroad trains. In both cases, the goals are to achieve convenient, fast, and safe travel. Automobiles offer individual mobility; air and rail transport offer efficient travel over long distances. As
speed and convenience have increased through advances in technology, there has also been increased concern about public safety.
In the United States, motor vehicle accidents are the primary cause of death for people under age 38. Current statistics show that there are also 1.7 million disabling injuries from motor vehicles every year. Because traffic accidents take a disproportionate toll from the younger population, the average number of life years lost is two to three times higher than for heart disease and cancer. A large proportion of these accidents can be attributed to human error. Sources of such error include lack of experience, misperception of events, poor vision, use of drugs and alcohol, fatigue, and high levels of risk tolerance.
Air travel, in contrast, is the safest form of mass transportation in terms of accident occurrence and loss of life. However, when accidents do occur, the toll is often catastrophic. In over two-thirds of these accidents, human error is cited as the primary cause. The sources of human error in air accidents are different from those in automobile accidents. In modern aircraft, pilot errors may stem from poor mental models of the functions being performed by highly complex automated systems, from information or work overload, or from failures in coordination and communication among members of cockpit crews or between cockpit crews and air traffic controllers.
We can anticipate that, as technology advances and greater demands are placed on both motor vehicles and commercial aircraft for convenience and speed, understanding the capabilities and limitations of the human operators will become even more critical. Human factors can make important contributions by developing coherent models of the human as an operator and of the team and organizational environment in which the human must function. In addition, human factors can provide insights into what functions should be automated and how this should be done to ensure efficient and safe operation.
The available knowledge base for understanding proficient driving and traffic safety skills in motor vehicle operation can be characterized as extensive but fragmented. As already mentioned, several sources of human error have been researched; what is missing, however, is a conceptual model that specifies a coherent research agenda for describing driving performance under a variety of conditions with a range of technological alternatives. The main lines of researchable questions have focused on deficits in vision, the influence of alcohol and drugs, attention and automaticity, aging, and the misestimation of risk as potential contributors to reduced driving performance.
Tests for visual acuity currently assess static acuity on one eye at a
time. However, recent research (Burg, 1967, 1968, 1971) suggests that static acuity alone is a poor predictor of accident probability and that tests for dynamic acuity are preferable. Moreover, the degree to which both eyes work together to provide the widest possible field of view has also been found to be a crucial factor. One problem is that the current tests used to measure visual acuity for drivers are based on evaluating only the threshold for resolution for high-contrast optotypes whereas, in many situations, the driver must distinguish low-contrast objects under low illumination. The tests of vision that appear to have higher predictive power for safe driving involve peripheral vision, contrast sensitivity, and motion perception (Johnson and Keltner, 1983).
Recent findings on alcohol and drug intoxication suggest that impairment that would affect driving can be detected in individuals with much lower blood-alcohol levels than those now used to determine legal liability. Also, strong interaction effects between individuals and conditions have been recorded; this should lead to a radical rethinking of the legal limits on blood alcohol.
Another line of research on drivers deals with vigilance and the useful field of view. There is evidence that repetitive tasks can result in general drowsiness or a narrowing of the effective field of view for attention, and some studies show that restrictions in this field can pose serious problems for older drivers (Ball and Owsley, 1992). Furthermore, as alertness decreases, reaction time can become significantly longer.
Work is also going forward on driver attitudes toward risk or risk tolerance. Recent findings suggest that some drivers have a risk tolerance threshold that is constant. If such drivers have vehicles with automatic seat belts or air bags that lower risk, they tend to increase risk by driving faster than they otherwise would. The net effect is that the benefits of technological advances are canceled out.
Research on motor vehicle drivers has focused on the characteristics of the individual and how those characteristics interact with performance. Human factors research on aircraft operation, however, has concentrated both on the individual and on teamwork in multiperson operator crews. In recent years programmatic movement in this area has emerged under the rubric of crew resource management (CRM). CRM is based on the recognition that many vehicles—particularly large commercial aircraft—are actually piloted by a team rather than a single individual. The performance of the vehicle as a system thus depends in part on how well the team members accommodate to one another in addition to how well they operate as competent specialists. The approach has, as a first focus, the training of the crew. Current practices put great emphasis on the creation of training situations that exercise the teamwork functions. In a dynamic flight simulator, for example, the whole crew is exercised simultaneously. What takes place is not just
skill learning in the sense of interpersonal communication skills but also a solid sense of mutual confidence that can allow team members to compensate for one another's weak points.
Research Needs and Opportunities
There is no shortage of needs for rigorous research in advanced transportation systems. A truly basic need is for a better systemic representation of the human in the system. Ideally, the community of human factors researchers would create a model to identify the human attributes that contribute most to performance. The framework should include the distribution of individual differences for each such attribute and something similar to a multiple regression equation, describing in quantitative terms how each attribute contributes to the quality of performance in challenging work situations. Given such a framework, the analysis of a set of particular driver behaviors would be more straightforward and would be more likely to point clearly to reforms at the practical level, such as the design of tests used to screen drivers for licensure.
Another key opportunity exists in the area of automation and function allocation. The present atmosphere appears to favor the development and evaluation of computer-based aids rather than complete reassignment of operator responsibilities to a computer. For example, it is now possible to use information from satellites to determine one's exact position in three-dimensional space. Such positional information could be used, in turn, to drive map-type displays in a cockpit of an aircraft, bridge of a boat, cabin of a locomotive, or instrument panel in a car. What effect would such displays have on the performance of the operator and on the system as a whole? Would the presumptive improvement in performance make up for the additional cost? Would people drive automobiles more recklessly if they thought that the new display reduced risk?
The time has come when the operator's role could be totally automated in many transportation systems. Would passengers (or, indeed, nonparticipating bystanders) permit the total automation of any common carrier vehicles? What are the predominant attitudes of various cultures toward automation? How do these attitudes influence the willingness of individuals to disengage automated systems and revert to human control when conditions warrant?
A third important issue is the differential reactions of various cultures to team training such as CRM. For example, in collectivist cultures a high value is placed on the group, whereas in individualist cultures, such as the United States, more value is placed on individual performance. These values and their consequences should be explored in detail so that training can be appropriately designed. One important goal of human factors is to define
curricula that are relevant to the task, understandable to those being trained, and acceptable in the context of societal values.
Finally, there is a need to develop an improved methodology for understanding human error at the group level and for evaluating group and system performance. Accident investigations have found that a number of factors at the regulatory and organizational level, as well as at the individual level, contribute to the inadequacy of safeguards against fatal decisions. Effort should be directed toward developing a taxonomy of human factors problems at various levels that could be applied to the analysis of accidents and incidents. This would be an invaluable tool for researchers and for those responsible for ensuring safety (Jones, 1993).
The field of transportation provides rich opportunities to expand the scope of human factors in areas in which outcomes can have major consequences. Optimizing the interface between individuals and groups that work with complex technology and systems requires a multidisciplinary approach that embraces the full range of concerns of human factors specialists. Concern with a particular problem, whether vehicular or aviation safety, should not blind researchers to concepts that transcend problem areas and that reflect more broadly on human capabilities and limitations.
COGNITIVE PERFORMANCE UNDER STRESS
Stress is a fact of life in modern society and it manifests itself in a variety of ways, sometimes tragically. Incidents of violence in the workplace have sometimes been attributed, at least in part, to stress. Stress has also been implicated both in high-profile accidents, such as the Three Mile Island incident, the downing of an Iranian airliner by the Vincennes, and the crash of Air Ontario Flight 363 (Helmreich, 1992; Wickens, 1992). Stress is also involved in the countless less spectacular mishaps that occur everyday in the workplace, in the home, and elsewhere (Druckman and Swets, 1988; Manuso, 1983).
Since Hans Selye (1956) first directed the attention of the scientific community to the problem of stress and articulated his theory of the "general adaptation syndrome," the concept of stress has evolved considerably. In this report the term is taken to mean reactions to perceived significant threats to one's welfare, reactions that often entail heightened emotion (see Keinan, 1987; Yates, 1990). Stress, according to this conception, is dependent on both the person and the situation. What is threatening to one person at a particular time may not be so to another person or to the same person at a different time. A type of threat that is especially germane to understanding how stress affects cognitive performance is the threat of task demands,
especially time-limitation demands, that appear to be beyond one's capabilities to meet.
Some occupations are generally viewed as especially stressful—air traffic controller, for example. Generally, any occupation that puts one in the position of being able to cause great harm to other people or to oneself as a consequence of making an error is likely to be considered stressful. Other jobs can be stressful for much more subtle reasons, and the stress they cause may be at a lower level but relatively continuous. Jobs at which people must work continuously at a near-maximum pace to maintain production quotas, or in which inadequate personal space or privacy create social tension, or in which one has little job security are examples of jobs that can cause chronic stress.
The psychological literature on stress is huge, and a considerable amount of work has been done on the effects of stress on cognition (Hamilton and Warburton, 1979; Hockey, 1983, 1986; Hockey and Hamilton, 1983). We believe that the need for research on this topic will increase over the foreseeable future for several reasons: threats to job security that will come from increasing competitive pressures in many industries, the need to accommodate to increasingly rapid change in the workplace because of technological advances and other factors, the greater cognitive demands of many jobs and the possibility that errors will have far-reaching implications, and the stress evoked by new methods of electronic monitoring and surveillance.
There seems to be general agreement among researchers that stress adversely affects short-term memory, although it is not clear whether stress reduces capacity or imposes a greater load on short-term memory (Cohen, 1978). Another consensus conclusion is that stress reduces the scope of perceptual attention (Baddeley, 1972), a phenomenon sometimes referred to as perceptual narrowing or "tunneling," but there is some doubt about which stimuli do and do not receive attention when this narrowing occurs (Yates, 1990). Resolving these and similar theoretical questions that have been prompted by previous research would facilitate the development of adequate approaches to training people to function well under stress.
One difficulty that has plagued laboratory studies of stress and its effects on performance is the impossibility, for ethical reasons, of exposing subjects to situations that are threatening in a nontrivial way. Informed consent requirements also preclude the element of surprise in experimentation, inasmuch as subjects must be told the details of the experimental procedure in advance. For these reasons, much of the laboratory work on
the effects of stress on performance has limited applicability to the real-world situations that seriously, and often unexpectedly, threaten people's safety or well-being.
Another limitation of most laboratory studies of stress is that even the mild stressors used are applied for relatively short periods of time. It is very difficult to imagine practically feasible laboratory situations for studying the effects of long-term or chronic stress. But such stress is as much a problem as is short-term or acute stress. As suggested by the work of Cohen (1980) and Cohen and Spacapan (1978), the effects of chronic stress hold surprises that seem inaccessible via standard laboratory techniques.
Research Needs and Opportunities
A priority for future research on the effects of stress on cognitive performance must be a search for feasible solutions to the limitations of laboratory methods. If laboratory results are to be generalizable to the real-world situations of interest, ethical but effective ways must be found to study the influences of stressors that more closely approximate those in real-world human-technology systems. We mention here two strategies that might be applied to this end: (a) analyses of naturally occurring incidents and (b) simulations, including competitive games. Both of these strategies are used currently; we recommend that they be exploited more fully and creatively.
Incident analysis could be considerably facilitated by the development of formal protocols for use in specific incidents, such as those involving nuclear power plant operation, aviation, and surgical procedures. The availability of such protocols, appropriately evaluated, would make it practical for incident analyses to be commissioned regularly (e.g., by the Nuclear Regulatory Commission, the Federal Aviation Administration, and various surgery review boards) to better understand their causes. Statistical analyses of incident-recording databases, such as the Aviation Safety Reporting System that is maintained by the National Aeronautics and Space Administration, is one means of investigating the effects of stress on performance; nonintrusive study of people in stressful situations in which they have voluntarily placed themselves is another.
The simulation of stressful situations is a possible approach if simulators can be built that create a convincing illusion of stress without actually putting subjects at risk. How to build such simulators is itself a research challenge because of the limitations of our present knowledge of the determinants of stress, but common experience with frightening films, amusement park rides, and similar emotion-arousing stimuli attest to the possibility of feeling stressed even while not believing one's self to be in actual danger. Virtual reality should offer the possibility of simulating stressful
situations with a high degree of realism without exposing people to physical threats. How best to exploit these possibilities in the study of the effects of stress on performance is a question that will require some research to answer.
In addition to learning more about the effects of stress on cognitive performance, a major challenge for human factors research is the development of effective ways to counteract the more detrimental of those effects. Approaches to handling stress include attempting to eliminate stressors or to lessen their strength, reducing the stress reactions of individuals, selecting people with high tolerance for stress for stressful jobs, training people to function effectively even when stressed, and ''stress-proofing" systems so they will function smoothly even when their operators are stressed. All of these approaches are worth pursuing.
We know that people differ both in the situations they find stressful and in their ability to function under a given level of stress (see Hockey et al., 1986, especially Section IV). And considerable attention has been given to the question of how to distinguish people who are more and less stress-resistant (Allred and Smith, 1989; Kobasa, 1979; Parkes, 1986). We need to know more about such individual differences in response to stress, but perhaps even more importantly, we need to be able to predict how individuals are likely to be affected by specific stressors in specific situations.
Some research has been done on how to train people so that they will be able to perform their tasks well even under highly stressful conditions, but more is needed. It is not clear from the results obtained to date whether it is better to train people primarily under nonstressful conditions or to give them a lot of practice under conditions that approximate the stressfulness under which they may have to function. It is clear that high degrees of stress usually impede learning, but it does not follow that, given the need to be able to function under stress eventually, the training methods that are most successful in the short run will prove the most efficient from a sufficiently long-range perspective. There is a need for some systematic research on this problem to explore the range of possible approaches that will best prepare people to perform under the full spectrum of situations they could encounter on the job. These might include hybrid approaches that would, for example, provide training under nonstressful conditions and "overtraining" under stress.
Complementing the need to determine how best to train people to function effectively under stress is the need to make systems as stress-resistant as possible. The research objective in this case is to determine how to design human-technology systems so their operational goals will be realized even when their operators are stressed and more prone to make errors than normally. Considerably less is known about effects of stress on the performance of groups or person-machine complexes than about the effects of
stress on the performance of individuals (Davis et al., 1992), so this challenge involves the need to acquire knowledge at a relatively fundamental level. In addition, however, there is a current practical need to identify features that could be designed into systems to safeguard against known effects of stress on human performance. To illustrate the point: because stress often results in restricting one's range of attention, a system might be given the capability to force operators, during emergency situations, to explicitly acknowledge their attention to every item on a checklist designed for that contingency.
Eliminating or reducing high levels of stress in work situations should be a continuing objective of research. However, prudence dictates the assumption that, no matter how successful we are in this objective, there will arise from time to time in many modern work contexts stressful situations that either cannot be anticipated in detail or cannot be precluded. It is therefore essential that research seek effective ways to deal with those situations through personnel selection, training, and system design.
AIDING INTELLECTUAL WORK
Almost all work is done with the assistance of tools of one sort or another. This is true of intellectual as well as physical work. As computer technology has become widely available in the workplace, many software tools have been developed to help people perform intellectual tasks, such as writing, designing, decision making, and analyzing. These tools include electronic dictionaries, thesauruses, spelling verifiers, spreadsheets, design aids, conferencing systems for group work, models for weather forecasting, medical diagnostic aids, and systems to help people visualize structures such as complex molecules or airflow patterns on an aircraft wing.
It seems certain that efforts to develop additional and more powerful aids to intellectual work will continue for the foreseeable future. This effort will be driven in part by continuing rapid advances in the enabling technologies and in part by the promise that such aids hold for increasing productivity and creativity in the workplace and for enriching people's intellectual experience in many other contexts.
The human factors issue of overriding importance is to help ensure that the technology really has the effect of enhancing the quality of life on the whole. Human factors efforts are needed (a) to evaluate the effectiveness of such aids, (b) to provide—through task analyses—a better understanding of what further aids would be useful, and (c) to participate in the design, implementation, and iterative improvement of future aids.
Although many electronic aids to intellectual work have been developed, some of which are used daily in work settings, surprisingly few systematic studies have been done to evaluate their effectiveness. The evaluative data that do exist suggest that some aids do not increase productivity (Attewell, 1994).
Studies conducted in the 1970s on the use of dictating equipment illustrate the importance of checking prevailing beliefs about the difficulty or effects of introducing new technologies in the workplace against objective data. In this case, common assumptions about the difficulty of learning to dictate effectively, and about the efficiency of dictation as compared with manual writing for experienced users of dictation equipment, proved to be wrong (Gould, 1980). Experimental studies also showed that people found it easier to compose "voice documents" for listening than to dictate documents to be read. Such findings were instrumental in shifting the focus of some developers from dictation systems to voice-message systems (Gould and Boies, 1983).
Other experiments have assessed the effects on productivity of the increasing trend among office professionals, including managers, to use text processing systems to compose and edit text themselves instead of using the services of a secretary or clerical staff (Card et al., 1984; Gould, 1982). Again, major savings in time or improvements in product quality were not always found.
Field studies of organizations that have introduced innovative technological aids to intellectual tasks in the workplace have also shown that it is possible for these aides to increase productivity while lowering employee morale and job satisfaction (Kraut et al., 1989). Negative reactions sometimes occur, for example, because using the technological aids changes the amount of social contact and face-to-face interaction that people have in their jobs.
Recent research has also focused on the development or evaluation of aids to facilitate collaborative work among the members of a group (Turner and Kraut, 1992). Some of this work has been done in the laboratory, some in real work situations, and some in both (Olson et al., 1992). The main conclusion to be drawn from this early work is that much additional research will be needed before firm design guidelines for the development of aids for intellectual work by groups can be articulated.
A few efforts have been made by human factors groups to initiate the development of one or more aids to intellectual work and to see this through to implementation in a work setting (e.g., Gould et al., 1993; Harris, 1980; Landauer et al., 1993). These are atypical, however; more commonly human factors researchers have become involved in development efforts only
after they are under way, usually in support rather than leadership roles, and often for the sole purpose of validation testing at the end of a developmental cycle.
Research Needs and Opportunities
Evaluative studies are needed, not so much to determine the relative merits of existing aids but to provide the insights required to improve upon them and to ensure that they accomplish what is intended. Evaluative research should focus not only on aids, but also on the processes that are used to develop them. We need to know what developmental approaches are most effective and how they might be improved.
There is also a need to study the detailed demands of intellectual work in specific settings with a view to identifying how the development of new tools could aid that work. It would be useful, for example, to study successful physicians diagnosing, nurses providing nursing care, musicians composing, architects designing, and teachers teaching; the aim would be to identify the intellectual skills involved and develop a theory about the task demands that would guide the search for new tools to aid performance. This effort should be conditioned, however, by the recognition that frequently providing new tools results not so much in making the current job easier to do but in changing the nature of the job by making it possible to do things that could not be done before.
Several studies, using either observational techniques or questionnaires, have been done of how white-collar workers spend their time (Klemmer and Snyder, 1972; Kraut and Streeter, 1995; Mintzberg, 1973; Panko, 1992). Although these studies have been valuable, they have not led to the development of new intellectual work aids, nor were they intended to do so. Insights that could lead to such development are most likely to come from studies designed for that purpose, and this probably means studies that relate specific activities to personal and organizational goals and that attempt to determine how well the activities serve those goals. One general conclusion that all of these studies support is that interpersonal communication is a major component of most white-collar jobs, so it would seem to follow that a major opportunity for performance enhancement should lie in the development of more effective aids to that communication.
Human factors researchers can also contribute to the design, implementation, and evaluation of new aids for intellectual work. They bring to the process an iterative design philosophy and a unique focus on users' needs that has proved to be important in the development of systems whose ultimate characteristics were impossible to specify in detail in advance.
Some evaluation studies must focus on people in actual work settings, as opposed to laboratory simulations of these settings. Studies should also
focus on the effects of technological aids on both individuals and organizations. There is a need to determine the extent to which changes in individual productivity translate into similar changes at organizational levels, and there is a need too to ensure that productivity gains at an organizational level are not realized at the cost of individual job satisfaction. Acquiring reliable information on the long-term effects of the use of specific technological aids to intellectual tasks in the workplace, especially those intended to facilitate group work, is likely to require studies of considerably longer duration than those that are typically conducted (Cool et al., 1992).
When a work activity that has been performed one way is suddenly performed in a very different way because new technology was introduced, this can have important but quite subtle effects. Malone (1983), for example, has shown that the arrangement of papers on people's desks can serve to remind them of what has to be done and in what priority. When the desktop is replaced by an electronic file and a video monitor, the traditional cues are lost, and if the system has not been designed to provide an effective substitute for them, work may suffer. One human factors challenge associated with the design of intellectual work aids is to identify nonobvious but important aspects of current ways of performing tasks for which some provision should be made in an electronic aid.
For some time, there has been compelling evidence that diagnostic judgments based on actuarial data are, in many contexts, typically more accurate than judgments made by human diagnosticians without the aid of the actuarial data (Dawes et al., 1989). A number of decision aiding systems have been developed that exploit this fact, but the diagnosticians have generally resisted using them. Although many reasons have been hypothesized for the reluctance of experts to use these aids, the situation is still not well understood, and the question of how to get diagnosticians and decision makers to avail themselves of aids that would improve their performance is a continuing challenge.
Successful development and use of new tools to help people perform intellectually demanding work will require collaboration among scientists and engineers from several disciplines. Human factors researchers have a critical role to play in this effort. We believe that they can and should take the initiative in identifying tasks whose performance could be enhanced through appropriately designed aids and that human factors researchers have much to contribute to both the design and the evaluation of these aids. The need for interdisciplinary collaboration is itself a challenge, however, to all the disciplines involved and may require the human factors community to broaden its conceptualization of what constitutes legitimate research. In particular, human factors researchers must be willing, at least in some instances, to take responsibility for initiating and successfully completing interdisciplinary projects to produce significant aids, with all the burdens
and long-term commitment this entails. Iterative design with constant ongoing evaluation involving intended users of an aid is an approach that the human factors community has advocated and with which it has had some success.
This summary report identifies important areas of need and opportunity for human factors research during the next few decades. It is the committee's hope that this volume will be of use both to sponsors of human factors research and to people who do human factors research.
The committee has not attempted to lay out a research agenda. It recognizes that the priorities of different research-sponsoring agencies will differ depending on the agencies' specific missions and goals and that individual scientists and scientists-in-training will be drawn to different problem areas depending both on funding opportunities and on personal interests, knowledge, and expertise. We believe, however, that there are many opportunities for human factors research to address important national and global problems, some of which have not been the focus of such research in the past. This report and the papers in Part II indicate what, in the committee's view, some of those opportunities are.
Allred, K.D., and T.W. Smith 1989 The hardy personality: cognitive and physiological responses to evaluative threat. Journal of Personality and Social Psychology 56:257-266.
Attewell, P. 1994 Information technology and the productivity paradox. Pp. 13-53 in D.H. Harris, ed., Organizational Linkages: Understanding the Productivity Paradox. Panel on Organizational Linkages, Committee on Human Factors, National Research Council. Washington, D.C.: National Academy Press.
Baddeley, A.D. 1972 Selective attention and performance in dangerous environments. British Journal of Psychology 63:537-546
Baker, L.A., A.T. Herlihy, P.R. Kaufmann, and J.M. Eilers 1991 Acidic lakes and streams in the United States: the role of acidic deposition. Science 252:1151-1154.
Ball, K., and C. Owsley
1992 The useful field of view: a new technique for evaluating age-related declines in visual function. Journal of the American Optometric Association 63:71-79.
Baron, S., D.S. Kruser, and B.M. Huey, eds. 1990 Quantitative Modeling of Human Performance in Complex, Dynamic Systems. Panel on Human Performance Modeling, Committee on Human Factors, National Research Council. Washington, D.C.: National Academy Press.
Baum, A., and J.E. Singer, eds. 1981 Conservation: Psychological Perspectives, Vol. 3. Advances in Environmental Psychology series. Hillsdale, N.J.: Erlbaum.
Belkin, N.J., and W.B. Croft 1987 Retrieval techniques. Annual Review of Information Science and Technology 22:109-145.
Bell, D. 1973 The Coming of the Post-Industrial Society. New York: Basic Books.
Blair, D.C. 1984 The management of information: basic distinctions. Sloan Management Review 26(1):13-23.
Bogner, S. 1991 Human factors and medicine. P. 682 in Proceedings of the Human Factors Society 35th Annual Meeting. Santa Monica, Calif.: Human Factors Society.
Bowe, F.G. 1984 Personal Computers and Special Needs. Berkeley, Calif.: Sybex Computer Books.
Brodner, P. 1986 Skill-based manufacturing vs "Unmanned Factory": which is superior? International Journal of Industrial Ergonomics 1:145-153.
Brooks, H.M., P.J. Daniels, and N.J. Belkin 1986 Research on information interaction and intelligent information provision mechanisms. Journal of Information Science: Principles and Practice 12(1):37-44.
Bureau of the Census 1987 Statistical Abstract of the United States. Bureau of the Census. Washington, D.C.: U.S. Department of Commerce. 1989 Statistical Abstract of the United States. Washington, D.C.: U.S. Department of Commerce.
Bureau of Labor Statistics 1988a Comparative Real Gross Domestic Product, Real GDP per Capita, and Real GDP per Employed Person, Thirteen Countries, 1950-1987. Washington, D.C.: U.S. Department of Labor. 1988b International Comparisons of Manufacturing Productivity and Labor Cost Trends. Washington, D.C.: U.S. Department of Labor.
Burg, A. 1967 The Relationship Between Vision Test Scores and Driving Record: General Findings. Department of Engineering Report No. 67-24. Los Angeles, Calif.: University of California. 1968 The Relationship Between Vision Test Scores and Driving Record: Additional Findings . Department of Engineering Report No. 68-27. Los Angeles, Calif.: University of California. 1971 Vision and driving: a report on research. Human Factors 13(1):79-87.
Card, S.K. 1982 User perceptual mechanisms in search of computer command menus. Pp. 190-196 in CHI '82 Conference on Human Factors in Computer Science . New York: Association for Computing Machinery.
Card, S.K., J.M. Robert, and L.N. Keenan 1984 On-line composition of text. Pp. 231-236 in Proceedings of Interact'84, First IFIP Conference on Human-Computer Interaction, Vol. 1. London, England: Elsevier North Holland.
Card, S.K., J.D. Mackinlay, and G.G. Robertson 1990 The design space of input devices. Pp. 117-124 in CHI Proceedings . New York: Association for Computing Machinery.
Carlisle, J.H. 1975 A Tutorial for Use of the TENEX Electronic Notebook-Conference (TEN-C) System on the ARPANET. Report No. ISI/RR-75-38. Arlington, Va.: Defense Advanced Research Projects Agency.
Chen, H., and V. Dhar 1991 Cognitive process as a basic for intelligent retrieval systems design. Information Processing and Management 27(5):405-432.
Clarkson, M.A. 1991 An easier interface. BYTE February:277-282.
Coach, J.V., T. Garber, and L. Karpus 1979 Response maintenance and paper recycling. Journal of Environmental Systems 8:127-137.
Cohen, S. 1978 Environmental load and the allocation of attention. Pp. 1-29 in A. Baum, J.E. Singer, and S. Vallins, eds., Advances in Environmental Psychology. Hillsdale, N.J.: Erlbaum. 1980 Aftereffects of stress on human performance and social behavior: a review of research and theory. Psychological Bulletin 88:82-108.
Cohen, S., and S. Spacapan 1978 The aftereffects of stress: an attentional interpretation. Environmental Psychology and Nonverbal Behavior 3:43-57.
Commission on the Skills of the American Workforce 1990 America's Choice: High Skills or Low Wages. Rochester, N.Y.: National Center on Education and the Economy.
Commission on Workforce Quality and Labor Market Efficiency 1989 Investing in People: A Strategy to Address America's Workforce Quality. Washington, D.C.: U.S. Department of Labor.
Committee on Human Factors 1983 Research Needs for Human Factors. National Research Council. Washington, D.C.: National Academy Press.
Cone, J.D., and S.C. Hayes 1980 Environmental Problems/Behavioral Solutions. Monterey, Calif.: Brooks/Cole Publishing.
Cool, C., R.S. Fish, R.E. Kraut, and C.M. Lowery 1992 Iterative design of video communication systems. Pp. 25-32 in J. Turner and R. Kraut, eds., Proceedings of the Conference on Computer-Supported Cooperative Work. New York: ACM Press.
Croft, W.B., and R.H. Thompson 1987 I3R: a new approach to the design of document retrieval systems. Journal of the American Society for Information Science 36(6):389-404.
Crossen, P.R., and N.J. Rosenberg 1989 Strategies for agriculture. Scientific American 261(3):128-135.
Davis, J.H., T. Kameda, and M.F. Stasson 1992 Group risk taking: selected topics. Pp. 163-199 in J.F. Yates, ed., Risk-Taking Behavior. Chichester, England: Wiley.
Dawes, R.M., D. Faust, and P.E. Meehl 1989 Clinical versus actuarial judgment. Science 243:1668-1673.
De Greene, K.B. 1991 Emergent complexity and person-machine systems. International Journal of ManMachine Studies 35(2):219-234.
Druckman, D., and R. Bjork, eds. 1991 In the Mind's Eye: Enhancing Human Performance. Committee on Techniques for the Enhancement of Human Performance, National Research Council. Washington, D.C.: National Academy Press.
Druckman, D., and J.A. Swets, eds. 1988 Enhancing Human Performance: Issues, Theories, and Techniques . Committee on Techniques for the Enhancement of Human Performance, National Research Council. Washington, D.C.: National Academy Press.
Durding, B.M., C.A. Becker, and J.D. Gould 1977 Data organization. Human Factors 19:1-14.
Durlach, N.I., and A.S. Mavor, eds. 1995 Virtual Reality: Scientific and Technological Challenges. Committee on Virtual Reality Research and Development, National Research Council. Washington, D.C.: National Academy Press.
Elkind, J.I. 1990 The incidence of disabilities in the United States. Human Factors 32:397-405.
Fisher, D.L., E.J. Yungkurth, and S.M. Moss 1990 Optimal menu hierarchy design: syntax and semantics. Human Factors 32:665-683.
Gagnon, M., C. Sicard, and J.P. Sirois 1986 Evaluation of forces on the lumbo-sacral joint and assessment of work and energy transfers in nursing aides lifting patients. Ergonomics 29:407-421.
Geller, E.S. 1986 Prevention of environmental problems. Pp. 361-383 in Handbook of Prevention. New York: Plenum.
Geller, E.S., R.R. Winett, and P.B. Everett 1982 Preserving the Environment: New Strategies for Behavior Change . Elmsford, N.Y.: Pergamon Press.
Giroux, L., and R. Belleau 1986 What's on the menu? The influence of menu content on the selection process. Behavior and Information Technology 5:169-172.
Goldstein, I.L. 1988 Tomorrow's workforce today. Industry Week 41-43. 1992 Training in Organizations: Needs Assessment, Development, and Evaluation, 3rd ed. Pacific Grove, Calif.: Brooks/Cole.
Goodman, P.S. 1979 Assessing Organizational Change: The Rushton Quality of Work Experiment. New York: John Wiley & Sons.
Goodman, P.S., and T.L. Griffith 1991 Process approach to implementation of new technology. Journal of Engineering and Technology Management 8:161-185.
Goodman, P.S., R. Devadas, and T.L. Griffith 1988 Groups and productivity: analyzing the effectiveness of self-managing teams. Pp. 295-327 in J.P. Campbell and R.J. Campbell, eds., Productivity in Organizations. San Francisco: Jossey-Bass.
Gould, J.D. 1980 Experiments on composing letters: some facts, some myths, and some observations. Pp. 97-118 in L. Gregg and E.R. Steinberg, eds., Cognitive Processes in Writing. Hillsdale, N.J.: Erlbaum and Associates.
1982 Writing and speaking letters and messages. International Journal of Man-Machine Studies 16:147-171.
Gould, J.D., and S.J. Boies 1983 Human factors challenges in creating a principal support office system: the speech filing system approach. ACM Transactions of Office Information System 4(1):273-298.
Gould, J.D., J.P. Ukelson, and S.J. Boies 1993 Improving application development productivity by using ITS. International Journal of Man-Machine Studies 39:113-146. Also available as IBM Research Report, 1991, RC-17496.
Gray, C.L., Jr., and J.A. Alson 1989 The case for methonal. Scientific American 261(5):108-114.
Guzzo, R.A. 1988 Productivity research: reviewing psychological and economic perspectives. Pp. 63-82 in J.P. Campbell and R.J. Campbell, eds., Productivity in Organizations. San Francisco: Jossey-Bass.
Hackman, J.R., ed. 1990 Groups That Work (and Those That Don't): Creating Conditions for Teamwork. San Francisco: Jossey-Bass.
Hamilton, P., and D. Warburton, eds. 1979 Human Stress and Cognition. Chichester, England: Wiley.
Harris, D.H. 1980 Visual detection of driving while intoxicated. Human Factors 22(6):725-732.
Harter, S.P., and A.R. Peters 1985 Heuristics for online information retrieval: a typology and preliminary listing. Online Review 9(5):407-424.
Hawkins, F. 1988 Human Factors in Flight. Brookfield, Vt.: Gower.
1975 The ARPANET network. In R.L. Grimsdale and F.F. Kuo, eds., Computer Communication Networks: 1973 Proceedings of the NATO Advanced Study Institute. Leyden, Netherlands: Noordhoff International Publishing.
Heart, F., A. McKenzie, J. McQuillan, and D. Walden
1978 ARPANET Completion Report. Cambridge, Mass.: Bolt Beranek and Newman Inc.
1992 Human factors aspects of the Air Ontario crash at Dryden, Ontario. In V.P. Moshansky, ed., Commission of Inquiry into the Air Ontario Accident at Dryden, Ontario: Final Report. Ottawa, Canada: Minister of Supply and Services.
Hockey, G.R.J., ed.
1983 Stress and Fatigue in Human Performance. Chichester, England: Wiley. 1986 Changes in operator efficiency. In K. Boff, L. Kaufman, and J. Thomas, eds., Handbook of Perception and Performance, Vol. II. New York: Wiley.
Hockey, R., and P. Hamilton
1983 The cognitive patterning of stress states. Pp. 331-362 in G.R.J. Hockey, ed., Stress and Fatigue in Human Performance. Chichester, England: Wiley.
Hockey, G.R.J., A.W.K. Gaillard, and M.G.H. Coles, eds.
1986 Energetics and Human Information Processing. Dordrecht, Netherlands: Martinus Nijhoff.
1986 Environmental psychology. Annual Review of Psychology 37:381-407.
Houghton, R.A., and G.M. Woodwell 1989 Global climatic change. Scientific American 260(4):36-44.
Human-Computer Interaction, special 1992 issue.
Interacting with Computers, special 1992 issue.
Jensen, C.R., and L.A. Anderson 1987 Comparing three dimensional representation of data to scatterplots. Pp. 1174-1178 in Proceedings of the 31st Annual Meeting of the Human Factors Society. Santa Monica, Calif.: Human Factors Society.
Johnson, C.A., and J.L. Keltner 1983 Incidence of visual field loss and its relation to driving performance. Archives of Ophthalmology 101:371-375.
Johnston, W., and A. Packer 1987 Workforce 2000: Work and Workers for the 21st Century. Indianapolis, Ind.: Hudson Institute.
Jones, S.G. 1993 Human factors in incident reporting. Pp. 567-572 in Proceedings of the Seventh International Symposium on Aviation Psychology. Columbus: Ohio State University.
Keinan, G. 1987 Decision making under stress: scanning of alternatives under controllable and uncontrollable threats. Journal of Personality and Social Psychology 52:639-644.
Kellso, J.R. 1989 CIM in action: microelectronics, manufacturer charts course towards true systems integration. Industrial Engineering 21:18-22.
Kelly, R.T., J.R. Callan, T.A. Kozlowski, and E. Menngola 1990 Human Factors in Self-Monitoring of Blood Glucose. Task 4 Final Report. FDA/CDRH-90/60. Springfield, Va.: NTIS.
Kerr, E.B., and S.R. Hiltz 1982 Computer-Mediated Communication Systems: Status and Evaluation . New York: Academic Press.
Khalil, T.M., E. Abdel-Moty, and T.M. Asfour 1990 Ergonomics in the management of occupational injuries. Pp. 41-53 in B.M. Pulat and D.C. Alexander, eds., Industrial Ergonomics: Case Studies. Norcross, Ga.: Industrial Engineering and Management Press.
Kiger, J.L. 1984 The depth/breadth trade-off in the design of menu-driven user interfaces. International Journal of Man-Machine Studies 20:201-213.
Klemmer, E.T., and F.W. Snyder 1972 Measurement of time spent communicating. Journal of Communication 22:148-158.
Knights, D., H. Wilmott, and D. Collinson 1985 Job Redesign: Critical Perspectives on the Labor Process. Brookfield, Vt.: Gower Publishers.
Kobasa, S.C. 1979 Stressful life events, personality, and health: an inquiry into hardiness. Journal of Personality and Social Psychology 37:1-11.
Koenig, M.E.D. 1990 Information services and downstream productivity. Annual Review of Information Science and Technology 25:55-86.
Kornhauser, A. 1965 The Mental Health of the Industrial Worker. New York: Wiley.
Kraus, L.E., and S. Stoddard 1989 Chart Book on Disability in the United States: An InfoUse Report . National Institute on Disability and Rehabilitation Research. Washington, D.C.: U.S. Department of Education.
Kraut, R.E. 1987 Technology and the Transformation of White Collar Work. Hillsdale, N.J.: Lawrence Erlbaum Associates.
Kraut, R.E., S. Dumais, and S. Koch 1989 Computerization, productivity, and quality of work-life. Communications of the ACM 32(2):220-238.
Kraut, R.E., and L.A. Streeter 1995 Coordination in software development. Communications of the ACM 38(3):69-81.
Kuo W., and J.P. Hsu 1990 Update: simultaneous engineering design in Japan. Industrial Engineering 22:23-28.
Landauer, T., D. Egan, J. Remde, M. Lesk, C. Lochbaum, and D. Ketchum 1993 Enhancing the usability of text through computer delivery and formative evaluation: the SuperBook project. In C.M. McKnight, A. Dillon, and J. Richardson, eds., Hypertext: A Psychological Perspective . New York: Horwood.
la Riviere, J.W.M. 1989 Threats to the world's water. Scientific American 261(3):80-94.
Latham, G. 1988 Human resource training and development. Annual Review of Psychology 39:545-582.
Levi, L. 1990 Occupational stress: spice of life or kiss of death? American Psychologist 45:1142-1145.
Liu, Y., and C.D. Wickens 1992 Visual scanning with or without spatial uncertainty and divided and selective attention. Acta Psychologica 79:131-153.
Locke, E.A., and G.P. Latham 1990 A Task Theory of Goal Setting and Task Performance, Englewood Cliffs, N.J.: Prentice Hall.
MacDuffie, J.P., and J.F. Krafcik 1990 Integrating Technology and Human Resources for High Performance Manufacturing: Evidence from the International Auto Industry. Paper presented at the Transforming Organizations Conference, Massachusetts Institute of Technology, Boston.
Mackinlay, J.D., G.G. Robertson, and S.K. Card 1991 The perspective wall: detail and context smoothly integrated. Pp. 173-179 in S.P. Robertson, G.M. Olson, and J.S. Olson, eds., Human Factors in Computing Systems: Reaching Through Technology. CHI '91. New York: Association for Computing Machinery.
Makhoul, J., F. Jelinek, L. Rabiner, C. Weinstein, and V. Zue 1990 Spoken language systems. Annual Review of Computer Science 4:481-501.
Malone, T.W. 1983 How do people organize their desks: implications for designing office automation systems. ACM Transactions on Office Automation Systems 1:99-112.
Manuso, J. 1983 The Equitable Life Assurance Society program. Preventive Medicine 12:658-662.
McDonald, W.I. 1984 Quality control of home monitoring of blood glucose concentrations. British Medical Journal 288:1915.
Merwin, D.H., and C.D. Wickens 1991 Comparison of 2D planar and 3D perspective display formats in multidimensional data visualization. Proceedings of the International Society for Optical Engineering. Bellingham, Wash.: SPIE.
Miller, D.P. 1981 The depth/breadth tradeoff in hierarchical computer menus. Pp. 296-300 in Proceedings of the 25th Annual Meeting of the Human Factors Society. Santa Monica, Calif.: Human Factors Society.
Mintzberg, H. 1973 The Nature of Managerial Work. New York: Harper and Row.
Mohnen, V.A. 1988 The challenge of acid rain. Scientific American 259(2):30-38.
Moss, J.P., and D.E. Delawter 1986 Self-monitoring of blood glucose. American Family Physician 33:225-228.
Muckler, F.A. 1987 The human-computer interface: the past 35 years and the next 35 years. In G. Salvendy, ed., Cognitive Engineering in the Design of Human-Computer Interaction and Expert Systems, Proceedings of the Second International Conference on Human-Computer Interaction, Honolulu, Hawaii. Amsterdam, Netherlands: Elsevier.
Myers, N. 1989 Deforestation Rates in Tropical Forests and Their Climatic Implications . London, England: Friends of the Earth.
National Academy of Engineering/National Research Council 1991 People and Technology in the Workplace. Washington, D.C.: National Academy Press.
National Commission on Employment Policy 1986 Computers in the Workplace: Selected Issues. Washington, D.C.: National Commission on Employment Policy.
National Commission on Excellence in Education 1983 A Nation at Risk: The Imperative for Educational Reform. Washington, D.C.: U.S. Government Printing Office.
National Council on Education Standards and Testing 1992 Raising Standards for American Education. Washington, D.C.: U.S. Government Printing Office.
National Research Council 1977 Drinking Water and Health. Safe Drinking Water Committee, Commission on Life Sciences. Washington, D.C.: National Academy of Sciences.
1983 Changing Climate: Report of the Carbon Dioxide Assessment Committee . Washington, D.C.: National Academy Press.
1986 Human Resource Practices for Implementing Advanced Manufacturing Technology. Manufacturing Studies Board. Washington D.C.: National Academy Press.
1990 The Improvement of Tropical and Subtropical Rangelands. Board on Science and Technology for International Development, Office of International Affairs. Washington, D.C.: National Academy Press.
1991 Rethinking the Ozone Problem in Urban and Regional Air Pollution. Committee on Tropospheric Ozone Formation and Measurement, Board on Environmental Sciences and Toxicology. Washington, D.C.: National Academy Press.
1993 National Collaboratories: Applying Information Technology for Scientific Research,
Committee on a National Collaboratory: Establishing the User-Developer Partnership, Computer Science and Telecommunications Board. Washington, D.C.: National Academy Press.
Newby, G.B. 1992 An Investigation of the Role of Navigation for Information Retrieval. ASIS Meeting.
Newell, G.R., and V.G. Vogel 1988 Personal risk factors: what do they mean? Cancer 62:1595-1701.
Norman, K. 1991 The Psychology of Menu Selection. Hillsdale, N.J.: Erlbaum.
Office of Technology Assessment 1988 Urban Ozone and the Clean Air Act: Problems and Proposals for Change. Washington, D.C.: U.S. Government Printing Office. 1990 Worker Training: Competing in the New International Economy. Washington, D.C.: U.S. Government Printing Office.
Olson, J.S., G.M. Olson, M. Storrosten, and M. Carter 1992 How a group-editor changes the character of a design meeting as well as its outcome. Pp. 91-98 in J. Turner and R. Kraut, eds., Proceedings of the Conference on Computer-Supported Cooperative Work . New York: ACM Press.
O'Malley, K.G., and K.G. Ricca 1990 Optimization of a PACS display workstation for diagnostic reading. Pp. 940-946 in Medical Imaging IV: PACS System Design and Evaluation . Proceedings of SPIE, the International Society for Optical Engineering in cooperation with the American Association of Physicists in Medicine, Vol. 1234. Bellingham, Wash.: SPIE.
Panko, R.R. 1992 Managerial communication patterns. Journal of Organizational Computing 2(1):95-122.
Parkes, K.R. 1986 Coping in stressful episodes: the role of individual differences, environmental factors, and situational characteristics. Journal of Personality and Social Psychology 51:1277-1292.
Pickett, R.M., and T.J. Triggs, eds. 1974 Human Factors in Health Care. Lexington, Mass.: D.C. Heath.
Piore, M., and C. Sabel 1984 The Second Industrial Divide. New York: Basic Books.
Pool, R. 1993 Beyond databases and e-mail. Science 261(August 13):841-843.
Postel, S. 1985 Thirsty in a water-rich world. International Wildlife 15(6):32-37.
Reason, J. 1990 Human Error. New York: Cambridge University Press.
Repetto, R. 1990 Deforestation in the tropics. Scientific American 262(4):36-42.
Roske-Hofstrand, R.J., and K.R. Paap 1986 Cognitive networks as a guide to menu organization: an application in the automated cockpit. Ergonomics 29(11):1301-1311.
Rosomoff, H.L., C. Green, M. Silbert, and R. Steele 1981 Pain and low back rehabilitation program at the University of Miami School of Medicine. In K.Y. Lorenzo, ed., New Approaches to Treatment of Chronic Pain. NIDA Research Monograph 36. Washington, D.C.: U.S. Department of Health and Human Services.
Russell, J.A., and L.M. Ward 1982 Environmental psychology. Annual Review of Psychology 33:651-688.
Saegert, S., and G.H. Winkel 1990 Environmental psychology. Annual Review of Psychology 41:441-477.
Salvaggio, J.L. 1989 The Information Society: Economic, Social, and Structural Issues . Hillsdale, N.J.: Lawrence Erlbaum.
Sanderson, P.M. 1989 The human planning and scheduling role in advanced manufacturing systems: an emerging human factors domain. Human Factors 31(6):635-666.
Schlesinger, W.H., J.F. Reynolds, G.L. Cunningham, L.F. Huenneke, W.M. Gerrell, R.A. Virginia, and W.G. Whitford 1990 Biological feedbacks in global desertification. Science 247:1043-1048.
Schwartz, S.E. 1989 Acid deposition: unraveling a regional phenomenon. Science 243:753-763.
Seidler, K.S., and C.D. Wickens 1992 Distance and organization in multifunction displays. Human Factors 34:555-569.
Selye, H. 1956 The Stress of Life. New York: McGraw-Hill.
Senders, J., and N. Moray, eds. 1991 Human Error: Cause, Prediction, and Reduction. Hillsdale, N.J.: Erlbaum.
Shneiderman, B. 1987 Designing the User Interface: Strategies for Effective Human-Computer Interaction. Reading, Mass.: Addison-Wesley.
Smith, M.J. 1987 Occupational stress. Pp. 844-860 in G. Salvendy, ed., Handbook of Human Factors. New York: Wiley.
Snowberry, K., S.R. Parkinson, and N. Sisson 1983 Computer display menus. Ergonomics 26(7):699-712.
Soule, M.E. 1991 Conservation: tactics for a constant crisis. Science 253:744-750.
Sproull, L., and S. Kiesler 1991 Computers, networks and work. Scientific American 265(3):116-123.
Steinhart, P. 1990 No net loss. Audubon July:18-21.
Stern, P.C. 1992 Psychological dimensions of global environmental change. Annual Review of Psychology 43:269-302.
Stern, P.C., O.R. Young, and D. Druckman, eds. 1992 Global Environmental Change: Understanding the Human Dimensions . Committee on the Human Dimensions of Global Change, National Research Council. Washington, D.C.: National Academy Press.
Stolarski, R.S. 1988 The Antarctic ozone hole. Scientific American 258(1):30-36.
Stolarski, R.S., R. Bojkov, L. Bishop, C. Zerefos, J. Staehelin, and J. Zawodny 1992 Measured trends in stratospheric ozone. Science 256:342-349.
Strassmann, P.A. 1985 Information Payoff: The Transformation of Workers in the Electronic Age. New York: Free Press.
Stubbs, D.A., P.W. Buckle, M.P. Hudson, and P.M. Rivers 1983 Back pain in the nursing profession, II. The effectiveness of training. Ergonomics 26:767-779.
Swanson, G.M. 1988 Cancer prevention in the workplace and natural environment: a review of etiology, research design, and methods of risk reduction. Cancer 62:1725-1746.
Swets, J.A. 1988 Measuring the accuracy of diagnostic systems. Science 240:1285-1293.
Swets, J.A., R.M. Pickett, S.F. Whitehead, D.J. Getty, J.B. Schnur, J.B. Swets, and B.A. Freeman 1979 Assessment of diagnostic technologies. Science 205:753-759.
Tannenbaum, S., and G. Yukl 1991 Training and development in work organizations. Annual Review of Psychology 41:399-441.
Turner, J., and R. Kraut, eds. 1992 Proceedings of the Conference on Computer-Supported Cooperative Work. New York: ACM Press.
Vachon, R.A. 1990 Employing the disabled. Issues in Science and Technology 6(2):44-50.
Vanderheiden, G.C. 1990 Thirty-something million: should they be exceptions? Human Factors 32:383-396.
Vigil, P. 1985 Computer literacy and the two cultures revisited. Pp. 240-242 in C.A. Parkhurst, ed., Proceedings of the American Society for Information Science (ASIS) 48th Annual Meeting. White Plains, N.Y.: Knowledge Industry Publications Inc.
Vincente, K.J., and R.C. Williges 1988 Accommodating individual differences in searching a hierarchical file system. International Journal of Man-Machine Studies 29:647-668.
Walker, C., and R. Guest 1952 Man on the Assembly Line. Cambridge, Mass.: Harvard University Press.
Wallace, D.R. 1985 Wetlands in America: labyrinth and temple. Wilderness 49:12-27.
Weischedel, R., J. Carbonell, B. Grosz, W. Lehnert, M. Marcus, R. Perrault, and R. Wilensky 1990 Natural language processing. Annual Review of Computer Science 4:435-452.
Wexley, K. 1984 Personnel training. Annual Review of Psychology 31:519-551.
Wickens, C.D. 1992 Engineering Psychology and Human Performance, 2nd. ed. New York: Harper Collins.
William T. Grant Foundation 1988 The Forgotten Half, Pathways to Success for America's Youth and Young Families. Commission on Work, Family, and Citizenship. Washington, D.C.: William T. Grant Foundation.
Williams, G.M. 1991 Causes and prevention of cancer. Statistical Bulletin of Metropolitan Insurance Companies 72:6-10.
Wilson, E.O. 1989 Threats to biodiversity. Scientific American 261(3):108-116.
Wulf, W. 1993 The collaboratory opportunity. Science 261(August 13):854-855.