11
Aiding Intellectual Work

John D. Gould

INTRODUCTION

Nearly all human intellectual work is aided by technology. In the next decade, new technology will make possible many new potential aids. The main thrust of this chapter is to structure the problem of aiding intellectual work in such a way that human factors researchers can contribute toward the development and evaluation of electronic systems that will indeed aid their users.

The Problem

The problem focused on in this chapter is how to carry out human factors research aimed at augmenting human intellectual work with electronic aids. Aids include artifacts or tools, as well as procedures, techniques, organizational structures, and facilitated, encouraged communication patterns. Most human intellectual work tends to be mentally demanding, with a priority on timeliness and with a high premium on getting the ''right information" (i.e., not just the right answer to a question asked, but information that is relevant to questions that should have been asked but were not).

Today, all human intellectual work is augmented to some degree. Electronic aids for intellectual work include ubiquitous, simple "single-function" tools, some of which are largely electronic versions of established



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research 11 Aiding Intellectual Work John D. Gould INTRODUCTION Nearly all human intellectual work is aided by technology. In the next decade, new technology will make possible many new potential aids. The main thrust of this chapter is to structure the problem of aiding intellectual work in such a way that human factors researchers can contribute toward the development and evaluation of electronic systems that will indeed aid their users. The Problem The problem focused on in this chapter is how to carry out human factors research aimed at augmenting human intellectual work with electronic aids. Aids include artifacts or tools, as well as procedures, techniques, organizational structures, and facilitated, encouraged communication patterns. Most human intellectual work tends to be mentally demanding, with a priority on timeliness and with a high premium on getting the ''right information" (i.e., not just the right answer to a question asked, but information that is relevant to questions that should have been asked but were not). Today, all human intellectual work is augmented to some degree. Electronic aids for intellectual work include ubiquitous, simple "single-function" tools, some of which are largely electronic versions of established

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research books (e.g., dictionaries, thesauruses, foreign language translators) and some of which are more (e.g., calculators, spelling verifiers). There are spreadsheets for business people, high-level programming languages and tools for computer application developers, Cadam and very large-scale integrated (VLSI) tools for hardware designers, and conferencing systems for group work. To help users transfer learning from one computer application to another, software developers have attempted to standardize user interface styles (e.g., Apple Corporation's Macintosh style and IBM's CUA style). Many jobs simply could not be performed without electronic aids (e.g., work related to forecasting, reservations, large databases, simulations, and much medical practice). Aiding intellectual work performance is a broad topic. Several domains have been addressed by recent National Research Council committees and are not explicitly covered here. One report summarized evidence on whether human performance is enhanced through techniques that have their roots in academic research, as well as techniques that have their roots in commercial enterprise. The various techniques used to enhance human performance include training, pain management, stress management, expert guidance, meditation, self-help subliminal audiotapes, self-assessment techniques, mind-altering drugs, and sports-psychology techniques to aid performance under pressure (see Druckman and Bjork, 1991; Druckman and Swets, 1988). Another National Research Council report (Ferber et al., 1991) discusses how human intellectual work performance can be affected by the linkages between work and such family and personal areas as finances, marital status, child care, the need to support elderly relatives, marital stress, illness, and working at home. Other studies have reviewed intellectual work aids for (a) people with special needs (e.g., sensory and motor impairments, communication difficulties) (see Chapter 3) and (b) students in primary, secondary, and collegiate schools. Why Is Aiding Intellectual Work Important? There are three main reasons for understanding how to better aid human intellectual work. It Increases Productivity and Creativity Productivity is a national problem: many studies report that the productivity of the U.S. labor force is relatively stagnant. Some widely used electronic aids have not helped much. Nevertheless, there is the belief, supported by informal evidence, that electronic aids, if developed with users and their work organizations in mind, do enhance productivity and creativity and can allow people to work in new ways. For example, using

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research electronic spreadsheets, business professionals routinely run "what if" analyses in seconds, a task that previously took hours or days or that would simply not have been done a few years ago. A variety of intellectual workers use simulations to provide quick studies of situations that previously could be studied only by actually building an object itself (if indeed it could be built). Scientists move single atoms, study their effects, and produce new combinations with awesome possibilities. With new visualization systems, engineers and scientists can see how wings are affected by wind or how a thunderstorm forms and thus develop an understanding that cannot be gained from studying the thermodynamics and the underlying isolated equations themselves. Powerful visualization machines have led to a change in the design of drugs: "scientists use graphics workstations to visualize a protein in three dimensions and determine which drugs will fit best into its active sites. Such visualization is so powerful that no pharmaceutical researcher would try to do the job without it" (Pool, 1992). An entire new field (experimental mathematics) and a way of thinking for mathematicians has developed based upon these new visualization tools (Pool, 1992). It Is Socially Responsible The abilities of people to cope with increasing intellectual work demands and their resulting self-images are affected by tools they are given (e.g., see the section below on "Organizational Impact of a New Technology"; see also Kraut et al., 1989). Some workers are deficient in problem-solving skills, spelling, composing, calculating, language skills (including second language problems), giving instructions, and so forth. Intellectual aids have the potential to reduce these deficiencies. Intellectual aids also have the potential to help people deal with problems outside work (e.g., financial planning, finding support groups, interacting with their children), and these, in turn, impact on their work lives (Ferber et al., 1991). Human factors researchers can help identify which intellectual tasks, if aided, could lead to the most benefits (for individuals and society), and they can help identify aids that can effectively help people with specific intellectual deficiencies compensate for what is latent or missing. Our Expanding Technology Makes It Easier to Do Rapid advances in electronic technology provide opportunities to improve existing intellectual aids, create effective and unforeseen new ones, and reduce costs, thereby bringing heretofore expensive aids to larger audiences. The productivity and creativity of individuals and work groups can be dramatically affected, in unforeseen ways, by the faster, cheaper processors with larger storage that are becoming available. In the next decade, the

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research costs of electronic storage and of computer processing are expected to continue decreasing at about the same rate as they have been (about 10 to 20 times per decade). The speed of computer processing is expected to continue increasing at about the same rate as it has been (about 10 to 20 times per decade). Communications bandwidth is expected to grow even more dramatically—about 100 times during the next decade. Human Factors Special Interest The evaluation and creation of intellectual aids draw on a variety of disciplines involved in human factors work, including assessment, cognitive science, applied and experimental psychology, work science, social science, organizational theory, and systems theory. It is the human-centered focus and the assessment and system development methodologies pioneered by these disciplines that provide the potential contributions. THREE RECOMMENDED HUMAN FACTORS RESEARCH NEEDS AND STRATEGIES Three human factors research efforts are recommended to improve human intellectual aids. Although they differ in emphasis, the three research needs overlap, as do their respective research strategies and methodologies. Several studies used as examples to illustrate one human factors research strategy can, from a different angle, be used to illustrate one of the other research strategies. Assessing Existing Intellectual Aids Today, little experimental and formal empirical evidence exists as to the impact of various electronic aids on individual worker and organizational productivity. Human factors research should be directed at empirically and experimentally assessing the value of existing intellectual aids, particularly as related to individual and organizational productivity. Exploring the Nature of the Tasks to be Aided The fundamental nature of the intellectual tasks to be aided must be understood. Stated differently, the characteristics of aids that will improve worker and organizational productivity and workers' quality of life must be discovered. Human factors research should empirically study ongoing intellectual work to identify which work tasks are "aidable," the cognitive characteristics of these tasks, and the characteristics that the aids should have. There exist a variety of human factors methodologies that can be

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research used to conduct empirical investigations of intellectual work in creative jobs and work organizations in order to understand what types of intellectual aids would be valuable. These methods include serious observational visits to work locations, task analyses, interviews, and thinking-aloud protocols. Participating in Interdisciplinary Efforts The disciplines that underlie human factors research are necessary to improve existing intellectual aids and to create new useful ones. But they are not sufficient. Human factors researchers should participate in serious interdisciplinary efforts with other scientists and engineers. This is necessary both because human factors researchers have the potential to make a contribution and to help ensure that human factors research has a significant impact. This chapter provides frameworks for thinking about how to address these three research needs, suggestions for following the recommended research strategies, and examples of existing research. RESEARCH NEED AND STRATEGY 1: ASSESS EXISTING AIDS It is important to know the effectiveness of the various intellectual work aids. It would seem reasonable to assume from observation that, to augment their intellectual work, people are successful in using some aids (e.g., actuarial tables) but not others (e.g., whatever aids many mutual fund managers used in the last decade). Although free enterprise market forces and differential commercial success of various intellectual work aids would seem to be a good indicator of their effectiveness, this may not always be the case. The experimental results that do exist sometimes question the value of the generally accepted intellectual aid studied (see the examples of studies below; see also Attewell, 1994). Future human factors research should assess existing, ubiquitous intellectual aids to learn how well, if at all, they aid human intellectual performance and productivity; to understand more deeply how they affect people's minds and the productivity of their work organizations; and to use the results as insights in defining new aids that would be valuable. In addition to increasing productivity, knowledge about the effectiveness of intellectual work aids might help identify their common characteristics that have long-term positive effects (e.g., making workers more productive, improving the quality of work life). These discoveries might center on characteristics of the aids themselves or on characteristics of the processes used to develop them. For example, a number of questions might be asked about aid development processes. What design processes lead to the best aids? Do the

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research system design processes recommended by human factors people lead to good aids (e.g., iterative design, user-centered design; see Helander, 1988)? If so, then are there more case study data in support of these design and development processes? If not, then these recommended processes can be modified. Experimental assessment of intellectual aids could help identify the cognitive implications of widely used aids. For example, does a calculator lead students to become poorer at basic math skills, as is often suspected, or better at math skills, which one could argue on the basis of accurate feedback? Intellectual aids can have surprising, unpredictable effects and intellectual implications. A personal example is presented here: Prior to the availability of copiers, I used to make notes and know a relatively small number of journal articles in detail; now I have a more peripheral knowledge of a larger number of journal articles and, when I need to know any details, I rely on remembering the existence and location of the copies that I have stacked in my office. Experimental studies of existing aids would focus energy on tasks for which the aids are designed to help, rather than on convenient lab or toy tasks that are often used in psychological investigations. This may lead to developing new methodology and a worthwhile theory about the work under study. It also can lead either to the identification of ways to improve an aid or to ideas about the design of new aids. Figure 11.1 summarizes key points to be kept in mind when designing experiments or empirical field studies to assess an existing intellectual aid. Although a few readers may think that some of these points are obvious, many of us are familiar with studies that fail to follow "obvious" suggestions. Examples of Studies A few studies are described here to illustrate how to successfully assess a potential aid. Composing Aids Composing by Dictation In the 1970s dictation was often used to produce office correspondence. Vendors and employers encouraged their professional employees to compose letters and memos this way; office dictation systems, including hardware, software, and required procedures, were being designed, purchased, and used. There were several generally accepted notions about dictation at that time, as summarized in Figure 11.2. To test these commonly held views, in a series of about 20 laboratory experiments, participants were given a specific topic on which to compose a letter and a specific composition method with which to do it. They composed similar

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research Key Methodological Points in Assessing Existing Aids • Try to pick a ubiquitous, important task and a ubiquitous, important aid. • Continually ask the question: does this aid do what it claims to do? Does it do it in the SHORT run and in the LONG run? • Figure out how the task can be studied experimentally in the laboratory or empirically in the field. Often short-term laboratory experiments are not as informative as longer-term empirical studies conducted in context. • In laboratory experiments, include appropriate experimental controls, e.g., have participants do the same work with and without the aid (if this is possible). In empirical field studies, strive to do comparative studies of multiple groups or longitudinal studies of the "aided" group, especially if newly "aided." • Use tasks, equipment, material, procedures which relevant users and vendors would agree, a priori, provide a fair evaluation and could lead to their altering their behavior depending upon the outcome. • Design experiments so that results generalize to intended users (e.g., use appropriate people as experimental participants) and to a useful range of variation in the intellectual task under study (e.g., if it is querying a database make sure enough different types of queries are studied). • Choose meaningful dependent variables, including performance ones and attitudinal ones. • Try to identify and understand theoretically the underlying mental processes involved in the tasks and use of the aid. • Provide limits of generalization of the results in the study report. • Suggest how to improve the aid studied, or the possible use of alternative ones. • Prior to starting the experimental work, write conclusions sections that vary depending upon what is found. Show them to a few friends and get their reaction. This will help you determine whether it will be possible to draw any conclusions from the study design under consideration. FIGURE 11.1 Key points in designing experiments to assess existing intellectual aids. letters with each method studied. Their performance and behavior were measured while they composed, and the quality of their work was measured in various ways. The results showed that, contrary to what was generally supposed, it does not take a long time to learn to dictate with dictating equipment (Gould, 1978a). Eight college graduates who had never dictated spent part of one day learning to dictate to a machine. The next day, they dictated four business letters and hand wrote four similar business letters. The experimental results showed that they dictated and wrote letters in about the same time and with about the same resulting quality. Quality was rated by judges who viewed typed copies of the letters and who did not know which method was used to compose them.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research Common Notions about Dictation in the 1970s 1. "Dictating requires a long time to learn." 2. "Eventually, dictating is much faster than writing. That is, potential maximum output rates, as measured informally, are 200 words per minute (wpm) and 40 wpm, respectively. 3. "Dictating may be qualitatively superior to writing because the higher potential output rate permits faster transfer of ideas from limited capacity working memory, thus reducing forgetting through interference or decay." 4. On the other hand, handwriting was thought to have "an advantage over dictating because it is easier to review and modify." FIGURE 11.2 Hypotheses about different composition methods in the 1970s. SOURCE: Gould (1980:101). A second key result was that, contrary to what was generally supposed, (Figure 11.2), people experienced at using dictation equipment did not dictate several times faster than they wrote. Indeed, eight business executives who dictated regularly for years dictated routine business letters about 60 percent faster than they wrote them (p < .001), they dictated more complex letters (e.g., essays on the Bicentennial celebration, on the issue of capital punishment, on their favorite teacher) about 25 percent faster than they wrote them (p < .01) (Gould, 1978b). These experienced dictators dictated routine business letters about 20 percent faster than did novice dictators (p < .05), and they dictated more complex letters in about the same time as did the novice dictators. Dictated and handwritten letters, when typed, were rated as similar by judges blind as to the method of composition and the experience of the authors. Part of the aid assessment strategy recommended in this chapter is to develop a theoretical understanding of users' tasks and to relate the results of experiments to that understanding. These dictation studies found that the reason people do not dictate five times faster than they write, even though they can say words five times faster than they can write them (40 wpm vs. 200 wpm), is that the generation component of composition (i.e., the actual moving of one's hand or mouth) takes only a small fraction (25 percent or less) of the total time. Planning (i.e., pausing and thinking) takes two-thirds of the total time. A third key result was the discovery that inexperienced dictators feel it takes them longer to learn to dictate than it actually does because they think the quality of their dictation is poorer than it actually is. Novice dictators, just after composing a letter, rated their dictated letters as significantly poorer than their written letters (Gould and Boies, 1978). Upon receiving a typed copy of their dictated and written letters and incorporating proofreading and editing changes, they elevated the ratings of their dictated letters to those of their handwritten letters. Experienced dictators, on the other hand,

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research rated their dictated and written letters equivalently at both stages. Judges later rated the quality of the final versions of the dictated and written letters as equivalent. A fourth key result had to do with the just-emerging method of composing by speaking "voice documents," which recipients listen to (rather than read). Experimental results showed speaking to be a much faster method of composing than dictating or writing. Authors could use syntax, phraseology, and organization that they had learned over the years to be appropriate for listening but not necessarily for reading. In addition, in contrast to dictating, authors did not have to give typing instructions, which is potentially a cognitively disrupting "secondary" task (Gould, 1980). This information on voice documents had an impact on technology. In the middle 1970s, researchers at IBM had begun work on a "super dictating system," one that allowed users to apply great editing power to their aural documents. The results of the research project described here were a key factor in IBM's decision to shift the emphasis away from a super dictation system and toward a voice messaging system (Gould and Boies, 1983). Since then, voice messaging has grown dramatically as an industry (e.g., voice mail, telephone answering machines, etc.), whereas dictation has declined. Composing With Text Editors In the 1970s, office professionals began using text editors to compose their own memos and documents—and to create the final text versions without the aid of secretaries. This trend led to research in the late 1970s to experimentally assess the productivity of professionals using text editors to compose documents (Gould, 1982), the method of composition that has since become dominant in much American industry. In the first study, 10 professionals, who regularly used a computer-based, mainframe, line-oriented text editor to compose their own correspondence and longer documents, composed four letters with a text-editor (T letters) and four letters with handwriting (W letters). A secretary typed each W letter, and then the participants revised and proofread the typed copy. The results questioned the productivity advantage of using the line-oriented text editors of the 1970s. The basic result was that participants spent significantly more time composing T letters (29.5 minutes) than they did composing W letters (19.2 minutes) (p <.001). There was no difference in the judged quality and effectiveness of the T and W letters (based upon ratings by judges afterwards). Participants made many more modifications to T letters (41.3) than to W letters (8.5). There was no difference in the length of T and W letters, and there were no observable differences in the style of T and W letters. The report concluded that no productivity advantages were found for professionals who composed their own letters with their own text editor.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research These results seemed questionable to some, particularly because of the lack of differences in quality between the T letters, with all their additional modifications, and the W letters. A few years later, Card et al. (1984) repeated this study, using the same tasks, materials, and procedure, but using a "display-oriented" editor (called Bravo) rather than a line-oriented editor. Again, participants' T and W letters were of the same length and were judged to be of the same quality and style. Again, participants made several times more modifications to T letters than to W letters, and again these did not lead to higher quality in the T letters. However, in this experiment the composition times for the two methods were about equivalent (mean times for T and W, respectively, were 22.8 and 21.7 minutes versus 29.5 and 19.2 minutes in the Gould study). That is, the mean time for T letters was reduced to about that of W letters. Besides extending the findings to other editors, an important point for this chapter is that results showed that experiments aimed at assessing aids can lead to cumulative knowledge. Comment Note how these experiments reflect the recommended methodological points of Figure 11.1. Important aids were studied. Participants were experienced with all methods of composition. This helps generalization to the intended audience. Appropriate tasks, that is, letters rather than, for example, sentences or minor utterances, were used. A variety of letter types were studied, which leads to greater generalization of the results. Meaningful dependent variables—time and effectiveness—were measured in a variety of ways. In real work, these variables reflect productivity. Since the goal of communication is often to persuade, having judges select the most persuasive letters in some of the studies was better than simply judging spelling, syntax, and stylistic considerations. The experimental results were repeatable, extendable, and cumulative. Some of the work had an impact on composing technology. Human factors research should not just question the value of existing aids but, whenever possible and perhaps aided by theory, find something that is a good replacement. Speaking documents as a method of composition was so identified. Each report contained a theoretical explanation of the results, a serious discussion of limitations, and a pointed conclusion paragraph. For example, the Gould (1982) report on text-editing systems pointed out that only one text editor was studied; that even though there were no other experimental data on the use of text editors in composing tasks, it was known that the text editor used can affect the speed of routine secretarial editing tasks (Roberts, 1979); that line-oriented editors can lead to longer editing times for very simple editing tasks than display-oriented editors (Card et al., 1979); that the results are limited to one-to-two-page letters, typical of those composed

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research in industry; that these were not letters in which participants were deeply ego-involved or did an extensive number of drafts. Missing from these studies, however, was a serious discussion, based upon interviews with participants afterwards, about why they were not going to modify their behavior after learning about the results. In other words, a powerful dependent variable in assessment studies is to try to take some ''aid" away from people and see how much they resist this or fight to get it back. Related to this is the observation that many developers do not, even when appropriate, themselves use the aid that they are making for others to use. Everyday Aids Norman (1988) analyzed the way many "everyday" aids (e.g., doorknobs, VCRs, stoves, faucets, automobile controls) work in his popular book on the subject. The book is valuable for its motivational, theoretical, and methodological content. It is largely anecdotal, empirical, and analytical (rather than experimental) in its approach. It shows an excitement to get started, a motivation to focus on existing aids. It articulates theories of mental mechanisms involved in using common aids. It correctly describes how successful artifact design proceeds. The book finds fault with many existing implementations of intended aids, but sometimes points to successful ones as well. Norman's methods can be extrapolated to aiding intellectual work. Behavioral and Motivational Effects of Automation One work setting that has become increasingly automated is the flight deck of modern jet transports. Many intellectual tasks, such as flight planning and navigation, have been delegated to flight management computers, with the tasks of flying becoming increasingly centered on entering data and monitoring electronic displays. This, of course, has been done to "aid" the crew and improve airline safety. But some researchers have raised questions about unintended consequences of this automation, such as complacency, overreliance on malfunctioning automated systems, reduced job satisfaction, changes in authority relationships, and loss of skills. One recent study found no differences in technical performance (errors) between crews flying an automated version and those flying a standard version of the same aircraft on a simulated flight involving mechanical malfunctions and diversions from planned routing (Wiener et al., 1991). There are probably many broad, longer-term issues surrounding the impact of using "electronic colleagues" that go beyond performance results from laboratory studies.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research them in using the data, and design and carry out the necessary follow-up studies to verify whether the actuarial method helps in the new domain of concern. Additional Approaches There are several other approaches to studying ongoing intellectual work. There are many variations of task analysis methodology (see Drury et al., 1987). One can apply task analysis to the work of an isolated individual or to that of individuals working within an organization. Task analysis can be limited to ''logical analysis" of all the possible steps in a work process (as with an entirely new, nonexisting work system), or it can involve behavioral observations. Task analysis has been carried out mainly in military organizations, but there are many examples in industrial settings. The same handbook containing the Drury et al. paper contains chapters describing task analyses applied to process control systems (Woods et al., 1987) and to air traffic control systems (Lenorovitz and Phillips, 1987). Another possible approach is to carefully study, through interviews, observations, task analyses, and other methods, experienced, successful workers performing crafts of high intellectual content (e.g., a physician diagnosing, a nurse treating, a musician composing, an architect designing, a teacher teaching). The goals would be to identify the intellectual skills involved, develop a theory about the task, and identify the characteristics of an aid that would presumably be helpful. Some designers find that, in the early stages of designing and developing a complex system, they want a list of classes of subtasks that most need augmenting or even automating. The challenge is to identify in general terms, if possible, these classes (e.g., spatial orientation, data interpretation, handling radioactive materials). Another possible approach is to identify intellectual skills that cut across a large number of human intellectual tasks and then identify the characteristics of useful aids for that work. Existing examples of such aids are electronic verifiers for spelling and even grammar and electronic calculators for arithmetic. The idea is to determine not what the tasks have in common, but what common mental operations are carried out while working on a variety of tasks. RESEARCH NEED AND STRATEGY 3: INTERDISCIPLINARY DEVELOPMENT WORK Human factors researchers, to date, have been more likely to assess the value of an aid developed by others (e.g., the studies under strategy 1) than to join with others to develop an aid and then carefully assess its value and

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research impact. There is a need for human factors researchers to become involved in interdisciplinary efforts to create superior intellectual aids. This work is needed for several reasons. First, working alone is generally not enough. It is hard to find examples of human factors researchers who, by themselves, develop effective electronic aids. Making good electronic aids requires many disciplines. Second, many engineers and scientists have come to realize that "human factors" determine the difference between success and failure of most aids. Third, there is evidence that human factors involvement in development projects is cost-effective (Karat, 1992). Fourth, iterative testing and improvement of proposed aids are vital if the aids are to increase productivity and enhance the quality of work life. If human factors researchers are involved, such testing and improvement are likely to be carried out much more thoroughly. Thus, the potential aid under development has a greater probability of improving organizational or individual productivity or other aspects of work lives. Chapters 4 and 5 have made similar recommendations for interdisciplinary work in, respectively, emerging medical and environmental domains, areas that offer great rewards for successful collaboration. Fifth, successfully instantiating ideas into technology is a much bigger problem than most human factors researchers uninitiated in this process realize, and it is something they often cannot carry out on their own. This is true no matter how well the ideas seem to have worked in the experimental psychology laboratory or with prototypes. The studies referred to above on decision making and those described below by Landauer et al. (1993) illustrate this. Finally, many of the disciplines underlying human factors, such as computer science and experimental psychology, have a much larger supply of techniques, procedures, and results than there is a demand for by others. This proposed advanced development strategy can identify, select, and shape the techniques that others want and the context in which they must fit. Human factors researchers should become partners with others in designing, developing, and iteratively improving good aids. This recommended strategy is not just a plug for employment opportunities for human factors researchers or a profession-oriented expression of the importance of human factors. The idea is not just to work in support roles, as most industrial and government human factors workers now do, but to work as equal, long-term participating partners, with all the commitment, risk, and pressure that leadership demands. The idea is that as co-partners, human factors researchers should develop a sense of responsibility for the initiation and successful completion of an interdisciplinary project to produce a significant aid, with all the burdens this entails. For many human factors researchers, this requires new learning, motivations, work organizations, and professional payoff

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research (particularly as related to the university promotion system). It requires a new conceptualization of what constitutes legitimate research. As outlined below, there are already some situations that can serve as organizational models of how to do the work and write it up for professional credit. This recommended interdisciplinary work—wherein end users actually use, on a regular basis, the aid that human factors researchers jointly made with other necessary disciplines—may be hard to carry out because of personal and institutional barriers. It is longer-term work than is typical of many graduate school projects and may require longer-term commitments of funding than traditional research. To be successful, it will also require professional recognition and rewards, including ammunition for gaining tenure. There has been a start toward giving this work professional recognition. Case studies describing innovative research and development efforts involving human factors researchers in advanced development work exist, including Xerox's Star system (Smith et al., 1982), IBM's Audio Distribution System (Gould and Boies, 1984), and IBM's multimedia 1984 Olympic Message System (Gould et al., 1987). It is generally accepted that these systems were innovative and have influenced others who have made similar ones. Figure 11.5 summarizes key points to keep in mind carrying out interdisciplinary development work. These points are based upon personal experience and published case studies. Key Characteristics about Effective Multidisciplinary Development Work • Strong technical leader and mentor, not lacking greatly in people skills—a technical champion. • Small group of talented people. • Individual members do what they are good at. • Commitment to colleagues. • Drive to become famous as a group for making an outstanding aid. • Willing to do what is necessary to be successful—no 80/20 or 90/10 rule, where the remainder is left to others. • Use iterative engineering, including good development tools. • Make early prototypes. • Involve intended users from the outset. • Use the aid that you are making yourselves whenever possible (if applicable). • Follow up, and measure the value of the aid created. FIGURE 11.5 Key points in interdisciplinary work aimed at making effective aids for intellectual work.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research Examples of Studies SuperBook Many developers believe that electronic computers hold greater potential than printed books for helping people find desired information. Yet Landauer et al. (1993), working at Bellcore, noted that nearly every experimental study failed to find an advantage for various electronically presented text systems over comparable printed versions. They concluded that a significant human factors problem is to develop an electronic text system that people will use to better advantage than a standard printed book. Because of experimental results on the use of command languages and navigational tools, as well as results related to other relevant cognitive issues they had been studying (summarized in Landauer et al., 1993), Landauer et al. felt that they had a chance to change this history. They observed that much information already exists in published books and that this is the favorite medium of most authors. So, in contrast to most development efforts, which focus on electronic books only, they focused on how to help people find information in already published books that are also stored in computers. They called this approach SuperBook. They combined "full text indexing" with a "dynamic, hierarchical table of contents" of the stored book. A review of some of their previous research is in order. Two fundamental problems that users have in trying to find helpful information in electronic databases have to do with the command language they use and how they navigate through databases. With respect to command languages, almost all electronic information retrieval system—be they library subject catalogs, dial-up query systems, relational databases, or computer file systems—require users to select or enter some words that must match words used as identifiers by the authors or indexers. After many observations, Landauer et al. became convinced that most occasional users are usually unsuccessful in using standard query languages or in producing Boolean expressions from ands and ors to get desired information from databases. Thus, they rejected this approach as a way of helping casual users of electronic data bases. Why not discover what words people naturally prefer to use? When trying to devise new command names that would help novices learn word processors, the Bellcore group (Landauer et al., 1993:76) decided: that any two people were unlikely to agree on a best term, so the common expedient of a one word command, file name or table label will usually fail to put a user in touch with data stored by someone else. … The chances that any two people would choose the same name for a command were less than twenty percent, and the most popular term was chosen by only thirty percent of potential users. This observation was extended to the use of

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research terms for queries in data bases of several kinds, from classified ad listings to recipe files and program names. The Bellcore researchers built upon these observations by experimentally evaluating inventions of their own derived from their results. For example, unlimited aliasing (i.e., using every term that a sample of users apply to an item as an index to that item) led to a hit rate of over 70 percent in an interactive retrieval task on a small database of 188 recipes, with little increase in false positives. The same level of success was achieved with full text indexing (i.e., every word contained in each item in the database is part of the index for that item). Unfortunately, these results do not scale up to much larger databases because too many irrelevant items (i.e., false positives) are returned. Note how these researchers were not misled by optimistic lab results based upon limited tasks. With respect to navigation, the Bellcore experimental studies (Landauer et al., 1993:80) found that people have fundamental problems when using menus to navigate through electronic databases: Typically menu selection schemes have a branching factor of four to ten options; more often than not, users stray from the path to their target by the third hierarchical level. Put differently, if there are more than about thirty items in the data base, the chances of finding the right one in a single error-free traversal drop below half. They concluded provisionally that (Landauer et al., 1993:82): menu-traversal methods are fundamentally, and probably irremediably flawed as a primary method for information retrieval. As conceived by humans, almost all categories are essentially fuzzy; most items belong only partially or with only moderate probability to any one category, and can often fit reasonably well into several. The Bellcore group also retained a "rather cautious use of links in the SuperBook system" (Landauer et al., 1993:129). Thus, as the experiments showed, neither full text indexing nor a dynamic, hierarchical table of contents is sufficient by itself, but the Bellcore group thought a combination of the two might help. Full text indexing returns a superset of desired items, as they had already found, but the dynamic table of contents, they reasoned, should provide contexts to help users narrow this superset of items to those that are relevant to their intentions. So they built a prototype, and used it themselves (Landauer et al., 1993:98): We were … perfectly delighted with our system design. We were so overwhelmed with its obvious superiority to anything we had previously seen, and so pleased with our early in-house use that we dubbed it "SuperBook." We immediately began technology transfer efforts, efforts to make

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research this marvelous new tool available to other organizations at Bellcore. … Fortunately, [a new group member] reminded us of some of our public pronouncements about, and past experiences with the necessity of evaluation studies. To judge how well the SuperBook actually aided people, the Bellcore group gave college students queries which they had to answer using either a published statistics book or its electronic clone. Using the first version of the electronic aid (version 0), participants found slightly more correct answers with SuperBook than with the original printed book, but took significantly longer to do so. So the Bellcore group made many changes to its electronic aid, most based upon analyses of its experimental data. With the next version, participants answered their queries faster and more accurately with SuperBook than with the original printed book. Based upon what they learned in this experiment, the Bellcore researchers made more changes to their electronic aid and tested it again. This third evaluation study found that participants performed even better with SuperBook. This interdisciplinary effort illustrates many important points. No one discipline could have done all the design, implementation, testing, and transfer efforts. Human factors researchers, like experts in other disciplines, can be overenthusiastic about their own inventions when empirical data are absent. Development of a successful aid usually takes a long time. The work shows the potential payoff of interdisciplinary effort directed at a very hard problem. The work reflects elements of all three recommended research strategies. The experimental evaluations showed the status of the aid and suggested possible improvements. And the testing still goes on: Landauer et al. (1993) have carried out several other experimental evaluations with other materials and have made SuperBook available to over 40 research labs. Aid for Detecting Drunk Drivers Harris (1980) designed and carried out a study to develop an aid to help police officers spot drunk drivers. In the end, Harris provided officers in patrol cars with a low-tech, effective aid—a list of driver behaviors that best predict which motorists are driving while intoxicated (DWI). More important here is the process that led up to his designing, implementing, and evaluating this aid. His study is valuable for its methodology, completeness, impact, clear write-up, and (inexpensive) aid developed to help police select which drivers to stop. In general, six percent of drivers at night have a blood-alcohol level of 0.10 or greater (most states define this as DWI), and another 9 percent have

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research a blood-alcohol level between 0.05 and 0.10 (some states define this as "driving while impaired"). First, Harris studied 1,288 DWI arrests throughout the United States, analyzing the reported driver behaviors associated with each of these arrests. These driver behaviors could all be observed by police officers making the arrests (e.g., swerving, straddling) and could therefore serve as visual cues to aid in deciding whether or not to stop a potentially drunk driver. Harris also rode with officers in two states as they stopped drivers, and he noted the driving behaviors of those motorists and the conditions under which they were stopped. Nearly all of the stopped drivers (93 percent) were given a blood-alcohol test when they were stopped: 38 percent had a blood-alcohol level greater than 0.10 (DWI), and another 23 percent had a blood-alcohol level between 0.05 and 0.10. (This 61 percent is already 4 times higher than the 15 percent that could be expected by chance.) From these analyses he produced a drunk driver detection guide, a small card listing several visual cues (e.g., following too closely) and the percentage of drivers with this behavior who have a blood-alcohol level greater than .01 (e.g., 60%). The card was for police to carry with them or to attach wherever convenient. Harris then went on to study the value of this aid by studying 10 law enforcement agencies across the United States. before and after they used his guide. He found that DWI arrest rates increased significantly overall (from 66 to 74 percent; p < .01) for these 10 agencies. (No mention is made of false positives—arrests that did not lead to convictions.) Harris (1980:731) also reported: Experienced police officers … were generally skeptical that … the guide would enhance their own DWI detection ability. However, most officers considered the guide to be a valuable aid for increasing patrol awareness of useful detection cues, training inexperienced patrol officers, preparing DWI arrest reports, and supporting court testimony. Other Approaches The Landauer et al. (1993) studies provide an excellent example of extending the cognitive findings of one's own group to the design of a real electronic aid. Human factors researchers could take the initiative to extend other existing behavioral and cognitive knowledge to applied domains (a task much harder than many experimental and cognitive psychologists generally suppose). For example, for some years many secretaries and professionals have used commercially available computer-executable, rule-based styles to lay out and format their documents, for example, in prespecified fonts, margins, number of columns, line spacing, headings, and graphics. Now, if there exists behavioral knowledge of how to design textbooks and related reading material to enhance learning, human factors people, working

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research with computer scientists, could develop a similar rule-based style to lay out documents according to this knowledge. Another example would be to extend the results on human decision making, described above, to real-world aids. The Landauer et al. (1993) studies provide an excellent example of extending one's own cognitive findings to the design of a real electronic aid. Another approach might take existing technology or aids and combine them in behaviorally innovative ways. Perhaps the Xerox Star system (Smith et al., 1982), the highly influential graphic user interface developed at Xerox Parc in the 1970s, is an example of this. The developers took existing hardware (bit-mapped displays, mouse), existing software techniques, behavioral knowledge, and cognitive engineering models and integrated them into something much greater than the sum of the parts. Many observers agree that this work influenced subsequent graphic user interfaces made by other major corporations. ACKNOWLEDGMENTS I thank Liz Gould, Bob Helmreich, Bob Kraut, Tom Landauer, Ray Nickerson, Lynn Streeter, Jacob Ukelson, and Frank Yates, who provided helpful written reviews and suggestions to improve this chapter, and several other committee colleagues for additional suggestions. REFERENCES Attewell, P.A. 1994 Information technology and the productivity paradox. Pp. 13-53 in D.H. Harris, ed., Organizational Linkages: Understanding the Productivity Paradox. Panel on Organizational Linkages, Committee on Human Factors, National Research Council. Washington, D.C.: National Academy Press. Bentley, R., J.A. Hughes, D. Randall, T. Rodden, P. Sawyer, D. Shapiro, and I. Sommervile 1992 Ethnographically-informed systems design for air traffic control. Pp. 123-129 in J. Turner and R. Kraut, eds., Proceedings of the Conference on Computer-Supported Cooperative Work. New York: ACM Press. Berlin, L.M., and R. Jeffries 1992 Consultants and apprentices: observations about learning and collaborative problem solving. Pp. 130-137 in J. Turner and R. Kraut, eds., Proceedings of the Conference on Computer-Supported Cooperative Work. New York: ACM Press. Boehm, B.W. 1988 Improving software productivity. IEEE Computer September:43-56. Card, S.K., W.K. English, and B. Burr 1979 The Keystroke-Level Model for User Performance Time With Interactive Systems. Report SSL-79-1. Palo Alto, Calif.: Xerox Palo Alto Research Center. Card, S.K., J.M. Robert, and L.N. Keenan 1984 On-line composition of text. Pp. 231-236 in Proceedings of Interact'84, First IFIP Conference on Human-Computer Interaction, Vol. 1. London, England: Elsevier North Holland.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research Cool, C., R.S. Fish, R.E. Kraut, and C.M. Lowery 1992 Iterative design of video communication systems. Pp. 25-32 in J. Turner and R. Kraut, eds., Proceedings of the Conference on Computer-Supported Cooperative Work. New York: ACM Press. Curtis, B., H. Krasner, and N. Iscoe 1988 A field study of the software design process for large systems. Communications of the ACM 31:1268-1287. Dawes, R.M., D. Faust, and P.E. Meehl 1989 Clinical versus actuarial judgment. Science 243:1668-1673. Druckman, D., and R.A. Bjork, eds. 1991 In the Mind's Eye: Enhancing Human Performance. Committee on Techniques for the Enhancement of Human Performance, National Research Council. Washington, D.C.: National Academy Press. Druckman, D., and J.A. Swets, eds. 1988 Enhancing Human Performance: Issues, Theories, and Techniques . Committee on Techniques for the Enhancement of Human Performance, National Research Council. Washington, D.C.: National Academy Press. Drury, C.G., B. Paramore, H.P. Van Cott, S.M. Grey, and E.P. Corlett 1987 Task analysis. Pp. 370-401 in G. Salvendy, ed., Handbook of Human Factors. New York: Wiley. Ferber, M.S., B. O'Farrell, and L. Allen, eds. 1991 Work and Family: Policies for a Changing Work Force. Panel on Employer Policies and Working Families, Committee on Women's Employment and Related Social Issues, National Research Council. Washington, D.C.: National Academy Press. Gould, J.D. 1978a An experimental study of writing, dictating, and speaking. Pp. 299-319 in J. Requin, ed., Attention and Performance VII. Hillsdale, N.J.: Lawrence Erlbaum Associates. 1978b How experts dictate. Journal of Experimental Psychology: Human Perception and Performance 4(4):648-661. 1980 Experiments on composing letters: some facts, some myths, and some observations. Pp. 97-118 in L. Gregg and E.R. Steinberg, eds., Cognitive Processes in Writing. Hillsdale, N.J.: Erlbaum and Associates. 1982 Writing and speaking letters and messages. International Journal of Man-Machine Studies 16:147-171. 1988 How to design usable systems. Pp. 757-789 in M. Helander, ed., Handbook of Human-Computer Interaction . Amsterdam, Netherlands: North-Holland, Elsevier Science Publishers. Gould, J.D., and S.J. Boies 1978 How authors think about their writing, dictating, and speaking. Human Factors 20:495-505. 1983 Human factors challenges in creating a principal support office system: the speech filing system approach. ACM Transactions of Office Information Systems 4(1):273-298. 1984 Speech filing: an office system for principals. IBM Systems Journal 23:65-81. Gould, J.D., S.J. Boies, S. Levy, J.T. Richards, and J.W. Schoonard 1987 The 1984 Olympic Message System: a test for behavioral principles of system design. Communications of the ACM 30(9):758-769. Harris, D.H. 1980 Visual detection of driving while intoxicated. Human Factors 22(6):725-732.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research Helander, M., ed. 1988 Handbook of Human-Computer Interaction. Amsterdam, Netherlands: North-Holland, Elsevier Science Publishers. Helmreich, R.L., J.A. Wilhelm, S.E. Gregorich, and T.R. Chidester 1990 Preliminary results from the evaluation of Cockpit Resource Management training: Performance ratings of flightcrews. Aviation, Space and Environmental Medicine 61(6):576-579. Jones, T.C. 1986 Programming Productivity. New York: McGraw-Hill. Kanki, B.G., and M.T. Palmer 1993 Communication and crew resource management. In E.L. Wiener, B.G. Kanki, and R.L. Helmreich, eds., Cockpit Resource Management. San Diego, Calif.: Academic Press. Karat, C. 1992 Cost-justifying human factors support on software development projects. Human Factors Society Bulletin 35(11):1-4. Klemmer, E.T., and F.W. Snyder 1972 Measurement of time spent communicating. Journal of Communication 22:148-158. Kraut, R., S. Dumais, and S. Koch 1989 Computerization, productivity, and quality of work-life. Communications of the ACM 32(February):220-237. Kraut, R.E., and L.A. Streeter 1995 Coordination in software development. Communications of the ACM 38(3):69-81. Landauer, T., D. Egan, J. Remde, M. Lesk, C. Lochbaum, and D. Ketchum 1993 Enhancing the usability of text through computer delivery and formative evaluation: the SuperBook project. In C.M. McKnight, A. Dillon, and J. Richardson, eds., Hypertext: A Psychological Perspective . New York: Horwood. Lenorovitz, D.R., and M.D. Phillips 1987 Human factors requirements engineering for air traffic control systems. Pp. 1724-1770 in G. Salvendy, ed., Handbook of Human Factors . New York: Wiley. Malone, T.W. 1983 How do people organize their desks: implications for designing office automation systems. ACM Transactions on Office Automation Systems 1:99-112. McCauley, C. 1991 Selection of National Science Foundation Graduate Fellows. A case study of psychologists failing to apply what they know about decision making. American Psychologist December:1287-1291. Mintzberg, H. 1973 The Nature of Managerial Work. New York: Harper and Row. Murphy, A.H., and B.G. Brown 1984 A comparative evaluation of objective and subjective weather forecasts in the United States. Journal of Forecasting 3:369-393. Norman, D. 1988 The Psychology of Everyday Things. New York: Basic Books. Olson, J.S., G.M. Olson, M. Storrosten, and M. Carter 1992 How a group-editor changes the character of a design meeting as well as its outcome. Pp. 91-98 in J. Turner and R. Kraut, eds., Proceedings of the Conference on Computer-Supported Cooperative Work . New York: ACM Press.

OCR for page 291
Emerging Needs and Opportunities for Human Factors Research Panko, R.R. 1992 Managerial communication patterns. Journal of Organizational Computing 2(1):95-122. Pool, R. 1992 Computing in science. Science 256(April 3):44-64. Predmore, S.C. 1991 Microcoding of communications in accident analyses: crew coordination in United 811 and United 232. Pp. 350-355 in Proceedings of the Sixth International Symposium on Aviation Psychology. Columbus: Ohio State University. Roberts, T.L. 1979 Evaluation of Computer Text Editors. Unpublished PhD dissertation. Computer Science Department, Stanford University. Smith, D.C., C. Irby, R. Kimball, B. Verplank, and E. Harslem 1982 Designing the star user interface. Byte 7(4):242-282. Turner, J., and R. Kraut, eds. 1992 Proceedings of the Conference on Computer-Supported Cooperative Work. New York: ACM Press. Wiener, E.L., T.R. Chidester, B.G. Kanki, E.A. Palmer, R.E. Curry, and S.E. Gregorich 1991 The Impact of Cockpit Automation on Crew Coordination and Communication. I. Overview, LOFT Evaluations, Error Severity, and Questionnaire Data. NASA Contractor Report No. 177587. Moffett Field, Calif.: NASA-Ames Research Center. Woods, D., J.F. O'Brien, and L.F. Hanes 1987 Human factors challenges in process control: the case of nuclear power plants. Pp. 1724-1770 in G. Salvendi, ed., Handbook of Human Factors. New York: Wiley.