On Key Processes

CROSS-DISCIPLINARY, SOCIAL-CONTEXT RESEARCH

John Leslie King
University of California, Irvine

A key challenge in developing an every-citizen interface to the national information infrastructure is in recognizing the ongoing evolution in our concepts of what the "interface" encompasses. The development of concern over interface issues in the past 30 years reflects the complexities of these issues and provides some direction for improving these interfaces.




BACKGROUND

The term interface in the computing field has been appropriated by a relatively narrow community of interest, namely those interested in human-machine interactions at the ergonomic and perceptual level. Most of this study, which goes under the name of human-computer interaction (HCI or CHI) is strictly limited to studies of individual human actors interacting with specific packages of hardware and software. This focus of work has been very successful, producing among other things the innovations of "pointing" aids such as the mouse, trackball, touchscreen, and digital pad, as well as the graphical screen interface ubiquitous in all modern operating systems. These advancements have their intellectual roots in fairly circumscribed zones of disciplinary concern, namely cognitive psychology and human factors engineering. In essence the focus of this line of work is on the human working with a computer-based system.

In the past several years the HCI focus has broadened somewhat to include small groups of individuals each working with a computer-based system but for the purpose of working collaboratively with other members of the group. This is a significantly different conceptual focus, with individuals working through the computer to interact with other individuals. Prominent developments in this domain have been technologies for computer-mediated communication, "groupware," and computer-supported cooperative work. The intellectual roots of this work go beyond cognitive psychology into other realms of the social sciences, especially social psychology, but also into anthropology, organizational psychology and sociology, and economics. The applications of these technologies have caught the attention of scholars interested in fundamental questions of human discourse, social network construction and maintenance, identity and personality formation and expression, and the social construction of meaning and reality. These rapidly growing areas of interest have been stimulated by the stunning speed with which major components of the national information infrastructure such as the Internet and the World Wide Web have invaded social life in all dimensions.

These developments illustrate the evolving capacity of computer-based systems to affect basic human activities and reflect the fact that the concept of "interface" between humans and information technology is an elastic concept that expands to deal with the new opportunities and problems presented by technological change. Three observations can be made from this evolving concern with interface.

1. The parochial concerns of any particular group that engages interface issues at any given moment tend to appropriate and dominate the evolving meaning of interface-related research. The routine disciplinary politics of research institutions affect researchers in the interface field. Interface research was for many years (and to a considerable degree still is) politically marginalized within the field of academic computer science. Even within the interface field, some researchers whose work is fundamentally grounded in psychology feel themselves to be marginalized by those whose work is based on traditions of engineering in which psychology plays little part. The lesson here is that the dominant definitions of what constitutes the "real" issues in interface research and what constitutes the "right" approaches to doing such research are very misleading. It is necessary to look beyond these politically constructed definitions of what ought to be done and focus on the broader challenges of what emerging applications will require.

2. The trend in conceptual evolution of interface concerns is generally "upward" in the layers of social focus, from the individual to small group, to organizational, sectoral, institutional, and cultural. To the extent that being human involves essential attributes of group identity, organization of production and consumption, formation and sustainment of social order and culture, and so on, these too require attention in interface research. This notion is captured well in recent use of the term usability to describe research aimed at developing information technologies that actually accomplish what those who use them desire. This focus has clearly emerged in the computer-supported cooperative work (CSCW) research community and has appeared as well in discussions of organizational usability and even institutional usability of information technologies. It is certain that a concern about every-citizen interfaces to the national information infrastructure must embody such perspectives. While it is true that at some level all interface issues can be traced to rudimentary human-computer interface concerns as represented by the parochial HCI community, these broader issues of usability involve concerns that have nothing at all to do with the narrow HCI focus and must be addressed by research methods that traditional HCI researchers would never consider.

3. Although the concern with interface issues is usually tied to the evolving HCI, CSCW, and other perspectives, important aspects of research into group, organizational, and institutional usability have been under way for many years. Although largely ignored by the computer science research community, the vast range of economically vital computing applications in organizational information processing have drawn much attention from researchers in management information systems, library and information science, medical informatics, and other fields. Transaction-processing systems, which remain among the largest and most complex computerized information systems, were made possible only by careful study and learning-by-doing design to meet interface needs at the individual, work group, organizational, and institutional levels. To pick just one case in point, designers of the airline reservations system, which literally revolutionized air travel, had to overcome numerous complicated problems at all social levels, including being modified to comply with court-ordered remedies against unfair competitive practices. Similar stories can be told regarding credit data-reporting systems, financial accounting and reporting systems, personnel management systems, computer-integrated manufacturing systems, and so on. The lesson here is that a great deal of useful information on the development of effective interfaces at the higher social levels is available in the applications-oriented research communities.




RESEARCH NEEDS

The point of the discussion above is not to argue against further investment in the well-established traditions of research in computer science, HCI, and so on. Such investments have yielded great payoffs and will continue to do so in the future. Instead, the point is to place in the foreground the need for focused research on the "higher-level" issues of interface development at the group organization, sectoral, institutional, social, and cultural levels. Moreover, these are not merely desirable venues for research investment, but rather are as essential to the goal of an every-citizen interface as well-established domains of federally supported research. Unfortunately, such research has been comparatively under-funded. There are numerous advocates of more traditional research needs to articulate requirements for such research. I will focus on the needs of the higher-level challenges.

The primary goal of research into the higher-level challenges of interface design is to reduce the cost and increase the speed of effective design. A great many extraordinary information technologies have been developed that demonstrate the virtues of good interfaces and that are highly usable in routine production. But the cost of developing them has been quite high because most of the systems have been built on the ruins of expensive failed efforts. It has been estimated that as many as half of large information systems projects fail to meet their objectives, and a significant fraction of those fail altogether. Many examples can be drawn from public-sector projects such as the disastrous World Wide Military Command and Control System (more than $5 billion), the Federal Aviation Administration's advanced technology program to replace the aging air traffic control system (more than $2 billion), and California's write-off of its ambitious overhaul of motor vehicle information systems (a mere $55 million). Similar failures abound in the private sector, but they are more easily hidden from view. The American Airlines effort to replicate its marvelous success in airline reservations systems in the French national railways and in its Encompass freight management system come to mind as just two examples. Research into higher-level challenges is aimed at learning what works and what does not and putting that knowledge to work.

The following constitute important areas of needed research investment at the higher level:

• Synthetic studies that pull together the extensive social learning already accumulated through important development projects. Most large-scale system development efforts occur in operational settings, not research laboratories. Research on such systems must be done in vivo, in the living systems. There are significant problems with doing such research that need to be acknowledged. Among others, such research usually presents researchers with difficulties in gaining access to the right study sites, reluctance of organizations to reveal their failures due to fear of embarrassment, reluctance to reveal details of successes for competitive reasons, and the high cost of travel and time on site. But these are really no more troublesome than the challenges faced by numerous other research communities that must go to the field for their data collection. The synthetic research suggested here must be theoretically driven, but ultimately it is empirical in nature and akin to research traditions in engineering and management. The questions are: What works? What doesn't work? And why?

• Analytical studies of the likely evolutionary pathways of complex sociotechnical systems over time. It is increasingly recognized that information technologies do not merely allow efficiency and effectiveness improvements in well-established activities. Much more importantly, they enable fundamental changes in the nature of the activities themselves. The whole "reengineering" movement, which was articulated by people from organizational information systems backgrounds, is predicated on the argument that fundamental changes in what is to be done are required, displacing the traditional focus on how things are to be done. Much of the learning about what will "really" happen is inevitably going to be empirical—we will watch and see. But advances in conceptual tools such as game theory have made possible much more sophisticated modeling of possible interactions among actors under different assumptions, including enablement from new information technologies. Such studies seldom provide real predictive power in the sense that they can tell designers or decision makers exactly what to do. But they have been applied with great success in narrowing the search space around the likely outcomes under different assumptions—a contribution that can greatly improve the efficiency in design of complex systems that must be built through learning by doing processes such as prototyping. Significant increases in investment in such research are needed.

• Collaborative research and development projects that allow different development strategies to be tested in real settings. There are numerous recommended strategies for improving the design of complex sociotechnical systems. Many of these have evolved from the traditions of system design and software engineering and range from structured analysis and design techniques to participatory design. Unfortunately, our understanding of the efficacy of these approaches amounts to little more than folklore. There have been few systematic studies to demonstrate the utility of these approaches in real development situations and the contingencies under which the different approaches offer advantages. Moreover,this lack of systematic study makes it difficult to identify holes in existing knowledge and theory that require further research attention.

These kinds of studies require multidisciplinary research approaches, involving specialists from information and computer sciences, management, and the social sciences. In many cases, particularly where applications are in expert domains (e.g., medicine), subject matter specialists will be needed as well.




AUDIO ACCESS TO THE NATIONAL INFORMATIONINFRASTRUCTURE

John C. Thomas
NYNEX Science and Technology





GOALS

There are trade-offs among the goals given below and the costs. While each is a desirable direction, any of these goals, if pushed to an extreme, could render other goals impossible.

• Universality. Every citizen should have some way of gaining access to information that has been made public. In some cases this may require special devices, language help, training, or economic assistance.

• Privacy. Individuals should have the ability to create, store, and modify information and restrict the access others have to that information. Information created about individuals by others should also be restricted in ways that the individual has some power over. At a minimum, individuals should know who has what information about them and should know when new information is being collected.

• Security. Information should be safe from unauthorized destruction, alteration, or copying.

• Usability. Information should be presented in a form that is maximally useful. This depends on the person, the task, the context, and their access to the system.

• Empowerment. The individual should be free to determine how, when, and where he or she access information. For example, some people may prefer, for certain tasks, a very "active" system in which agents make frequent suggestions. Others may prefer a more passive system.

• Responsibility. Economic or other incentives should be in place so that limited resources (bandwidth, storage capacity, computer power, creative human power that produces new information) are not simply "taken" for free by whoever gets there first. There should also be some incentive for those who create information to keep on creating.

• Translatability. Information entered in one medium should be capable of being translated into another medium. Not only should documents be easily translatable into another machine's format, but faxes should be translatable into ASCII, spoken words into ASCII, and, ultimately, even pictures and video should be somewhat describable as text.

• Ubiquity. People should have access from anywhere to information (perhaps at a premium). They should not have to physically move very far to gain this access.




RESEARCH APPROACH

In 13 years at IBM I saw many excellent results (in human-computer interaction and effective software development process) of research in our labs; other labs and universities have a minuscule impact on what was actually done at IBM. In starting the artificial intelligence laboratory at NYNEX, I led each group to have a long-term vision of where to push the technology but to choose a portfolio of short- and medium-term projects that pushed toward that vision but also provided a real benefit for NYNEX. This was not motivated primarily by a desire to stay funded—though that was a consideration too—but because I believe it is far too easy to "partial out" the really difficult issues of artificial intelligence (AI) if one works on "toy systems." I do not believe there are "frictionless planes" when it comes to human psychology or building complex systems. Only through applied work is real progress made.

The ultimate goal of speech synthesis, for example, might be to read text as a good human actor might. However, a little reflection will show this (and all other similar problems!) to be coextensive with the "general AI" problem. While this is a worthy vision, a mundane step we took in this direction was to work on a better synthesizer for names and addresses, focusing on improving the prosody. Here the approach of the main investigators (Kim Silverman and Ashok Kalymanswamy) was not to "prove" that a particular approach to prosody was the "right" one but to use everything that worked and to do their own original research when they hit unsolved problems.

None of this philosophy should be taken as meaning that there is no place for theory or no place for university research. We have had a number of good collaborations with universities, including MIT and the University of Colorado, where we work on real problems for a while and then spend time theorizing. But the theorizing is based on experiences with real problems, and the theory is then applied to the next real problem.

I also believe that working on complex, real-world problems requires the cooperation, and perhaps the friendly competition, of numerous research groups. The speech community has shown remarkable progress working together through the Defense Advanced Research Projects Agency in collecting and sharing data, trying various approaches, and publishing results. The dawning commercial success of this technology is a further driver toward progress. I believe, therefore, that the way to develop the research agenda for issues about providing an every-citizen interface to the national information infrastructure is to start doing it. As we run into problems, we should use those problems to define the research agenda.




RESEARCH ISSUES

I propose that we develop an audio access to the Internet. Such a system would allow anyone access to the Internet over an ordinary POTS telephone line by using spoken commands. Quite obviously, this would not provide the identical experience that sitting in front of a keyboard (for an experienced typist) and a huge full-color monitor might. Nonetheless, such a system will have several major practical benefits and would serve as a focal point for pushing some important technologies.

First, the penetration of personal computers into the lowest socioeconomic status quartile is low and staying low. This would allow people without the financial resources immediate access to the Internet. It would also allow people who are reluctant to buy a computer because they don't see the value of Internet access (or other computer applications) to "try out" Internet access without a significant up-front investment. Individuals could use such access to listen to speeches of political candidates on particular issues, find out about their benefits and bills, find out about community events and safety messages, find transportation schedules, get sports results, and so on. Some of this information currently does exist in various audio forms, but typically it is not updated very frequently, and users must dial a different number for each type of information.

Second, there are families and work groups where most of the group has Internet access but a few do not. An audio access to the Internet would allow such a group or family to communicate much more effectively.

Third, even people who do have computer access to the Internet find themselves in situations (e.g., in their cars, at payphones, at hotels without modems) where they do not have computer access but could use a phone to find important financial and trip information, listen to their e-mail, etc.

Fourth, audio input/output is already becoming an important enhancing medium for the Internet. Welsh lessons that include audio are available from Brown University. One can listen to music, people's voices, and sounds. Ideally, one might well navigate more easily through voice commands. In addition, speaker verification could add another level of security to Internet transactions.

Fifth, there are people with special needs (blind or paralyzed users) for whom audio access to the Internet would be crucial. In addition to these special needs, there is a huge population of people in the United States who can speak and listen to English but cannot read or write it. In a stroke, such an interface would help enfranchise and educate them.

In addition to providing practical benefits, a workable audio access to the Internet will force us to improve several important technologies.

• Speech recognition. While some access to the Internet could begin immediately, access could be improved via larger, speaker-independent vocabulary. Another improvement could be made if the system "adapted" on-line to the dialect and vocal tract configuration of individual speakers. Performance in some conditions will also be enhanced by better noise cancellation techniques.

• Speech synthesis. While current levels of intelligibility would be useful, certainly further enhancements can and should be made in the areas of aesthetic appeal, proper name and special symbol pronunciation, and prosody. In some cases, the Internet may provide additional clues for the synthesizer. For instance, in e-mail, the name and subject fields give potential clues about how the contents should be pronounced. Hyper-text markup language (HTML) tags in Web sites may give additional structural information about pauses and emphasis.

• Picture and video understanding. To provide some (not complete) audio information about pictures, graphics, and videos available on the Internet, an automatic scene describer would provide a cheaper solution than having human beings try to keep up with the exploding information by describing each scene. By having HTML tags and verbal materials to provide some structure and context, the task for machine scene understanding could be made more tractable, but there are still unsolved research issues here. The solutions would have other applications as well, including digital movie making and editing and security.

• Natural language processing. Much of the information on the Internet is in the form of text. Being able to do a better job of indexing, summarizing, and locating text would drive better natural language processing. Again, the HTML tags of Web sites provide a potential additional source for natural language understanding systems.

• User interface design. How should a dialogue over the phone be structured? Under what conditions should it be all speech and when should DTMF also be used? When is explicit confirmation needed? How can speech/audio be used as an adjunct to screen-based systems?

• System integration. Perhaps a mobile phone user would like to browse for certain kinds of information and bookmark Web pages for later perusal on a screen. How can the various networks be internetworked?

• Speaker verification. Perhaps the user could be presented with a phrase to repeat in his or her own voice. This could be used to help authenticate transactions. Clearly, better verification techniques would help make the system more secure and user friendly.

• Media translation. Speech recognition and synthesis are special cases, but it would also be nice to be able to translate a fax document into an e-mail document that could then be read over the phone. In general, information should be capable of being input in any medium and output in any other medium. While the (typed) fax to e-mail issue is nearly solved, handwriting still provides a challenge. We still do not see systems adapting to handwriting as well as humans do.

• Adaptive recognition. The human being seems to be very good at "speaker-independent" speech recognition. Yet a selection made by "pasting together" words from a random speaker seems very difficult for a human to understand compared to one from any single speaker. Today's speech recognition systems don't seem to care. Apparently, humans are doing something in the way of rapid speaker adaptation that we do not understand well enough to incorporate into a machine. Similarly, we are quite good at "adapting" to a particular individual's handwriting. Again, we don't know how to program a computer to do this very well. Similarly, if we view a graph, photo, or movie on a particular device in a particular lighting context, we make a fairly quick adaptation to the style and other aspects of context. And, again, we don't know how to do this very well in a machine. I believe a key to significant improvement in a number of the technologies listed here is a better understanding of how humans do adaptive recognition. Rather than studying it in a toy domain, however, I believe we should observe and test how people do this in a real context doing a real task.

• Intelligent searching. Today's search engines on the Internet are not very precise. They typically return very many false-positives. Audio input/output with natural language processing gives the possibility of more selective searches and also provides strong motivation since audio scanning is more onerous than visual scanning. One potential source of information is to use the user's current task and past history to help focus a search. Such an approach forces us to examine privacy and security issues.

All of these technologies could be explored in their own right, but I think that exploring them in the context of trying to provide a real-world system will produce the best research results as well as a practical benefit.