PART II
TRENDS IN HUMAN-COMPUTR INTERACTION
RESEARCH AND DEVELOPMENT
H. Rex Hartson
Virginia Polytechnic Institute
INTRODUCTION
Human-computer interaction (HCI) is a field of research and
development, methodology, theory, and practice with the objective of
designing, constructing, and evaluating computer-based interactive
systemsincluding hardware, software, input/output devices, displays,
training, and documentationso that people can use them efficiently,
effectively, safely, and with satisfaction. HCI is cross-disciplinary in its conduct
and multidisciplinary in its roots, drawing onsynthesizing and
adapting fromseveral other fields, including human factors (e.g., the roots
for task analysis and designing for human error in HCI); ergonomics
(e.g., the roots for design of devices, workstations, and work
environments); cognitive psychology (e.g., the roots for user modeling); behavioral
psychology and psychometrics (e.g., the roots of user performance
metrics); systems engineering (e.g., the roots for much predesign analysis);
and computer science (e.g., the roots for graphical interfaces, software
tools, and issues of software architecture).
Importance of Usability
The entire field of HCI shares the single goal of achieving high usability for users of computer-based systems. Rather than being fuzzy and vague as it is sometimes perceived, usability is tangible and can be quantified. Usability can be broadly defined as "ease of use," including such measurable attributes as learnability, speed of user task performance, user error rates, and subjective user satisfaction (Shneiderman, 1992; Hix and Hartson, 1993a). However, an easy-to-use system that does not support its users' needs, in terms of functionality, is of little value. Thus, usability has evolved toward the concept of "usability in the large"that is, ease of use plus usefulness.
Despite many research advances in interactive computer systems, usability barriers still obstruct access to, and blunt effectiveness of, an every-citizen interface for the national information infrastructuredisenfranchising and disenchanting users across society. As a result, the United States fails to accrue the potentially enormous returns of our collective investment in computing technology. These barriers impede human productivity and have a profound impact on computer users in business, government, industry, education, and indeed the whole nation.
In the not-too-distant past, computer usage was esoteric, conducted mostly by a core of technically oriented users who were not only willing to accept the challenge of overcoming poor usability but also sometimes welcomed it as a barrier to protect their craft from uninitiated "outsiders." Poor usability was good for the field's mystique, not to mention users' job security. Now, unprecedented numbers of Americans use computers, and user interface is often the first thing people ask about when discussing software. To most users the interface is the system. For the "every citizen" of today, communication with the system has become at least as important as computation by the system.
The goals of most organizations include increased employee and
organization productivity, decreased employee training costs,
decreased employee work errors, and increased employee satisfaction. These
are also exactly the benefits of achieving high usability in user interfaces.
Too often, especially in government and large businesses, training is used as
a costly substitute for usability, and almost as often it fails to meet its goals.
Attention to usability by developers no longer requires justification
in most quarters: "Usability has become a competitive necessity for
the commercial success of software" (Butler, 1996).
Product and Process
Achieving good usability requires attention to both product and
process. The product, in this case, is the content of the user
interaction design and its embodiment in software. An effective
process for developing interaction design is also important, and a poor understanding
of the process is often responsible for a product's lack of usability.
While state-of-the-art user interaction development processes are based on
formative usability evaluation in an iterative cycle, much of the state of
the practice is fundamentally flawed in that remarkably little formal
usability evaluation is performed on most interactive systems. This is
generally changing now in many industrial settings. However,
ensuring usability remains difficult when evaluation, because of real or
perceived costs, is not standard practice in interactive software
development projects.
User Interaction Versus User Interface Software
Developers attempting to incorporate usability methods into
their development environments often refer to their efforts in terms of
"evaluating software" or "evaluating user interface software." There are
many reasons for evaluating software, but usability is not one of them.
Usability is seated within the design of the user interaction
component of an interactive system, not in the user interface
software component, as shown simplistically below:
Development of the user interface
| Development of user interaction component | Development of user interface software component |
Development of the interaction component, toward which most HCI effort is directed, is substantially different from development of the user interface software. The view of the user interaction component is the user's perspective of user interaction: how it works; how tasks are performed using it; and its look and feel and behavior in response to what a user sees, hears, and does while interacting with the computer.
In contrast, the user interface software component is the programming code by which the interaction component is implemented. The user interaction component design should serve as requirements for the user interface software component. Design of the user interaction component must be given attention at least equal to that given the user interface software component during the development process, if usability in interactive systems is to be ensured.
The overview of HCI topics, issues, and activities that follows
is loosely divided into theory, interaction techniques, and
development methods. Reflecting its diverse roots, HCI is host to activities in
many topical areas, some of which are reviewed here. An attempt has
been made to capture a broad, inclusive cross section of a very dynamic
field, but this paper is not intended to be an exhaustive survey, and no
claims are made for completeness. Emphasis is given to topics of most
importance to the usability of an every-citizen interface.
THEORY
HCI theory has its avid proponents. If the proportion of literature devoted to theory is to be taken as an indication, theory plays a strong role in HCI, but in fact theory has not seen broad, direct application in the practice of HCI.
Much theory comes to HCI from cognitive psychology (Hammond
et. al., 1987; Barnard, 1993). Norman's (1986) theory of action
expresses, from a cognitive engineering perspective, human task
performancethe path from goals to intentions to actions (inputs to the computer) back
to perception and interpretation of feedback to evaluation of whether
the intentions and goals were approached or met. The study of learning
in HCI (Carroll, 1984; Draper and Barton, 1993) and Fitts Law (relating
cursor travel time to distance and size of target) (MacKenzie, 1992) also
have their roots in cognitive theory.
Task Analysis
To design a user interface (or any system) to meet the needs of
its users, developers must understand what tasks users will use a system
for and how those tasks will be performed (Diaper, 1989). Because tasks at
all but the highest levels of abstraction involve manipulation of user
interface objects (e.g., icons, menus, buttons, dialogue ses), tasks and
objects must be considered together in design (Carroll et al., 1991). A
complete description of tasks in the context of their objects is a rather
complete representation of an interaction design. The process of describing
tasks (how users do things) and their relationships (usually in a
hierarchical structure of tasks and subtasks) is called
task analysis and comes to HCI primarily from human factors (Meister, 1985). There are various
task analysis methods to address various purposes. In HCI the primary
uses are to drive design and to build predictive models of user task
performance. Because designing for usability means understanding user
tasks, task analysis is essential for good design; unfortunately, it is often
ignored or given only minimal attention.
Models of Human Information Processing
A significant legacy from cognitive psychology is the model of a human as a cognitive information processor (Card et al., 1983). The Command Language Grammar (Moran, 1981) and the keystroke model (Card and Moran, 1980), which attempt to explain the nature and structure of human-computer interaction, led directly to the Goals, Operators, Methods, and Selection (GOMS) model (Card et al., 1983). GOMS-related modelsquantitative models combining task analysis and the human user as an information processorare concerned with predicting various measures of user performance, most commonly task completion time based on physical and cognitive actions of users, with place holders and estimated times for highly complex cognitive actions and tasks. Direct derivatives of GOMS include NGOMSL (Kieras, 1988) and Cognitive Complexity Theory (CCT) (Kieras and Polson, 1985; Lewis et al., 1990), the latter of which is intended to represent the complexity of user interaction from the user's perspective. This technique represents an interface as the mapping between the user's job-task environment and the interaction device behavior.
GOMS-related techniques have been shown to be useful in discovering certain kinds of usability problems early in the life cycle, even before a prototype has been constructed. Some studies (e.g., Gray et al., 1990) have demonstrated a payoff in a few circumscribed applications where the savings of a small number of user actions (e.g., a few keystrokes or mouse movements) can improve user performance enough to have an economic impact, often because of the repetitiveness of a task.
Nonetheless, these models have not achieved widespread
application within the tight constraints of industrial schedules and budgets
because of the labor intensiveness of producing and maintaining these
relatively formal and structured task representations, the need for specialized
skills, and the difficulty in competing with the effectiveness of usability
evaluation using a prototype. Furthermore, these techniques generally do
not take into account individual differences in user classes and are often
limited to expert, error-free behaviors (not representative of "every
citizen" as a user). In any case, it is generally agreed that this kind of
analytical approach to usability evaluation cannot be considered a substitute
for empirical formative evaluationusability testing of a prototype with
users in a lab or field setting (see "User-Based Evaluation" below).
Human Work Activity
Another area feeding HCI theory and practice is "work activity theory" (Ehn, 1990; Bodker, 1991). Originating in Russia and Germany and now flourishing in Scandinavia (where it is, interestingly, related to the labor movement), this view of design based on work practices situated in a worker's own complete environment has been synthesized into several related mainstream HCI topics. For example, "participatory design" is a democratic process based on the argument that users should be involved in designs they will be using, in which all stakeholders, including and especially users, have equal inputs into interaction design. Muller (1991) and others have operationalized participatory design in an approach called PICTIVE, which supports rapid group prototype design using Post-ItTM notes, marking pens, paper, and other "low-technology" materials on a large table top.
This interest in design driven by work practices in context has led to the eclectic inclusion in some HCI practice of ethnography, an investigative field rooted in anthropology (LeCompte and Preissle, 1993), and other hermeneutic (concerned with ways to explain, translate, and interpret perceived reality) approaches as qualitative research tools for extracting design requirements. Contextual inquiry/design (Wixon et al., 1990) is an example of an adaptation of this kind of approach, where design and evaluation are conducted collaboratively by users and developers, while users perform normal work tasks in their natural work environment. Much of this collaboration is based on interviews that seek to make implicit work practices more explicit and to draw out structure, language, and culture affecting the work.
The task artifact framework of Carroll and Rosson (1992) and, to
some extent, scenario-based design follow an ethnographic focus on task
performance in a work context. Scenarios are concrete, narrative
descriptions of user and system activity for task performance (Carroll, 1995).
They describe particular interactions happening over time, being
deliberately informal, open ended, and fragmentary. Scenarios often focus
on interaction objects, or artifacts, and how they are manipulated by users
in the course of task performance.
Formal Methods
While not theory per se, formal methods have been the object of
some interest and attention in HCI (Harrison and Thimbleby, 1990). The
objectives of formal methodsprecise, well-defined notations and
mathematical modelsin HCI are similar to those in software engineering.
Formal design specifications can be reasoned about and analyzed for
various properties such as correctness and consistency. Formal specifications
also have the potential to be translated automatically into prototypes or
software implementation. Thus, in principle, formal methods can be used
to support both theory and practice; however, they have not yet had
an impact in real-world system development, and their potential is
difficult to predict.
DEVICES, INTERACTION TECHNIQUES, AND GRAPHICS
In contrast to theory, the influence of interaction devices and their associated interaction techniques represents a practical arena of real-world constraints as well as hardware design challenges. "An interaction technique is a way of using a physical input/output device to perform a generic task in a human-computer dialogue" (Foley et al., 1990). A very similar term, interaction style, has evolved to denote the behavior of a user and an interaction object (e.g., a push button or pulldown menu) within the context of task performance. In practice, the notion of an interaction technique includes the concept of interaction style plus full consideration of internal machine behavior and software aspects. In the context of an interaction technique, an interaction object (and its supporting software) is often referred to as a "widget." Libraries of widgetssoftware that supports programming of graphical user interfaces (GUIs)are an outgrowth of operating system device handler routines used to process user input and output in the now ancient and impoverished interaction style of line-oriented, character-cell, text-only, "glass teletype" terminal interaction. At first, graphics packages took interaction beyond text to direct manipulation of graphical objects, eventually leading to new concepts in displays and cursor tracking. Of course, invention of the mouse and advent of the Xerox Star and Lisa Macintosh by Apple accelerated the evolution of the now-familiar point-and-click interaction styles. It is not surprising that many of the computer scientists who developed early graphics packages also introduced GUI interaction techniques as part of their contribution to the HCI field (Foley and Wallace, 1974; Foley et al., 1990). To some extent, standardization of interactive graphical interaction techniques led to the widgets of today's GUI platforms and corresponding style guides intended for ensuring compliance to a style but sometimes mistakenly thought of as usability guides.
This growth of graphics and devices made possible one of the
major breakthroughs in interaction stylesdirect manipulation
(Shneiderman, 1983; Hutchins et al., 1986; Weller and Hartson, 1992)changing the
basic paradigm of interaction with computers. Unlike previous
command-line-oriented interaction in which users plan tasks in terms of hierarchies
of goals and subgoals, entering a command line for each, direct
manipulation allows opportunistic and incremental task planning. Users can
try something and see what happens, exploring many avenues for
interactive problem solving. This kind of opportunistic interaction is also
called display-based interaction (Payne, 1991).
DEVELOPMENT METHODS AND SOFTWARE ENGINEERING
The difference between user interaction and user interface
software, mentioned in the Introduction, results in a need for separate and
fundamentally different development processes for the two components of
a user interface.
Development Life Cycles
Studies deriving principles for user interaction development
(e.g., Gould et al., 1991) vary, but all agree that interaction development
must involve usability evaluation. Just adding some kind of "user testing"
to an existing software process is not enough, however. Usability
comes from a complete process, one that ensures usability and attests to when
it has been achieved (Hix and Hartson, 1993a). Most researchers and
practitioners also agree that an interaction development process must be
iterative, unlike the phase-oriented "waterfall" method, for example, for
software development. Although software can be correctness driven,
user interaction designbecause of infinite design possibilities and
unpredictable, dynamic, and psychological aspects of the human usermust
be self-correcting. Thus, interaction development is an essentially
iterative process of design and evaluation, one that must, in the end, be
integrated with other system and software life cycles. Within this cycle, the
interaction design is an iteratively evolving design specification for the
user interface software. The star life cycle (Hartson and Hix, 1989) for
interaction development explicitly acknowledges these differences from
software development, being unequivocally iterative, and allows the
process to start with essentially any development activity and proceed to
any other activity before the previous one is completed, with each
activity informing the others.
Development Activities
Design and Design Representation
Design is closely coupled to, and driven by, early systems analysis activities such as needs, task, and functional analyses. Good interaction design involves early and continual involvement of representative users and is guided by well-established design guidelines and principles built on the concept of user-centered design (Norman and Draper, 1986). Design guidelines address such issues as consistency, use of real-world metaphors, human memory limits, screen layout, and designing for user errors. Additionally, designers are expected to follow style guides (less oriented toward usability than toward compliance with some "standard" style) in their use of widgets.
Although some more recent guidelines enjoy the support of empirical studies, guidelines have typically been scattered throughout the literature, based mostly on experience and educated opinion. In a classic work, Smith and Mosier (1986) compiled guidelines for character-cell, textual interface design. Others (Mayhew, 1992; Shneiderman, 1992) have followed to help cover graphical interfaces.
Many practitioners believe it is enough to know and use interface design guidelines, possibly in addition to an interface style guide (e.g., for Windows). Experience, however, has shown that guidelines and style guides do not eliminate the need for usability evaluation. Experience has also demonstrated that, although guidelines are not difficult to learn as factual knowledge, their effective application in real design situations is a skill acquired only through long experience.
The creative act of design must also be accompanied by the
physical act of capturing and documenting that design. Although many
constructional techniques exist for representing software aspects of
interface objects, behavioral representation techniques are needed
for communicating, among developers, the interaction design from a
behavioral task and user perspective. The User Action Notation (UAN) is
one such technique (Hartson et al., 1990; Hartson and Gray, 1992). The UAN
is a user- and task-oriented notation that describes the behavior of a
user and an interface during their cooperative performance of a task.
The primary abstraction of the UAN is a user taska user action or group
of temporally related user actions performed to achieve a work goal. A
user interaction design is represented as a quasi-hierarchical structure of
asynchronous tasks. User actions, interface feedback, and internal state
information are represented at various levels of abstraction in the UAN.
In addition to design representation, design rationale (MacLean et al.,
1991) is captured to record and communicate the history and basis for
design decisions, to reason about designs, and to explore alternatives.
Prototyping
Rapid prototypes of interaction design are early and inexpensive
vehicles for evaluation that can be used to identify usability problems in
an interaction design before resources are committed to implementing
the design in software. Much interest has been focused on low-fidelity
prototypes (e.g., paper and pencil). Counter to intuition, low-fidelity
prototypes have allowed developers to discover as many usability problems
as found using interactive computer-based prototypes (Virzi et al., 1996).
Paper prototypes are most useful early in the life cycle because they
are more flexible in exploring variations of interaction behavior at a cost
of less fidelity in appearance. Later in the life cycle, changes made to
the behavior of a coded prototype are more expensive than changes made
in appearance. Almost all projects eventually move to computer-based
rapid prototypes for formal usability evaluation.
User-based Evaluation
Summative evaluation is used to make judgments about a finished product, to gauge the level of usability achieved and possibly compare one system with another. In contrast, formative evaluationthe heart of the star life cycleis used to detect and fix usability problems before the interaction design is coded in software (Hix and Hartson, 1993a,b; Nielsen, 1993), aiding in the improvement of an interaction design while a product is still being developed. For formative evaluation, unlike summative evaluation, statistical significance is not an issue. Formative evaluation relies on both quantitative and qualitative data. The quantitative data are used as a gauge for the processto be sure usability is improving with each design iteration and to know when to stop iterating. Borrowing an adage from software engineering (and probably other places before that), "if you can't measure it, you can't manage it." The instruments used to quantify usability include benchmark tasks and user questionnaires. Benchmark tasks, drawn from representative and mission-critical tasks, yield objective user performance data, such as time on task and error rates (Whiteside et al., 1988). Questionnaires yield subjective data such as user satisfaction (Chin et al., 1988). In analyzing quantitative data, results are compared against preestablished usability specifications (Whiteside et al., 1988)operationally defined and measurable goals used as criteria for success in interaction design.
Even more valuable than these quantitative data are the qualitative data gathered in usability evaluations. Identification of critical incidentsoccurrences in task performance that indicate a usability problemare essential in pinpointing design problems. Verbal protocol (capturing users' thinking aloud) helps designers understand what was going through a user's mind when a usability problem occurred, which may help in ascertaining its causes and in offering useful solutions.
These quantitative and qualitative data typically come from lab-based evaluations involving users as "subjects." While very effective, this process can be expensive. The need for faster, less costly usability methods has led to approaches, such as discount usability engineering (Nielsen, 1989), that trade off less-than-perfect and complete results for a lower cost. Inspection methods (Nielsen and Mack, 1994) use systematic examinations of design representations, prototypes, or software products. Cognitive walkthroughs (Lewis et al., 1990; Wharton et al., 1992) and claims analysis (Carroll and Rosson, 1992) are effective inspection methods, especially early in development, but can still be labor intensive and require special training, which is intimidating to developers in search of cost-effective methods. Heuristic evaluation (Nielsen and Molich, 1990; Nielsen, 1992), which involves reviewing compliance of an interaction design to a checklist of selected and generalized guidelines, is an even less expensive inspection method but is limited by the scope of guidelines used.
Inspection methods are effective at finding some kinds of
usability problems but do not reliably pinpoint all types of problems that can
be observed in lab-based testing. In fact, lab-based usability evaluation
remains the yardstick against which most new methods are compared
in formal studies. Most real-world development organizations continue
to be willing to pay the price for extensive lab-based usability
evaluation because of its effectiveness in helping them identify and understand
usability problems, their causes, and solutions.
Usability Engineering
Many HCI practices, such as the employment of usability specifications and various kinds of evaluation, have been gathered under the banner of usability engineering (Nielsen, 1993). This is a good appellation because it includes a concern for cost in the notion of discount usability methods (Nielsen, 1989), the practical goal of achieving specifications and not perfection, and techniques for managing the process. The latter is important because iterative processes are sometimes perceived by management as "going around in circles," which is not attractive to a manager with a limited budget and dwindling production schedule.
Usability specifications provide this essential management
control for the iterative process. The quantitative usability data are analyzed
in each iteration, and the results are compared with the usability
specifications, allowing management to decide if iteration can stop. If the
specifications are not met, data are assessed to weigh cost and severity or
importance of each usability problem, assigning a priority ranking for
designing and implementing solutions to those problems that, when fixed, will
give the largest improvement in usability for the least cost.
Development Tools
Almost any software package that provides support for the interface development process can be called an interface development tool, a generic term referring to anything from a complete interface development environment to a single library routine (Myers, 1989, 1993). New software tools for user interface development are appearing with increasing frequency.
Interface development tools can be divided into at least four types (Hix and Hartson, 1993a). Toolkits are libraries of callable routines for low-level interface features and are often integrated with window managers (e.g., X,Windows) Interface style support tools are interactive systems that enforce a particular interface style and/or standards (e.g., OSF Motif, Common User Access). User interface management systems (UIMs) are development environments that can include both prototyping and run-time support, with the goal of allowing developers to produce an interface implementation without using traditional programming languages. Of these groups, the UIMSs perhaps are the most interesting, have the most potential, and suffer the most difficult technical problems (Myers, 1995).
These first three categories of development tools primarily address user interface software. A fourth category, interaction development tools, provides interactive support for user interaction development. Of all the interaction development activities, the one most commonly supported by tools in this group is formative evaluation (Macleod and Bevan, 1993; Hix and Hartson, 1994).
Although tools now exist on many programming platforms to lay
out objects of a user interface quickly and easily, usability problems are
not necessarily addressed by adding this kind of technology to the
process; many interface development tools are potentially a faster way to
produce poor interfaces.
COST JUSTIFICATION
Economic justification for usability effort in interactive system development is now beginning to be established (Bias and Mayhew, 1994). Broad acceptance in business and industry requires further demonstration of a return on investment; documented cases and success stories are essential. The bottom line is that usability engineering does not add to overall cost, for two reasons: (1) usability does not add as much cost to the development process as many people think, and (2) good usability saves many other costs.
Considering cost added to the process, one must realize that any added cost is confined. Interaction development is a small part of total system development. It occurs early in the process, when the cost of making changes is still relatively low, and mainly impacts only a prototype, not the final system software.
Considering the cost savings attributable to good usability, it is easy to establish that poor usability is costly and that good usability is all about lowering costs. Usability is simply good business. The most expensive operational item in an interactive system is the user. People who develop software are concerned with the cost of development, but the people who buy and use a software application are concerned with the costs of usage. Development costs are mostly one-time costs, while operational costssuch as training, productivity losses, help desks and field support, recovery from user errors, dissatisfied employees, and system maintenance costs (the cost of trying to fix problems after release)accrue for years.
Unless the net of analysis is cast broadly enough, the problem with cost-benefit analysis is that one group pays development costs and another group gets the benefits. People who purchase computer systems are asking which costs more: user-based tasks that are quick, efficient, and accurate or errorprone tasks that take more time? Confused users or confident and competent users?
Beyond this kind of argumentation, used in software engineering
for years, substantial measurable economic advantage can be accrued from
usability. Case studies have demonstrated that large sums of real money can
be saved by increasing user (employee) productivity alone (Bias and
Mayhew, 1994). In the end, these are the cases that will make the difference.
THE FUTURE
HCI is a relatively young and broadly diverse field with a
rapidly growing impact on the world of computing. Usability, especially in
every-citizen interfaces, is becoming recognized as crucial for the
national information infrastructure. The future of HCI in this context can
be viewed from a perspective of product and process.
Future of HCI in Products
A rich part of the future of HCI is in its application areas, which are growing more rapidly than the HCI methods needed for their development. As an example, it is unlikely that usability methods developed for desktop applications will apply directly to virtual environments, one of the most exciting areas of applications development. Despite intense and widespread research in virtual environments, very little work has been applied toward developing the usability methods that will be required to evaluate this new technologya necessary coupling if virtual environments are to reach their full potential. Similarly, groupware and computer-supported cooperative work (Baecker, 1993; Grudin, 1994), multimedia (Blattner and Dannenberg, 1992), hypermedia, and interface access for the disabled or impaired persons (Williges and Williges, 1995) will require development of new methods for design and usability evaluation. Educational technology for the classroom, the World Wide Web, and the home is emerging as a giant application area. Perhaps nowhere is usability more important than in the discipline of education, where understanding and communication of concepts and ideas are the stock and trade.
Finally, the Internet, the World Wide Web, and cyberspace are incredibly fast-growing application domains bringing new kinds of usability challenges. The World Wide Web is a technological and sociological frontier with many analogies to the frontier that was the American West over a century agolawlessness and disorganization, with exploration and expansion in every direction.
Studies show that users having trouble with an interactive system often cannot find solutions in user manuals or from on-line help; they are more likely to ask a friend, colleague, or co-worker for help. This strategy can work in a local setting where there are other users. However, users of the national information infrastructure will often be remote and distributed, using a network as their work setting. For these isolated users, who are less able to tolerate poor user interfaces and who will abandon applications they find too difficult to learn and use, there is often no one to ask when things go wrong at the computer, and usability will have a large impact on their productivity and satisfaction. For this large-scale environment with its diversity of user types and characteristics, its variety of application types, and potential user isolation, usability takes on special importance.
Additionally, the interaction styles and techniques of future products can be expected to expand beyond the currently ubiquitous WIMPwindows, icons, menus, pointersor desktop-style interface. While WIMP interfaces have provided a great step forward for interfaces in static situations (e.g., word processing, spreadsheets), innovative interaction techniques that go beyond the WIMP paradigm are necessary to meet user interface needs of demanding, real-time, high-performance applications such as those found in military applications, medical systems, "smart road" applications, and so on. Researchers are promoting a greatly expanded vision of interaction beyond the limited interaction styles now available via just keyboard and mouse, including extensions to current work in graphic and visual displays (Mullet and Sano, 1995), use of hands and feet (Buxton, 1986), eye movement (Jacob, 1993), haptic (touch) feel and force feedback (Baecker et al., 1995), audio and sound (Brewster et al., 1993; Gaver and Smith, 1995), voice (McCauley, 1984), and stylus and gesture (Goldberg and Goodisman, 1995).
Finally, many technology forecasters have predicted that the most significant area of future applications may be computing embedded in appliances, homes, offices, vehicles, and roads. Sometimes called wearable computers, these devices can be strapped to one's wrist or embedded in a shoe! A recent television news feature (CNN News, July 1996) described a project at Massachusetts Institute of Technology in which a pair of shoes will, indeed, be instrumented so that, as the wearer gets milk out for breakfast, sensors will note that the milk supply is getting low! Approaching the grocery store on the way home, the system speaks via a tiny earphone to remind the shoe's wearer of the need to pick up some milk.
The requirements for usability of desktop and other familiar
systems will pale in comparison to the importance of usability in this new era
of computing. That "every citizen" will not tolerate training courses,
user manuals, or on-line help to operate everyday objects such as
refrigerators and automobiles will compel designers to take seriously their
responsibility for usability. Issues of social impact carry high risks if this kind
of every-citizen interface is threatening, intimidating, or difficult to use.
In successful designs the computing component will be transparent,
with users not even thinking of themselves as users of computers. When
human factors was first adapted to user interfaces (e.g., Williges et al.,
1987), ergonomics was largely filtered out. Interestingly, new devices,
combining hardware, software, and firmware as "appliances," will require a
reintegration of ergonomics as a part of usability.
Future of HCI Processes
Developers of future HCI processes will struggle to keep pace with these new application areas and interaction styles. One area that is already changing among real-world system developers is the representation of roles and skills in interactive system development teams. Usability specialists, human factors engineers, and HCI practitioners are starting to take their long-overdue places alongside systems analysts and software engineers. These new roles imply the need for new kinds of training in HCI methods. These roles have already begun to be joined by those with technical writing and documentation skills and especially by those with graphics and visual design skills (Tufte, 1983; Mullet and Sano, 1995)for example, to use color effectively (Shubin et al., 1996) and to design icons, avatars, and rendered images.
It is also expected that a significant increase in future HCI activity will be applied to developing new methods. There is an ongoing need for new high-impact usability evaluation methods. High impact means cost effective, applicable to a wide variety of application types (e.g., World Wide Web applications), applicable to many new interaction styles (e.g., virtual environments), and suitable for gathering usability data from remote and distributed user communities.
Among the approaches to remote evaluation emerging now, most are either limited to subjective user feedback (Abelow, 1993) or require expensive bandwidth to support video conferencing as an extension of the usability lab (Hammontree et al., 1994). A method based on user-assisted critical incident gathering (Hartson et al., 1996) has been proposed to bypass the bandwidth requirements for full-time video transmission and to cut analysis costs.
Methods and software support tools are also in demand for boosting return on investment of resources committed to usability evaluation. Koenemann-Belliveau et al. (1994, p. 250) have articulated this need: "We should also investigate the potential for more efficiently leveraging the work we do in empirical formative evaluationways to 'save' something from our efforts for application in subsequent evaluation work." Most of the time results from usability evaluation are applied only to specific usability problems in a single design. Database tools for information management of the results would accrue immediate gains in effective usability problem reporting (Jeffries, 1994; Pernice and Butler, 1995). More significantly, a usability database tool would afford some "memory" to the process, amortizing, through reuse of analysis, the cost of results across design iterations and across multiple products and projects. Beyond organizational boundaries, a collective usability database could serve as a commonly accessible repository of a science base for the HCI community and as a practical knowledge base for exemplar usability problems, solutions, and costs.
The future of HCI is both exciting and challenging. In moving
beyond GUIs and in developing new methods, problems continue to increase.
But the promise of these new products and processes will come to fruition in
an every citizen interface for the national information infrastructure.
ACKNOWLEDGMENT
Many thanks to Dr. Deborah Hix, of Virginia Tech, for her help
in providing inputs and in reading this paper.
REFERENCES
Abelow, D. (1993). Automating Feedback on Software Product Use. CASE Trends, 15-17.
Baecker, R. M. (Ed.). (1993). Readings in Groupware and Computer-Supported Cooperative Work: Assisting Human-Human Collaboration. San Francisco: Morgan-Kaufmann.
Baecker, R. M., Grudin, J., Buxton, W. A. S., and Greenberg, S. (1995). Touch, Gesture, and Marking. In R. M. Baecker, J. Grudin, W. A. S. Buxton, and S. Greenberg (Eds.), Readings in Human-Computer Interaction: Toward the Year 2000. San Francisco: Morgan-Kaufmann, 469-482.
Barnard, P. (1993). The Contributions of Applied Cognitive Psychology to the Study of Human-Computer Interaction. In R. M. Baecker, J. Grudin, W. A. S. Buxton, and S. Greenberg (Ed.), Readings in Human Computer Interaction: Toward the Year 2000. San Francisco: Morgan-Kaufmann, 640-658.
Bias, R. G. and Mayhew, D. J. (Eds.). (1994). Cost Justifying Usability. Boston: Academic Press.
Blattner, M. M. and Dannenberg, R.B. (Eds.). (1992). Multimedia Interface Design. New York: ACM Press.
Bodker, Susanne. (1991). Through the Interface: A Human Activity Approach to User Interface Design. Hillsdale, NJ: Lawrence Erlbaum Associates.
Brewster, S. A., Wright, P. C., and Edwards, A. D. N. (1993). An Evaluation of Earcons for Use in Auditory Human-Computer Interfaces. Proceedings of INTERCHI Conference on Human Factors in Computing Systems. New York: ACM Press, 222-227.
Butler, K. A. (1996). Usability Engineering Turns 10. Interactions (January), 58-75.
Buxton, W. (1986). There's More to Interaction than Meets the Eye: Some Issues in Manual Input. In D. A. Norman and S. W. Draper (Ed.), User Centered System Design. Hillsdale, NJ: Lawrence Erlbaum Associates, 319-337.
Card, S. K. and Moran, T. P. (1980). The Keystroke-Level Model for User Performance Time with Interactive Systems. Communications of the ACM, 23, 396-410.
Card, S. K., Moran, T. P., and Newell, A. (1983). The Psychology of Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.
Carroll, J. M. (1984). Minimalist Design for Active Users. Proceedings of Human-Computer InteractionInteract '84, September, Amsterdam: North-Holland, 39-44.
Carroll, J. M. (Ed.). (1995). Scenario-Based Design: Envisioning Work and Technology in System Development. New York: John Wiley and Sons, Inc.
Carroll, J. M., Kellogg, W. A., and Rosson, M. B. (1991). The Task-Artifact Cycle. In J. M. Carroll (Ed.), Designing Interaction: Psychology at the Human-Computer Interface. Cambridge, England: Cambridge University Press, 74-102.
Carroll, J. M. and Rosson, M. B. (1992). Getting Around the Task-Artifact Cycle: How to Make Claims and Design by Scenario. ACM Transactions on Information Systems, 10, 181-212.
Chin, J. P., Diehl, V. A., and Norman, K. L. (1988). Development of an Instrument Measuring User Satisfaction of the Human-Computer Interface. Proceedings of CHI Conference on Human Factors in Computing Systems, May 15-19, New York: ACM, 213-218.
Diaper, D. (Ed.). (1989). Task Analysis for Human-Computer Interaction. Chichester, England: Ellis Horwood Limited.
Draper, S. W. and Barton, S. B. (1993). Learning by Exploration and Affordance Bugs. Proceedings of INTERCHI Conference on Human Factors in Computing Systems (Adjunct). New York: ACM, 75-76.
Ehn, P. (1990). Work Oriented Design of Computer Artifacts (2nd Ed.). Hillsdale, NJ: Lawrence Erlbaum Associates.
Foley, J. D. and Wallace, V. L. (1974). The Art of Natural Graphic Man-Machine Conversation. Proceedings of the IEEE, 63(4), 462-471.
Foley, J. D., van Dam, A., Feiner, S. K., and Hughes, J. F. (1990). Computer Graphics: Principles and Practice. Reading, MA: Addison-Wesley.
Gaver, W. W. and Smith, R. B. (1995). Auditory Icons in Large-Scale Collaborative Environments. In R. M. Baecker, J. Grudin, W. A. S. Buxton, and S. Greenberg (Eds.), Readings in Human-Computer Interaction: Toward the Year 2000. San Francisco: Morgan-Kaufmann, 564-569.
Goldberg, D. and Goodisman, A. (1995). Stylus User Interfaces for Manipulating Text. In R. M. Baecker, J. Grudin, W. A. S. Buxton, and S. Greenberg (Eds.), Readings in Human-Computer Interaction: Toward the Year 2000. San Francisco: Morgan-Kaufmann, 500-508.
Gould, J. D., Boies, S. J., and Lewis, C. (1991). Making Usable, Useful, Productivity-Enhancing Computer Applications. Communications of the ACM, 34(1), 74-85.
Gray, W. D., John, B. E., Stuart, R., Lawrence, D., and Atwood, M. (1990). GOMS Meets the Phone Company: Analytic Modeling Applied to Real-World Problems. Proceedings of INTERACT '90Third IFIP Conference on Human-Computer Interaction, August 27-31, Amsterdam: North-Holland Elsevier Science Publishers.
Grudin, J. (1994). Groupware and Social Dynamics: Eight Challenges for Developers. Communications of the ACM, 37(1), 92-105.
Hammond, N., Gardiner, M. M., Christie, B., and Marshall, C. (1987). The Role of Cognitive Psychology in User-Interface Design. In M. M. Gardiner and B. Christie (Eds.), Applying Cognitive Psychology to User-Interface Design. Chichester: Wiley, 13-53.
Hammontree, M., Weiler, P., and Nayak, N. (1994). Remote Usability Testing. Interactions (July), 21-25.
Harrison, M. and Thimbleby, H. (Ed.). (1990). Formal Methods in Human-Computer Interaction. Cambridge, England: Cambridge University Press.
Hartson, H. R. and Gray, P. D. (1992). Temporal Aspects of Tasks in the User Action Notation. Human-Computer Interaction, 7, 1-45.
Hartson, H. R. and Hix, D. (1989). Toward Empirically Derived Methodologies and Tools for Human-Computer Interface Development. International Journal of Man-Machine Studies, 31, 477-494.
Hartson, H. R., Siochi, A. C., and Hix, D. (1990). The UAN: A User-Oriented Representation for Direct Manipulation Interface Designs. ACM Trans. Inf. Syst., 8(3), 181-203.
Hartson, H. R., Castillo, J. C., Kelso, J., Kamler, J., and Neale, W. C. (1996). Remote Evaluation: The Network as an Extension of the Usability Laboratory. Proceedings of CHI Conference on Human Factors in Computing Systems. New York: ACM, 228-235.
Hix, D. and Hartson, H. R. (1993a). Developing User Interfaces: Ensuring Usability Through Product and Process. New York: John Wiley and Sons.
Hix, D. and Hartson, H. R. (1993b). Formative Evaluation: Ensuring Usability in User Interfaces. In L. Bass and P. Dewan (Eds.), Trends in Software, Volume 1: User Interface Software. New York: Wiley, 1-30.
Hix, D. and Hartson, H. R. (1994). IDEAL: An Environment for User-Centered Development of User Interfaces. Proceedings of EWHCI'94: Fourth East-West International Conference on Human-Computer Interaction, 195-211.
Hutchins, E. L., Hollan, J. D., and Norman, D. A. (1986). Direct Manipulation Interfaces. In D. A. Norman and S. W. Draper (Eds.), User Centered System Design. Hillsdale, NJ: Lawrence Erlbaum Associates, 87-124.
Jacob, R. J. K. (1993). Eye-Movement-Based Human-Computer Interaction Techniques: Toward Non-Command Interfaces. In H. R. Hartson and D. Hix (Eds.), Advances in Human-Computer Interaction. Norwood, NJ: Ablex, 151-190.
Jeffries, R. (1994). Usability Problem Reports: Helping Evaluators Communicate Effectively with Developers. In J. Nielsen and R. L. Mack (Eds.), Usability Inspection Methods. New York: John Wiley and Sons, Inc., 273-294.
Kieras, D. E. (1988). Towards a Practical GOMS Model Methodology for User Interface Design. In M. Helander (Ed.), Handbook of Human-Computer Interaction. Elsevier Science Publishers B. V., 135-157.
Kieras, D. and Polson, P. G. (1985). An Approach to the Formal Analysis of User Complexity. International Journal of Man-Machine Studies, 22, 365-394.
Koenemann-Belliveau, J., Carroll, J. M., Rosson, M. B., and Singley, M. K. (1994). Comparative Usability Evaluation: Critical Incidents and Critical Threads. Proceedings of CHI Conference on Human Factors in Computing Systems. New York: ACM, 245-251.
LeCompte, M. D. and Preissle, J. (1993). Ethnography and Qualitative Design in Educational Research (2nd Ed.). San Diego: Academic Press.
Lewis, C., Polson, P., Wharton, C., and Rieman, J. (1990). Testing a Walkthrough Methodology for Theory-Based Design of Walk-up-and-Use Interfaces. Proceedings of CHI Conference on Human Factors in Computing Systems, April 1-5, New York: ACM, 235-242.
MacKenzie, S. (1992). Fitts' Law as a Research and Design Tool in Human-Computer Interaction. Human-Computer Interaction, 7, 91-139.
MacLean, A., Young, R. M., Bellotti, V. M. E., and Moran, T. P. (1991). Questions, Options, and Criteria: Elements of Design Space Analysis. Human-Computer Interaction, 6, 201-250.
Macleod, M. and Bevan, N. (1993). Music Video Analysis and Context Tools for Usability Measurement. Proceedings of INTERCHI Conference on Human Factors in Computing Systems, April 24-29. New York: ACM, 55.
Mayhew, D. J. (1992). Principles and Guidelines in Software User Interface Design. Englewood Cliffs, NJ: Prentice-Hall.
McCauley, M. E. (1984). Human Factors in Voice Technology. In F. A. Muckler (Ed.), Human Factors Review. Santa Monica, CA: Human Factors Society, 131-166.
Meister, D. (1985). Behavioral Analysis and Measurement Methods. New York: Wiley.
Moran, T. P. (1981). The Command Language Grammar: A Representation for the User Interface of Interactive Computer Systems. International Journal of Man-Machine Studies, 15, 3-51.
Muller, M. J. (1991). PICTIVEAn Exploration in Participatory Design. Proceedings of CHI Conference on Human Factors in Computing Systems, April 27-May 2. New York: ACM, 225-231.
Mullet, K. and Sano, D. (1995). Designing Visual Interfaces. Mountain View, CA: SunSoft Press.
Myers, B. A. (1989). User-Interface Tools: Introduction and Survey. IEEE Software, 6 (1), 15-23.
Myers, B. A. (1993). State of the Art in User Interface Software Tools. In H. R. Hartson and D. Hix (Eds.), Advances in Human-Computer Interaction. Norwood, NJ: Ablex.
Myers, B. A. (1995). State of the Art in User Interface Software Tools. In R. M. Baecker, J. Grudin, W. A. S. Buxton, and S. Greenberg (Eds.), Readings in Human-Computer Interaction: Toward the Year 2000. San Francisco: Morgan-Kaufmann, 323-343.
Nielsen, J. (1989). Usability Engineering at a Discount. In G. Salvendy and M. J. Smith (Eds.), Designing and Using Human-Computer Interfaces and Knowledge-Based Systems. Amsterdam: Elsevier Science Publishers, 394-401.
Nielsen, J. (1992). Finding Usability Problems Through Heuristic Evaluation. Proceedings of CHI Conference on Human Factors in Computing Systems, May 3 - 7. New York: ACM, 373-380.
Nielsen, J. (1993). Usability Engineering. San Diego: Academic Press. Inc.
Nielsen, J. and Mack, R. L. (Ed.). (1994). Usability Inspection Methods. New York: John Wiley and Sons.
Nielsen, J. and Molich, R. (1990). Heuristic Evaluation of User Interfaces. Proceedings of CHI Conference on Human Factors in Computing Systems, April 1-5. New York: ACM, 249-256.
Norman, D. A. (1986). Cognitive Engineering. In D. A. Norman and S. W. Draper (Eds.), User Centered System Design. Hillsdale, NJ: Lawrence Erlbaum Associates, 31-61.
Norman, D. A. and Draper, S. W. (Eds.). (1986). User Centered System Design: New Perspectives on Human-Computer Interaction. Hillsdale, NJ: Lawrence Erlbaum Associates.
Payne, S. J. (1991). Display-based action at the user interface. International Journal of Man-Machine Studies, 35, 275-289.
Pernice, K. and Butler, M. B. (1995). Database Support for Usability Testing. Interactions (January), 27-31.
Shneiderman, B. (1983). Direct Manipulation: A Step Beyond Programming Languages. IEEE Transactions on Computers, 16(8), pp. 57-69.
Shneiderman, B. (1992). Designing the User Interface: Strategies for Effective Human-Computer Interaction (2nd Ed.). Reading, MA: Addison-Wesley.
Shubin, H., Falck, D., and Johansen, A. G. (1996). Exploring Color in Interface Design. Interactions (July/August), 36-48.
Smith, S. L. and Mosier, J. N. (1986). Guidelines for Designing User Interface Software (ESD-TR-86-278/MTR 10090). MITRE Corporation.
Tufte, E. R. (1983). The Visual Display of Quantitative Data. Cheshire, CT.: Graphics Press.
Virzi, R. A., Sokolov, J. L., and Karis, D. (1996). Usability Problem Identification Using Both Low- and High-Fidelity Prototypes. Proceedings of CHI Conference on Human Factors in Computing Systems. New York: ACM, 236-243.
Weller, H. G. and Hartson, H. R. (1992). Metaphors for the Nature of Human-Computer Interaction in an Empowering Environment: Interaction Style Influences the Manner of Human Accomplishment, 8(3), 313-333.
Wharton, C., Bradford, J., Jeffries, R., and Franzke, M. (1992). Applying Cognitive Walkthroughs to More Complex User Interfaces: Experiences, Issues, and Recommendations. Proceedings of CHI Conference on Human Factors in Computing Systems, May 3 - 7, New York: ACM, 381-388.
Whiteside, J., Bennett, J., and Holtzblatt, K. (1988). Usability Engineering: Our Experience and Evolution. In M. Helander (Ed.), Handbook of Human-Computer Interaction. Amsterdam: Elsevier North-Holland, 791-817.
Williges, R. C. and Williges, B. H. (1995). Travel Alternatives for the Mobility Impaired: The Surrogate Electronic Traveler (SET). In A. D. N. Edwards (Ed.), Extra-Ordinary Human-Computer Interaction: Interfaces for Users with Disabilities. New York: Cambridge University Press, 245-262.
Williges, R. C., Williges, B. H., and Elkerton, J. (1987). Software Interface Design. In G. Salvendy (Ed.), Handbook of Human Factors. New York: John Wiley and Sons, 1416-1449.
Wixon, D., Holtzblatt, K., and Knox, S. (1990). Contextual Design: An Emergent View
of System Design. Proceedings of CHI Conference on Human Factors
in Computing Systems, April 1-5. New York: ACM, 329-336.
ADDITIONAL READING
Representative supplementary references here are chosen for breadth. Cited references also are representative but not all are repeated here.
ACM. (1990). Resources in Human-Computer Interaction. New York: ACM Press.
Baecker, R. M. and Buxton, W. A. S. (Eds.). (1987). Readings in Human-Computer Interaction: A Multidisciplinary Approach. San Francisco, Morgan-Kaufmann.
Baecker, R. M., Grudin, J., Buxton, W. A. S., and Greenberg, S. (Eds.). (1995). Readings in Human-Computer Interaction: Toward the Year 2000. San Francisco: Morgan-Kaufmann.
Carroll, J. M. (Ed.). (1987). Interfacing Thought: Cognitive Aspects of Human-Computer Interaction. Cambridge, MA: The MIT Press.
Carroll, J. M. (Ed.). (1991).Designing Interaction: Psychology at the Human-Computer Interface. Cambridge, England: Cambridge University Press.
Hartson, H. R. and Hix, D. (1989). Human-Computer Interface Development: Concepts and Systems for Its Management. ACM Comput. Surv., 21(1), 5-92.
Helander, M. (Ed.). (1988). Handbook of Human-Computer Interaction. Amsterdam: North Holland.
Monk, A. F. and Gilbert, N. (1995). Perspectives on HCIDiverse Approaches. Cambridge, England: Cambridge University Press.
Olson, J. R. and Olson, G. M. (1990). The Growth of Cognitive Modeling in Human-Computer Interaction Since GOMS. Human-Computer Interaction, 5, 221-265.
Preece, J., Rogers, Y., Sharp, H., Benyon, D., Holland, S., and Carey, T. (1994). Human-Computer Interaction. Reading, MA: Addison-Wesley.
Rubin, J. (1994). Handbook of Usability Testing. New York: John Wiley and Sons.
Thimbley, H. (1990). User Interface Design. New York: ACM Press/Addison-Wesley.