Information Access and Usability
Christopher D. Wickens and Karen S. Seidler
We are a society increasingly driven by information. Accurate, up-to-date information underlies the effective and progressive functioning of governmental, economic, and social institutions, impacting everything from the focus and quality of public policy decisions to the competitiveness of industrial research and development. In this modern reality—sometimes labeled the information society (e.g., Salvaggio, 1989) or the postindustrial society (Bell, 1973)—knowledge and information become the primary resources, displacing manufactured goods and agricultural products as key commodities (Stonier, 1983).
A number of trends associated with the evolution of an information society beg attention from the human factors community. One trend is the greatly expanded quantity of information available to people in all professions. For the scholar, this translates into a 3 percent growth per year in the United States alone in the number of journals, professional conferences, and proceedings (King et al., 1981). Worldwide, the growth is even more rapid. This makes keeping abreast of current research on even a restricted topic daunting, and scholars are increasingly unsure of the extent to which they have located relevant literature, let alone digested it. Lawyers have witnessed a comparable overwhelming growth in the records they can consult for legal precedents. In the medical profession, the limited ability of physicians and other health professionals to extract from the literature relevant
information on patient care, teaching, and research has engendered so much concern (e.g., Huth, 1989) that a new subfield—medical informatics—has been created to address issues related to the development and operation of medical information systems (Hewins, 1990). Somewhat ironically, this subdiscipline is generating its own growing body of literature (see Chapter 4).
In industry, the exponential increases in system complexity have also resulted in voluminous documentation and procedures to support system operations and maintenance. The documentation required by the F-18 aircraft alone contains 300,000 pages. In nuclear facilities, it is not unheard of for manuals of procedures for dealing with accidents to exceed hundreds of pages. For the consumer, a growing wealth of information on products and costs is at least potentially available. In the face of this data-rich environment, users are experiencing frustration in locating, accessing, tracking, and interpreting the information that is germane to their concerns.
New information technologies, though, offer viable and powerful ways to manage information collections. Among these innovations are: (1) greatly enhanced storage and access capabilities (e.g., CD-ROM), (2) expanded communication systems that allow faster and greater connectivity, (3) hypermedia developments that allow greater flexibility in information representation and database ''navigation" (Gluschko, 1990), (4) the advanced graphics capabilities that make direct manipulation interfaces and data visualization possible (Newby, 1992), and (5) developments in artificial intelligence to support "intelligent" systems and interfaces that can, for example, infer user information needs and support dialogue in more natural language (Allen, 1991).
Yet advances in collection, storage, and dissemination technology do not ensure successful use of electronic information repositories. Muckler (1987) has documented the low "hit rate" obtained in typical library database searches, in terms of both the relevant documents that fail to be located (misses) and the unnecessary retrieval of many documents of limited usefulness or relevance (false alarms). In one case a computer search through 10 databases retrieved 16,816 abstracts that were identified as relevant for a particular topic. Only 166 were. More disturbing was the fact that in an independent, nonelectronic search through the literature, a user located 177 useful references that had not been found by any of the database searches. In a study of user success with a legal database, Blair and Maron (1985) found a similar pattern of results, with only a 20 percent recall of relevant documents. Even more alarming, though, was the finding that searchers believed they were recovering 75 to 80 percent of relevant documents. This finding is consistent with a general conclusion reached by MacGregor et al. (1987) that users overstated the success of their information retrieval from electronic databases.
Of course, there have been positive interactions as well. Wilson et al.
(1989) report a dramatic example: in eight incidents the lives of patients were saved because a search of the MEDLINE database uncovered information that led the physicians to change the course and direction of their treatment procedures. As we increasingly attend to the difficulties users have with information technologies, such a positive synergy between user and machine will become more common.
Yet this attention will need to extend beyond the technically sophisticated user. A growing proportion of the labor force will perform information-handling activities (i.e., production, maintenance, and interpretation) in the future. We have already seen a substantial growth in the number of white-collar workers (information handlers), from less than 18 percent of the workforce in 1900 to over half today (Koenig, 1990). And since communication and computing technologies have allowed data to be made directly available in the office, workers are increasingly being asked to confront the information stores without the aid of expert human search intermediaries (e.g., librarians) familiar with the database systems. Further, the growth in numbers of end users is not limited to the workplace. Electronic information databases are becoming more common in everyday private life. With the purchase of a modem and an on-line service, the average person can use a personal computer to tap into large databases at home and carry on many personal activities directly (e.g., make airline reservations, shop at home, conduct bibliographic searches). This means that a wide range of skills and tasks are being brought to the computer interface. These more casual or infrequent users are less likely to possess the intellectual resources or motivation to invest in a time-consuming learning process. The implications are that information system designers cannot rely on user perseverance or high technical skills to overcome poor interface design. Instead, user friendliness will become critical in attracting and keeping end users.
In summary, the issues may be represented by three major concerns. First, users must often confront enormous quantities of information in order to identify and utilize the small portion that is ultimately relevant and meaningful to them. Second, the interface and the database representation designed to manage the vast information collections may often not be compatible with the user's mental model of how the system should work, leading to user frustration and abandonment of the system. Third, even if a user is satisfied with the on-line operation of a system, he or she may be less satisfied with its output if many unwanted items are retrieved (i.e., high recall but low precision). Alternatively, although the user may be satisfied with the information retrieved, that satisfaction could be unwarranted because of an unsuspected high miss rate.
From a human factors perspective, we may think about the problems of information access and retrieval outlined above as resulting from the user's requirement to map an information need (sometimes ill defined) onto a very
large database whose contents and structure may or may not be immediately apparent. That is, information is somewhat "hidden" in the database. Lucky (1989) describes this as a problem of acquiring meta-information (information about information). That is, users must determine whether desired information is contained within the database, how it is represented, where it is located, and how to access it. Exploiting the new information technologies to address these problems defines a host of important human factors research issues for the next decade, such as the following: What is the optimum display interface through which the user can understand what information is available? What is the appropriate control/display interface through which the user can explore the information base? How strongly and how well can intelligent systems draw correct inferences about human needs and wishes and how well can those systems help the user formulate and refine those wishes? How can one adequately assess performance with information systems, since user satisfaction may not correlate with recall and precision indices?
In the following pages, we discuss these issues as they pertain to three selected classes of databases: (1) the restricted database, the well-structured database for a particular system in a restricted domain; (2) the fluid database, the less-structured database of scientific, academic, or general knowledge; and (3) the base constituted by scientific data. We recognize that the three classes are not discrete; that is, they share many overlapping features relating to information accessibility and interpretability. In fact, we view the restricted and fluid databases along a continuum that reflects the degree to which the database organization can be determined by a user prior to actual use. For convenience and parsimony of expression, we also take the liberty of using the term electronic database in its broadest sense, that is, to represent any type of electronic information repository (e.g., an on-line bibliographic system, a videotext system, a computerized maintenance manual). We conclude this chapter by focusing in detail on the set of human factors research issues that are common to all of these areas.
DATABASE REPRESENTATION ISSUES
The most structured information-representation domain, and the one most readily studied from a human factors viewpoint, is one in which the knowledge pertains to the functioning of a single, well-structured system—a restricted domain. Examples include the on-line help system for a word-processing system, the airline flight reservation systems (Boehm-Davis et al., 1992), the onboard in-flight library of information about an aircraft and its surrounding airspace conditions (Curran, 1991), all of the entries in a
maintenance document for a complex piece of equipment, and the contents of an industrial inventory. These systems represent the electronic counterparts to such conventional paper-based items as manuals, descriptions of procedures, schedules, encyclopedias, and handbooks.
In the restricted domain, the contents of the database often are less of a mystery to the user than is the location of an information item within the system and the means of accessing it. In the hard-copy counterparts, users may consult a table of contents or an index to locate information; if the document is alphabetically arranged (e.g., a dictionary), its structure is immediately apparent and users can infer the location of information. Alternatively, users might choose to thumb through the document or browse to find information of interest. To keep track of their place when cross-referencing, they might bend down corners of pages or use highlighters or bookmarks. Advanced electronic systems sometimes have analogous capabilities.
In the electronic version of an information system, the issue is how to design the interface so that the user can (a) readily locate needed items of information and (b) access the items efficiently (this might include needing to shift rapidly between sets of "related" items of information). The first need has to do with the organizational structure of the database (e.g., hierarchical, matrix, network; see Durding et al., 1977), and the second, with the navigational tools for moving from entry to entry (Seidler and Wickens, 1992). The organizational structure may or may not be independent of the navigational tools. For example, a database may be organized in a strict hierarchical fashion, but navigational tools may allow the user to directly access any node in the database from any other node with a single command, thus bypassing the hierarchical relationships. This contrast is closely related to that between menu-driven search and key word search.
Much of the empirical investigation relevant to these areas has focused on such issues as menu organization and key word systems (Norman, 1991; Shneiderman, 1987). For instance, the issue of the trade-off between menu depth (the number of levels in the hierarchy) and breadth (the number of items per level) has been a popular research topic (e.g., Kiger, 1984; Miller, 1981; Snowberry et al., 1983). The general consensus of these studies has been that search time lessens and accuracy improves as breadth is increased and depth decreased. Increasing depth seems to increase the likelihood of becoming "lost" in the information space. Other menu studies (e.g., Card, 1982; Giroux and Belleau, 1986) have looked at the ordering and organization of menu items within a page (e.g., alphabetical, semantic, or random) and have found systematic arrangements better than random ordering of items. There have also been studies looking at the advantages of adding menu descriptors (e.g., Latremouille and Lee, 1981; Snowberry et al., 1985); these have been found to improve performance when applied to upper levels of the menu hierarchy only. Most of these studies of menu organization,
though, have used relatively small, homogeneous, and often abstract databases (Fisher et al., 1990) unlike those typically found in operational settings. Also, search questions were generally goal-directed, which means that results may be inappropriate for understanding browsing behavior.
Research is still needed on how to organize both the structure and the navigational tools in a way that is compatible with both the users' mental model of the database and the task-defined needs to access the information in a particular sequence. A classic study by Roske-Hofstrand and Paap (1986) revealed that F-16 pilots used an in-flight information menu more readily when its organizational structure corresponded to their mental model of which systems were related to each other; the pilots were less likely to use the menu when the menu designers' own plan guided the structure. Prior to obtaining user input on the question of structure, the designer should be able to define the set of options. This can be difficult because there are so many different ways in which groups of "information nodes" could be organized. For an aircraft, they might be related in terms of system components: thus, all entries pertaining to a common system (e.g., fuel-use instruments) would be close together or commonly grouped. Alternatively, they might be grouped according to phase of flight, so all instruments that are frequently consulted in the same flight phase (and hence frequently used in sequence) would be located together, even if they relate to entirely different systems. There is little or no guidance as to whether one organizational structure is preferable, how either might relate to the users' mental model, how homogeneous these mental models may be across different users, or whether the organizational structure for one task (e.g., normal operation) might differ substantially from the structure needed for a different task (e.g., fault diagnosis and trouble-shooting). It is possible that these different structures can exist in parallel through implementation of networks or redundant nodes.
Allowing the user to tailor or structure the organization adaptively as he or she sees fit is an attractive approach. Yet this solution has three potential dangers not always considered when making design recommendations:
The user might not know the ideal organization because all possible scenarios in which information is needed cannot be anticipated a priori.
There may be undesirable set-up costs in establishing configurable displays.
Individual structuring will lead to a lack of consistency across different databases.
This last concern is crucial, as has been understood by those considering the lack of consistency across such systems as word-processing applications,
keyboards, and aircraft (Andre and Wickens, 1992; Wiener, 1989). What should be the trade-off, then, between flexibility and adaptability on the one hand, and consistency on the other? It may be that having too much freedom to structure information differently in a system is as bad as having too little. Human cognition is somewhat adaptable, and it may well be that the user can adapt to a non-ideal but fixed information structure more rapidly and effectively than the user can mold or adapt the structure to his or her ideal.
Identifying the elements that foster this adaptation is the challenge. It requires making the organizational structure and navigational procedures readily apparent to the user, so that users always know where they are in the database relative to where they want to be. This issue of visibility of database structure to the user is addressed by the implementation of electronic database "maps," whose utility has been well validated (e.g., Vincente and Williges, 1988). Major design challenges still confront the human factors community in addressing how such maps can spatially represent the vast number of nodes and entries in large databases in a way that allows the user to visualize the full structure readily (Mackinlay et al., 1991; Shneiderman, 1987).
When dealing with information in a restricted domain, the user generally must access or "unpack" specific pieces of information that exist at nodes of the database and whose identity is known before the search begins. For example, a technician might need to look up a particular maintenance procedure. The contents of the restricted-domain databases are fairly well defined in advance by the system designer. In contrast, many domains have boundaries that are far less concretely defined, and the organizational structure within these domains is much less well described by either a systems hierarchy or a simple n-dimensional matrix. Instead, the relationships between information sources in the fluid database, such as a library's holdings, derive meaning from the information needs and use context. The task in using such databases may extend beyond "unpacking" information within a node to understanding the relationships between items in the database. As examples, we may consider the on-line bibliographic retrieval of articles about a particular topic (Newby, 1992); books of a user's preference (Pejtersen, 1989); the set of legal decisions, statutes, and regulations bearing on a particular type of case; the set of aircraft incident reports with common features listed in the Aviation Safety Reporting System (Williams et al., 1992); and a set of medical cases with similar typologies (Hubbard et al., 1987; see also Chapter 4).
A database of this sort is characterized by the fact that a single organizational
framework is not typically imposed at the outset of data generation, or, if one is, then it must evolve to accommodate the unpredictable growth of information. This contrasts with the computer help system or the maintenance information system mentioned in the previous section; there the components and functions are specified in advance by design and, therefore, possess a fairly concise hierarchical or network structure. Instead, the fluid databases are contributed to by various people (or events) over time, have no tightly constrained structure or boundary, and are employed by users who may have a very divergent set of task needs. Any efforts by one individual to impose a particular parsimonious taxonomy will probably defeat other users with different needs and information uses.
Consider, for example, a database of human factors research. Different people will have very different criteria regarding the boundaries of such a database. Should it include the social psychology of multiperson interaction? Personality types as they affect this interaction? Physiology? Management and organizational practices? Correspondingly, how should the information sources in the data be organized? Their multidimensionality defies classification into a simple hierarchical framework. Four logical dimensions of organization of the human factors data immediately come to mind. That is, such data can be organized according to:
the domain of applicability (e.g., computer, transportation, consumer product),
the functions addressed (e.g., control, communications, monitoring),
the relevant psychological components (e.g., perception, memory, attention), and
the chronological order of the study.
Other dimensions are also possible.
The difficulty in organizing and condensing the entire contents of the fluid database makes a menu structure generally unwieldy as a search mechanism (although menus have sometimes been employed in front-end database aids and some very specialized domain databases, such as CANSEARCH, an expert system to search cancer therapy literature, described by Pollitt, 1987). Furthermore, menus do not offer search power for complex queries. Instead, the command mode, usually via key word entry, places the user in an active search role (Shneiderman, 1987) and is commonly used to enact the search request. In order to generate sets and subsets of information items, a user constructs a query statement that is then compared against a document representation via some information retrieval technique (Belkin and Croft, 1987). The information system may employ any one of a number of information retrieval techniques. The most common method used today in large operational systems is the exact match, whereby the users' query statement
must exactly match the document representation, often described by key words, in order for a document to be recovered. This method may involve Boolean, string, or full-text search. Exact match techniques generally result in high-precision retrieval but low recall. Other retrieval methods—partial match techniques—compare the query with a document or group of documents represented as sets of features or index terms. These terms are assigned a priori by either human indexers or machine-automated processes. Documents are then ranked or recovered based upon some measure of the similarity (e.g., probabilistic, vector-space, cluster, fuzzy set, spreading activation) between the query and document representation.
Some key word searches are performed relatively easily. These generally encompass what has been termed known-item (Blair, 1984) or data retrieval (Vigil, 1985) searches, such as when a user seeks a specific known title, author, date, or ISBN number of a record. Other types of searches (topical or subject searches), requiring more complex strategies and problem-solving behaviors, are generally less effective (Muckler, 1987). This is largely because users have trouble translating concepts into the formal logic of the computer. In indexed databases, the key words may not correspond to the user's representation of the terms. In full-text databases, there may be problems with synonymy and imprecise language. In both, key word systems users often experience difficulty in constructing and manipulating Boolean statements.
To assist users in their search tasks, artificial intelligence techniques are increasingly being applied to information system design. Successful prototype systems to date have already automated many of the mechanical aspects of the search process that were problematic for users. Examples include guidance in connecting to an on-line service or database, downloading results, and translating the user query into a Boolean statement (Hawkins, 1988). The more intellectual components of the search process, such as the formulation of the search query and selection of a search strategy, however, are much less well understood, although they have received attention. For these latter research attempts, a major focus has been describing the knowledge people use when searching for information, with the goal of incorporating this knowledge into the expert system (e.g., Croft and Thompson, 1987).
Other research has examined various aspects of the cognitive processes involved in information search, including learning (e.g., Polson and Friedman, 1988), problem solving (e.g., Chen and Dhar, 1991), memory (e.g., Prasse et al., 1988), and comprehension, again with the aim of supporting information systems design. For instance, in their research on problem-solving behavior in information search, Chen and Dhar (1991) identified five strategies used during on-line catalog search and reported improved performance with a system redesigned to respond to these strategies. Other problem-solving
research has included identification of search tactics (Bates, 1986), development and selection of search strategy (Brooks et al., 1986), identification of heuristics used in search formulation (Harter and Peters, 1985), and identification of query formulation subtasks (Harter, 1986). Some research has also focused on developing user models that can be incorporated into an intelligent system to enhance its capability to respond to the varied needs and the inherent flexibility of individual users (e.g., Brooks et al., 1986). Modeling knowledge acquisition has also been of interest as a way to facilitate machine learning that will allow intelligent systems to build their own capabilities over time.
Drenth et al. (1991) noted, however, that these models have rarely focused on anything beyond task identification and documentation of procedures. Our understanding of these cognitive processes remains limited. We have no comprehensive picture of the knowledge and expertise underlying the intellectual component of the search process, in part because of the failure to attend to methodological issues. Much of the behavior involved in human-computer interaction is covert. If we are to explore information retrieval behavior within a research paradigm reflecting realistic operational settings and constraints, we must identify and use nontraditional techniques (e.g., verbal protocol analysis, critical incident technique; Flanagan, 1954). Little attention to date has been devoted to exploring these alternative approaches or to developing innovative techniques to elicit and analyze data.
Other considerations must also be addressed before flexible, powerful, and intelligent intermediaries can truly be implemented and made available for use by the general population. A critical but rarely addressed issue concerns the value of information. What attributes define the subjective value or usefulness of information to the user (Morehead and Rouse, 1985)? How much responsibility should the computer be given for deciding (and, therefore, providing) what is of value, and what is not? How much information (title? abstract?) should be provided to users to help define the relevant set? Can the user specify the necessary attributes of value in advance of a search in such a way that an expert system could meaningfully use these to guide the search and retrieval process?
A somewhat related issue concerns the evaluation of retrieval system performance. We have noted hints and some documentation of the less than adequate performance of many database searches (Mann, 1987) but no formal procedure for evaluating their performance level is available. Even a simple signal detection theory analysis of hits and misses begs the question of what is considered signal and what is noise (Granda and Halstead-Nussloch, 1988; Morehead and Rouse, 1985). It may be that the evaluation of retrieval performance needs to encompass the use of the information, that is, how the information accessed affects task decisions (see Hewins, 1990).
Evaluating retrieval performance becomes particularly complex when
the users' goals are not restricted to locating a number of cases with certain attributes and examining each in detail but include understanding more global aspects of the database, such as the correlations or constraints within the database—for example, whether human factors studies involving group processes are related primarily to training or to performance. This kind of understanding is often achieved through browsing, and good performance metrics have not been established for browsing.
Our discussion of fluid domains has focused on the information search process. As in restricted domains, it is reasonable to suppose that search performance will be supported if effective graphic displays to support visualization of the database are available. However, three features of fluid databases present challenges for the design of such displays. First, the size of fluid databases may be overwhelming, so that very little of the relevant database can be viewed at any one time, and still less of it can be reviewed in any detail (e.g., the text that might describe a particular case). Consider, for example, how extensive is the human factors database of all articles published in the Human Factors Journal, Human Factors Proceedings, Ergonomics , and Applied Ergonomics. Second, database browsing is best done flexibly, in real time, so that sometimes a user can "unpack" and examine a particular item or set of items while at other times he or she might merely want to note the location of the items in relationship to other items (i.e., the user should be able to "zoom in" and "zoom out" to various levels of detail). This capability might, for example, help define ''clusters" or "use groups," or it might allow the user to determine the correspondence (or lack of correspondence) between the independent data sets of the same underlying space. For example, the density of researchers actively working in certain areas within a particular field could be overlaid on a representation of the frequency of problems calling for research in that field, in order to determine if there is a mismatch and how that mismatch might best be addressed. Third, the structure of such data may both be ill defined (i.e., different from user to user) and have a complex relational structure that does not easily lend itself to a two-dimensional visual rendering. For example, it is clear that many of the terms relating items within a database have "analog" characteristics so that the spatial proximity between items is a meaningful semantic concept. The degree of similarity between the domain represented by two cases (e.g., studies or incidents) is one example. Two legal cases may deal with identical circumstances or with circumstances that have varying levels of similarity. Analog relations might also be defined in terms of the date of court proceedings, the level of the court, and so on. These analog characteristics make "distance" a relevant concept for the database study. But how distance in a multidimensional space is best displayed in a way that is meaningful to and interpretable by the user is
not well understood. A three-dimensional structure calls into play issues of three-dimensional graphics (to be discussed below), while spatial dimensionality greater than three will not easily be envisioned. Recent advances in computer graphics displays, such as the Information Visualizer System, the "data wall," and the "cam tree," give innovative solutions to these problems (Card et al., 1990; Clarkson, 1991). The data wall, for example, is a three-dimensional rendering of a wall (actually a perspective building) upon whose surfaces reside visible information nodes. These can be organized according to any two-dimensional structure and examined at any level of distance (or detail) by zooming in and out. The cam tree allows visual representation and examination of a three-dimensional hierarchical database. Newby (1992) has explored issues in navigating through the three-dimensional spatial representation of bibliographical databases.
The issue of the visual representation of large-scale databases brings us to the third form of database: one in which the data are those revealed by scientific inquiry (e.g., scientific experiments, meta-analysis of experimental results, survey data); this defines a set of research issues related to scientific visualization.
The availability of massive amounts of scientific data in domains such as meteorology, geology, demographics, and microbiology, coupled with the power of computer graphics, has enabled the scientist to visualize data in a variety of sophisticated forms (Zorpette, 1989). Scientific visualization both allows and exploits such capabilities as flexibility of representation (e.g., two-dimensional vs. rotating three-dimensional views, various color-coding options), high-dimensionality renderings, dynamic updating, and multiformat representation such as windowing text, graphics, and sound. However, the rapid proliferation of such graphics capabilities has somewhat outstripped the development and application of human factors principles for their appropriate employment.
In fact, a review of the literature reveals very few empirical studies that have evaluated the effectiveness of different forms of data representation for the understanding of complex data sets (Jensen and Anderson, 1987; Liu and Wickens, 1992; Merwin and Wickens, 1991). This paucity of data is, in part, understandable. The advanced display technology is often complex, expensive, and hard to tailor in a way that is consistent with experimental protocol. Furthermore, it is extremely challenging to develop experimental questions that would correspond to the kinds of loosely structured, often exploratory questions that scientists typically ask of their data. The "task analysis" of scientific inquiry is not well understood.
HUMAN FACTORS ISSUES
In the previous section, we described three general domains of information representation, each of which can potentially be served by advanced interactive display technology. We now directly consider the human factors and human performance research issues associated with this technology.
Cognitive Task Analysis
Some research effort must be devoted to carrying out comprehensive task analyses of users interacting with complex databases. This includes users of complex, ill-structured information databases performing tasks of a browsing nature as well as scientists attempting to gain insight from complex data (Langley et al., 1987). The need for cognitive task analysis has direct relevance for the research issues on mental models, automation, and expert systems that are suggested below.
Inquiry must continue on the mental models that users have of both restricted and fluid databases (Carroll and Olson, 1987). This remains a key issue because of our current awareness that the effectiveness of database retrieval depends on how congruent the organization of the database and the navigational mechanism for traversing it are to the user's mental models (Allen, 1991; Seidler and Wickens, 1992). Yet how these models should be assessed, how flexible they are within and between users, and how different mental models should be "averaged" across users all remain issues of considerable uncertainty.
User Models, Automation, and Expert Systems
Distinct from the user's model of the system is the system's model of the user. This knowledge is essential for the effective development and implementation of automation and expert systems that will support information accessibility. The issue cuts across all three domains discussed above. From a computer's point of view, the issue is: "What is the user trying to achieve at this moment and how can I best satisfy those needs?" It is clear that such answers must be based on insight gained from research in the two previous areas.
Flexibility Versus Consistency
Automated systems, as we described above, will attempt to adapt themselves flexibly to suit the momentary needs of the human user. Yet too
much adaptability may well be counterproductive. If a well-intentioned, adaptive change in carrying out a command occurs when the user does not expect (or need) it, considerable harm could result. Yet the research community to date has provided little guidance on the level of sophistication at which intelligent adaptability may stop being beneficial and may actually be harmful. When, in short, is consistency better than flexibility?
In order to grasp the intricacies underlying information retrieval in all three domains, we must continue to develop and assess ways to explore user behavior in contexts reflecting realistic operational constraints. Also acutely needed are meaningful, practical metrics that can be used to consistently evaluate performance within and across systems. These metrics will have to be sensitive to system usability (see Shneiderman, 1987), quality of information, problem context, and resolution.
Greater attention to these methodological issues will allow studies to complement one another, leading, over time, to the development of comprehensive theories of human-computer interaction that can be used to guide effective system design.
The Spatial Metaphor
In each domain, we have argued that there are analog dimensions of relatedness, similarity, and ordering that may often be best served within a spatial framework. This, of course, is definitely true with many aspects of scientific visualization, such as the geosciences, in which Euclidian space is an underlying dimension of the data.
Given also that humans have extensive familiarity navigating and manipulating in space, a strong argument can be made for exploiting the spatial designs of databases and information networks. Yet these design decisions bring with them a number of unresolved issues relating to navigation and lostness (Billingsley, 1982; Gluschko, 1990; Mackinlay et al., 1991). Some examples of research issues are the following:
How should information for very large databases be portrayed, when users may need the flexibility to zoom in to higher levels of detail? What levels of detail and/or abstraction are necessary?
What sort of options should a user have to navigate through an information base? Should these be defined by spatial coordinates, nodes, or key words? Should there be constraints on motion (e.g., only along x, y, and z coordinate axes)? How many user options for navigation should be
available? What are the best control devices for an operator to navigate through this "virtual information space" (Card et al., 1990)?
What are the best ways of conveying information in a dimensionality greater than two? If some representation of three-dimensional space is used, what depth cues are necessary to provide a salient sense of depth for different kinds of databases (Wickens et al., 1989)? Should depth be absolute or ordinal? When should perspective be used to convey information regarding a third dimension and when should color or intensity be employed instead? When should the user be presented with two two-dimensional plans or views? When the dimensionality expands beyond three, the display-formatting issues grow exponentially. Other important research issues relate to the assignment of display dimensions to their referents (i.e., semantic dimensions). Are some assignments better than others? Are some display dimensions best for representing categorical, rather than continuous, semantic dimensions?
The issue of spatial representation is closely tied to the related concept of virtual reality (Eglowstein, 1990; Pausch, 1991), the attempt to render all aspects of database interaction, not just the visual display characteristics, in a three-dimensional spatial mode. These use such features as direct hand position sensing with simulated tactile feedback, spatially localized sound sources, and an "inside-out" perspective. Scores of potential applications have been proposed and, in some instances, demonstrated: from exploratory surgery to architectural design to scientific inquiry to education. Certainly human factors concerns relate to the "level of reality" that best supports performance. It is, for example, well recognized in both the training and the system design community that there are times when more reality does not support better performance; at those times performance is better supported by more abstract displays or discrete controls (Hutchins et al., 1985; Jones et al., 1985; Wickens, 1992). Other human factors issues pertain to fidelity trade-offs that may be imposed by hardware limitations. Can some image resolution be sacrificed to obtain faster speed? Or should the opposite trade-off be preferred (Pausch, 1991)? There is also an intriguing possible trade-off between performance and learning. To what extent do the features of virtual reality that support better performance within a virtual world actually inhibit the learning and long-term retention about the more abstract properties of that world (Wickens, 1992)?
The issue of training overlaps with issues of information access and utilization in at least three respects. First, it is apparent that adequate use of information technology will often depend upon users being trained early in their careers about the value and importance of computer-based information
systems. This lesson has been well learned in the medical profession, in which large numbers of health care professionals, untrained in medical information services, now fail to make use of the advantages those services provide (see Chapter 4).
The second issue relates back to user models. Different levels of user experience may dictate very different structuring needs of information systems (Allen, 1991). Hence, intelligent interfaces should be quite attentive both to the user's level of knowledge about the domain represented in the database and to the user's knowledge of the interface itself. Finally, there is the issue of the use of the interface tool itself to train or educate learner-users about the information it contains. As we have reported elsewhere (Wickens, 1992), there really is very little hard evidence that the extensive automated flexibility of an information database improves users' ability to master the information in it.
Allen, B.L. 1991 Cognitive research in information science: implications for design . Annual Review of Information Science and Technology 26:3-37.
Andre, A.D., and C.D. Wickens 1992 Compatibility and consistency in display-control systems: implications for aircraft decision aid design. Human Factors 34(6): 639-653.
Bates, M.J. 1986 Subject access in online catalogs: a design model. Journal of the American Society for Information Science 37:357-376.
Belkin, N.J., and W.B. Croft 1987 Retrieval techniques. Annual Review of Information Science and Technology 22:109-145.
Bell, D. 1973 The Coming of the Post-Industrial Society. New York: Basic Books.
Billingsley, P.A. 1982 Navigation through hierarchical menu structures: does it help to have a map? Pp. 103-107 in Proceedings of the 26th Annual Meeting of the Human Factors Society. Santa Monica, Calif.: Human Factors Society.
Blair, D.C. 1984 The management of information: basic distinctions. Sloan Management Review 26(1):13-23.
Blair, D.C., and M.E. Maron 1985 An evaluation of retrieval effectiveness for a full-text document retrieval system. Communications of the ACM 28(3):289-299.
Boehm-Davis, D.A., R.W. Holt, and A.C. Schultz 1992 The role of program structure in software maintenance. International Journal of Man-Machine Studies 36:21-63.
Brooks, H.M., P.J. Daniels, and N.J. Belkin 1986 Research on information interaction and intelligent information provision mechanisms. Journal of Information Science: Principles and Practice 12(1):37-44.
Card, S.K. 1982 User perceptual mechanisms in search of computer command menus. Pp. 190-196 in CHI '82 Conference on Human Factors in Computer Science . New York: Association for Computing Machinery.
Card, S.K., J.D. Mackinlay, and G.G. Robertson 1990 The design space of input devices. Pp. 117-124 in CHI Proceedings . New York: Association for Computing Machinery.
Carroll, J.M., and J.R. Olson, eds. 1987 Mental Models in Human-Computer Interaction. Committee on Human Factors, National Research Council. Washington, D.C.: National Academy Press.
Chen, H., and V. Dhar 1991 Cognitive process as a basic for intelligent retrieval systems design. Information Processing and Management 27(5):405-432.
Clarkson, M.A. 1991 An easier interface. BYTE February:277-282.
Croft, W.B., and R.H. Thompson 1987 I3R: a new approach to the design of document retrieval systems. Journal of the American Society for Information Science 36(6):389-404.
Curran, J. 1991 Multidimensional ELS User Interface. SAE Technical Paper Series 912108. Presented at the Aerospace Technology Conference and Exposition, September 23-26, Long Beach, Calif.
Drenth, H., A. Morris, and G. Tseng 1991 Expert systems as information intermediaries. Annual Review of Information Science and Technology 26:113-153.
Durding, B.M., C.A. Becker, and J.D. Gould 1977 Data organization. Human Factors 19:1-14.
Eglowstein, H. 1990 Reach out and touch your data. BYTE July:283-290.
Fisher, D.L., E.J. Yungkurth, and S.M. Moss 1990 Optimal menu hierarchy design: syntax and semantics. Human Factors 32:665-683.
Flanagan, J.C. 1954 The critical incident technique. Psychological Bulletin 51(4):327-358.
Giroux, L., and R. Belleau 1986 What's on the menu? The influence of menu content on the selection process. Behavior and Information Technology 5:169-172.
Gluschko, R.J. 1990 Hypertext: Prospects and Problems for Crew System Design. CSERIAC SOAR-90-22. Wright-Patterson Air Force Base, Ohio: CSERIAC.
Granda, R.E., and R. Halstead-Nussloch 1988 The Perceived Usefulness of Computer Information Sources: A Field Study. IBM Technical Report TR00.3495. Poughkeepsie, N.Y.: IBM.
Harter, S.P. 1986 Online Information Retrieval: Concepts, Principles, and Techniques . Orlando, Fla.: Academic Press.
Harter, S.P., and A.R. Peters 1985 Heuristics for online information retrieval: a typology and preliminary listing. Online Review 9(5):407-424.
Hawkins, F. 1988 Human Factors in Flight. Brookfield, Vt.: Gower.
Hewins, E.T. 1990 Information need and use studies. Annual Review of Information Science and Technology 25:145-172.
Hubbard, S., J.E. Henney, and V.T. DeVita, Jr. 1987 A computer data base for information on cancer treatment. New England Journal of Medicine 316:315-318.
Hutchins, E.L., J.D. Hollan, and D.A. Norman 1985 Direct manipulation interfaces. Human-Computer Interaction 1(4):311-338.
Huth, E.J. 1989 The underused medical literature. Annals of Internal Medicine 110(2):99-100.
Jensen, C.R., and L.A. Anderson 1987 Comparing three dimensional representation of data to scatterplots. Pp. 1174-1178 in Proceedings of the 31st Annual Meeting of the Human Factors Society. Santa Monica, Calif.: Human Factors Society.
Jones, E.R., R.T. Hennessy, and S. Deutch, eds. 1985 Human Factors Aspects of Simulation. Committee on Human Factors, National Research Council. Washington, D.C.: National Academy Press.
Kiger, J.L. 1984 The depth/breadth trade-off in the design of menu-driven user interfaces. International Journal of Man-Machine Studies 20:201-213.
King, D.W., D.D. McDonald, and N.K. Roderer 1981 Scientific Journals in the United States: Their Production, Use and Economics. Stroudsburg, Pa.: Hutchinson Ross.
Koenig, M.E.D. 1990 Information services and downstream productivity. Annual Review of Information Science and Technology 25:55-86.
Langley, P., H.A. Simon, G.L. Bradshaw, and J.M. Zytkow 1987 Scientific Discovery: Computational Explorations of the Creative Processes. Cambridge, Mass.: MIT Press.
Latremouille, S., and E.S. Lee 1981 The design of videotex tree indexes: the use of descriptors and the enhancement of single index pages. Pp. 65-112 in D. Phillips, ed., Telidon Behavioral Research II: The Design of Videotex Tree Indexes. Ottawa, Canada: Department of Communications.
Liu, Y., and C.D. Wickens 1992 Visual scanning with or without spatial uncertainty and divided and selective attention. Acta Psychologica 79:131-153.
Lucky, R.W. 1989 Silicon Dreams: Information, Man and Machine. New York: St. Martin's Press.
MacGregor, D., B. Fischhoff, and L. Blackshaw 1987 Search success and expectations with a computer interface. Information Processing and Management 23(5):419-432.
Mackinlay, J.D., G.G. Robertson, and S.K. Card 1991 The perspective wall: detail and context smoothly integrated. Pp. 173-179 in S.P. Robertson, G.M. Olson, and J.S. Olson, eds., Human Factors in Computing Systems. Reaching Through Technology. CHI '91. New York: Association for Computing Machinery.
Mann, T. 1987 Computer searches. Pp. 80-102 (Chapter 9) in A Guide to Library Research Methods. New York: Oxford University Press.
Merwin, D.H., and C.D. Wickens 1991 Comparison of 2D planar and 3D perspective display formats in multidimensional data visualization. In Proceedings of the International Society for Optical Engineering. Bellingham, Wash.: SPIE.
Miller, D.P. 1981 The depth/breadth tradeoff in hierarchical computer menus. Pp. 296-300 in Proceedings of the 25th Annual Meeting of the Human Factors Society. Santa Monica, Calif.: Human Factors Society.
Morehead, D.R., and W.B. Rouse 1985 Computer-aided searching of bibliographic databases: on line estimation of the value of information. Information Processing and Management 21:387-399.
Muckler, F.A. 1987 The human-computer interface: the past 35 years and the next 35 years. In G. Salvendy, ed., Cognitive Engineering in the Design of Human-Computer Interaction and Expert Systems. Proceedings of the Second International Conference on Human-Computer Interaction, Honolulu, Hawaii. Amsterdam, Netherlands: Elsevier.
Newby, G.B. 1992 An Investigation of the Role of Navigation for Information Retrieval. ASIS Meeting.
Norman, K. 1991 The Psychology of Menu Selection. Hillsdale, N.J.: Erlbaum.
Pausch, R. 1991 Virtual reality on five dollars a day. Pp. 265-269 in Proceedings CHI '91. New York: Association for Computing Machinery.
Pejtersen, A.M. 1989 The Book House: Modeling Users' Needs and Search Strategies as a Basis for System Design. RISO-M-2794. Roskilde, Denmark: Riso National Library, Department of Information Technology.
Pollitt, A.S. 1987 CANSEARCH: an expert systems approach to document retrieval. Information Processing and Management 23(2):119-138.
Polson, M.C., and A. Friedman 1988 Task-sharing within and between hemispheres: a multiple-resources approach . Human Factors 30(5):633-643.
Prasse, M.J., M. Dilon, M.J. Gordon, B. Mortland, and A. Repka 1988 F-TAS: a full-text access system. Pp. 327-332 in M.E. Williams and T.H. Hogan, comps., Proceedings of the 9th National Online Meeting .
Roske-Hofstrand, R.J., and K.R. Paap 1986 Cognitive networks as a guide to menu organization: an application in the automated cockpit. Ergonomics 29(11):1301-1311.
Salvaggio, J.L. 1989 The Information Society: Economic, Social, and Structural Issues . Hillsdale, N.J.: Lawrence Erlbaum.
Seidler, K.S., and C.D. Wickens 1992 Distance and organization in multifunction displays. Human Factors 34:555-569.
Shneiderman, B. 1987 Designing the User Interface: Strategies for Effective Human-Computer Interaction. Reading, Mass.: Addison-Wesley.
Snowberry, K., S.R. Parkinson, and N. Sisson 1983 Computer display menus. Ergonomics 26(7):699-712.
1985 Effects of help fields on navigating through hierarchical menu structures. International Journal of Man-Machine Studies 22:479-491.
Stonier, T. 1983 The Wealth of Information: A Profile of the Post-Industrial Economy. London, England: Methuen London in association with Thames Television International.
Vigil, P. 1985 Computer literacy and the two cultures revisited. Pp. 240-242 in C.A. Parkhurst, ed., Proceedings of the American Society for Information Science (ASIS) 48th Annual Meeting. White Plains, N.Y.: Knowledge Industry Publications, Inc.
Vincente, K.J., and R.C. Williges 1988 Accommodating individual differences in searching a hierarchical file system. International Journal of Man-Machine Studies 29:647-668.
Wickens, C.D. 1992 Engineering Psychology and Human Performance, 2nd ed. New York: Harper Collins.
Wickens, C.D., S. Todd, and K. Seidler 1989 Three-Dimensional Displays: Perception, Implementation, and Applications. University of Illinois Institute of Aviation Technical Report ARL-89-11/CSERIAC-89-1. Also CSERIAC SOAR 89-001, AAMRL, December. Wright-Patterson Air Force Base, Ohio. Savoy, Ill.: Aviation Research Laboratory.
Wiener, E. 1989 Errors and error management in high technology aircraft. In Proceedings of the 7th Meeting on Aeronautic and Space Medicine, June, Paris.
Williams, H.P., M.P. Tham, and C.D. Wickens 1992 Resource Management and Geographic Disorientation in Aviation Incidents: A Review of the ASRS Database. Technical Report ARL-92-3/NASA-92-2. Savoy: University of Illinois, Aviation Research Laboratory.
Wilson, S.R., N. Starr-Schneidkraut, and M.D. Cooper 1989 Use of the Critical Incident Technique to Evaluate the Impact of MEDLINE. Final Report AIR-64600-9/89-FR; NLM/OPE-90/01. Palo Alto, Calif.: American Institutes for Research in the Behavioral Sciences.
Zorpette, G. 1989 The main event. IEEE Spectrum 26(1):28.