Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
(1985) described a learner who was trying to learn a desktop interface and who initially tried to get a piece of paper from a paper pad icon by sweeping the cursor across the icon in a tearing motion. Here the desktop metaphor failed but also served to highlight effectively a specific fact about icon manipulation for the learner. The active learning view provides a means of reconciling the observation that mental models are often chaotic and misconceived and the fact that users do often succeed in learning and using soft- ware. The suggestion is that people develop models that are good enough to suit their current goals. Defective conceptual models may ultimately play useful roles in learning and adequately sup- port some user activity. It is an open question, however, whether they can actually facilitate learning and be used more effectively than more explicitly provided and more correct models (Mack et al., 1983~. RESEARCH RECOMMENDATIONS These observations on the state of research and application of the concept of what the user knows lead to the following research recommendations. 1. Detail what a mental mode! would consist of and how a person would use it to predict a system's behavior. The term mental m-ode! has been- used confusingly in the literature as referring to goal-oriented procedural knowledge, as well as knowledge about the components of the device, their functions, their relations to other components, and their workings. To date there have been no concrete characterizations of what a mental model is and how a person would run it to try out various simulated inputs. One attempt at this specification of a working mental model, a device mode! that is used for guiding external actions, resides in Davis's (1982) expert system for diagnosing electrical circuit failures. This mode! is used by the system to determine where physically a fault might be and, if it were at a particular location, what the device's expected behavior would be. Perhaps Davis's (1982) formalization of an internalized device mode! might serve as a base from which to build specifications of what a mental mode} would be and what mental operations would be necessary in order to use the mental model to make predictions about a system's behavior. 29
Yet, specifications of how a person would use a mental mode} to predict what a system will do is not sufficient to predict the user's behavior. Our understanding of mental models (if they ex- ist) needs to be embedded in a mode} of a full-blown cognitive system, one that has problem-solving and decision-making pro- cesses that are sufficient to initiate the mode! runs, collect the results, and decide on an external action. 2. Investigate whether people have and use mental modem of various kinds. Probably the most basic question in this area, still far from being answered, is whether people construct and use mental models at all. And, because of confusion of terminology in the literature, behavioral evidence is not clearly supportive. Even when we confine ourselves to the specific definition of mental mod- els used in this report, however, there is little evidence that people have and use mental models. So far, the majority of evidence for mental models has come from people's self-reports that they form and use them (which may be post-hoc rationalizations), and from some evidence that when taught a system mode! or analogy, per- formance is sometimes better and learning may be faster. Specific research is needed to demonstrate whether people have models and that their behavior is clearly distinguishable from that produced by having stored sequence/method representations. 3. Determine the behaviors that would demonstrate the model's form and the operations used on it. If a person has a mental model, there may be some observable behavior that would give an analyst evidence of its form. Traditionally, experimental psychologists have made inferences about the existence of mental events by carefully constructing test situations with systematically varied features and observing particular overt responses such as the time that it takes to make a certain judgment or carry out an action, or the amount and kinds of errors made. The construction of the appropriate comparative test situations and the inferences that can be drawn from the responses, times, and errors must be based on a clearer notion of the form that the mode} might have and the processes that may act on it. If the analyst can predict behavior knowing that the person has a mental model of a particular sort, then the analyst should be able to discover the mental models of other people from systematic examination of their behavior. Multidimensional scaling (Shepard et al., 1972), unfolding theory (Coombs, 1964), and ordered tree 30
analysis (Reitman and Rueter, 1980) are examples of techniques that allow the analyst to infer particular mental representations from behavior. Perhaps aspects of behavior can reveal the form of a working mental model. This work could follow from a program of research that built on the theoretical work outlined above. 4. Explore alternative views of sequence/method representa- tions and the behavior predicted from them. We currently have a better conception of what it means to have sequence/method representations and what processes may act on them to produce behavior than we do of mental models. GOMS represents the structure of goals, methods, and actions in a mental hierarchy for well-learned cognitive tasks. Kieras and Polson's (1985) pro- duction system formalism and its inference engine (a standard set of procedures for keeping track of where one is in a process and choosing the subsequent actions) is a concrete specification of this kind of knowledge and the processes that act on it. From that for- malism follow concrete predictions of behavior, such as particular responses (key presses), their times, and the errors. A body of em- pirical data is growing, answering questions about which aspects of the representation affect behavior. What is needed is more research in this vein. Formalisms of knowledge and operational mechanisms would be specified and the behavior of other kinds of sequence/method representations would be predicted. Empirical studies could then be formed to answer specific questions about the adequacy of the formalism, in detail, replacing the vague generalizations and contradictions that seem to plague research in this area today. 5. Explore the types of mental representations that may ex- ist that are not mechanistic. Most of the mental models that are conceived in this research are mechanistic in nature. The se- quence/method representations are mechanistic and serial. These consist of components and processes that mimic physical devices. There may be mental representations of other types, however, that drive people's exploratory and explanatory behavior. People claim to make inferences and explorations from stored visual and auditory images; mathematicians experiment mentally with com- putational systems, making inferences before showing any exter- nal behavior; people likely reason at different levels of abstraction about a system, making inferences of a very general nature in -planning before exploring details in a step-by-step fashion. There 31
may be visual, auditory, computational, or hierarchical systems that form helpful bases for people's reasoning. These other possi- ble types of mental representations should be made concrete, and their behavioral correlates should be explored. 6. Determine how people intermix different representations in producing behavior. This report has reviewed a variety of types of knowledge that may be held by a user of a computer system. It is likely that users have some knowledge stored in several of these representations: some well-known procedures for executing simple sequences; some well-formed GOMS-like structures for do- ing familiar but more complicated tasks; and some mental models that help the user explore alternative actions to take when an error occurs or when a novel task is presented to them. If all of these rep- resentations exist simultaneously, then we need to know when each is used and how the person moves between them and/or combines their operations or products. There is likely to be some problem- solving or decision-making apparatus that guides the overall task behavior, sometimes exploring unknown territory with a process like means-encis analysis or running a mental model, and other times executing well-learned actions from stored goal structures (see, for example, the extensive literature on automaticity; Shiffrin and Schneider, 1977~. An integrated performance view is called for. 7. Explore how knowledge about systems is acquire&. If we can discover~the form of the representation of knowledge that people have about computer systems, we would like aLso to know how that information was acquired. I`ewis's work (1986) on how people make inferences about a system from watching its behavior is a good example of how to specify concretely how people learn complicated tasks on computers. Work is also needed on how people acquire mental models, simple sequences, and methods. This work would have an unpact not only on the design of systems and their training, but also would give some basic knowledge about the problem of learning complex behavior in general. 8. Determine how individual differences have an impact on [earning of and performance on systems. Individuals' cognitive capacities differ, making different computer users more and less capable. Some of these differences are likely to arise from simply having more knowledge from longer exposure to the system. Expo- sure could provide a user with more task knowledge as well as more 32
specific and more accurate mental models. Some of the differences in performance, however, may arise from basic individual differ- ences in abilities. For example, Gomez et al. (1983) have shown that people who are not good at visual memory have difficulty with some word processors. further, they found that a system that required less recall of a command syntax reduced the perfor- mance differences found between those who could recall locations and those who could not. We need to know more about individ- ual cognitive differences and their concomitant effect on people's mental representations of and performance on complex tasks. The results will have implications for both the design of systems and the construction of training sequences for a particular system for particular users. 9. Explore the design of training sequences for systems. A related training issue surrounds the idea of "training wheels," the notion that a scaled-down system is easier to learn initially. Specifying and analyzing the mental mode! or sequence/method representations implied by the scaled-down system may lead de- signers to build more coherent systems and more effective training sequences. further, this analysis may indicate how information about the fuld system should be taught as an add-on to the train- ing wheels system. 10. Provide system designers with took to help them develop interfaces that invoke good representations in users. There is proba- bly some guidance that can be provided to systems designers while they design the user interface to ensure that the sequence/method representation or the mental mode! will be an effective guide to accurate performance. Such tools may come in the form of user in- terface management systems; which constrain the design set. The goal may be to constrain the ways that the designers can display things or constrain the ways that they can allow the user to invoke a command so that a coherent, easily understood set is formed, one that invokes in the user a good mental mode! or a coherent set of goal-actions pairs. Designing these guidance tools is an im- portant research topic, one that can aid the transfer of technology from the laboratory to the design and development arena. 11. Expand the task domain to more complex software. Most of the research in the area of mental models and sequence/method representations for human-computer interaction has focused on text processing and simple device models. Whatever results 33