National Academies Press: OpenBook

(NAS Colloquium) Neuroimaging of Human Brain Function (1998)

Chapter: A neural system for human visual working memory

« Previous: Components of verbal working memory: Evidence from neuroimaging
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

This paper was presented at a colloquium entitled “Neuroimaging of Human Brain Function,” organized by Michael Posner and Marcus E.Raichle, held May 29–31, 1997, sponsored by the National Academy of Sciences at the Arnold and Mabel Beckman Center in Irvine, CA.

A neural system for human visual working memory

LESLIE G.UNGERLEIDER*, SUSAN M.COURTNEY, AND JAMES V.HAXBY

Laboratory of Brain and Cognition, National Institute of Mental Health, Building 10, Room 4C104, 10 Center Drive, Bethesda, MD 20892–1366

ABSTRACT Working memory is the process of actively maintaining a representation of information for a brief period of time so that it is available for use. In monkeys, visual working memory involves the concerted activity of a distributed neural system, including posterior areas in visual cortex and anterior areas in prefrontal cortex. Within visual cortex, ventral stream areas are selectively involved in object vision, whereas dorsal stream areas are selectively involved in spatial vision. This domain specificity appears to extend forward into prefrontal cortex, with ventrolateral areas involved mainly in working memory for objects and dorsolateral areas involved mainly in working memory for spatial locations. The organization of this distributed neural system for working memory in monkeys appears to be conserved in humans, though some differences between the two species exist. In humans, as compared with monkeys, areas specialized for object vision in the ventral stream have a more inferior location in temporal cortex, whereas areas specialized for spatial vision in the dorsal stream have a more superior location in parietal cortex. Displacement of both sets of visual areas away from the posterior perisylvian cortex may be related to the emergence of language over the course of brain evolution. Whereas areas specialized for object working memory in humans and monkeys are similarly located in ventrolateral prefrontal cortex, those specialized for spatial working memory occupy a more superior and posterior location within dorsal prefrontal cortex in humans than in monkeys. As in posterior cortex, this displacement in frontal cortex also may be related to the emergence of new areas to serve distinctively human cognitive abilities.

In the past several years, there has been an explosive growth in the field of human brain imaging. Much of this work has demonstrated that information processing in the human brain involves the concerted activity of multiple, spatially distributed cortical regions. Major goals for those engaged in human brain mapping of cognitive function include identifying the regions that participate in various information processing operations, characterizing the functional role played by each region, and modeling the interactions among regions.

Functional brain imaging measures local hemodynamic changes, and these changes are used as indices of changes in neural activity (14). Because measurements typically are taken simultaneously in all regions of the brain, functional brain imaging is ideally suited for investigating the concerted activity among the ensemble of regions that participate in a specific cognitive function. Vision and visual memory have been especially fruitful domains for studies of distributed neural systems in the human brain. We share these functions with nonhuman primates, and there exists a rich literature on the neuroanatomy and neurophysiology of the monkey visual system. Thus, specific hypotheses and predictions can be derived from the animal literature for imaging studies of vision and visual memory in humans. In this review, we first will summarize the organization of visual processing pathways in the macaque monkey brain, and the further extensions of these pathways into prefrontal cortex for visual working memory. Next, we will examine the organization of human pathways for visual processing and working memory and describe the ways this organization resembles and differs from that in the monkey. Finally, we will present evidence that human and monkey prefrontal cortex play similar physiological roles in the maintenance of items in working memory.

Visual Processing Pathways in the Monkey

The monkey cortex contains at least 30 separate visual areas, occupying about one-half of the total cortex (5, 6). These areas are organized into two functionally specialized processing pathways, each having the primary visual cortex (V1) as its source and each being composed of multiple areas beyond V1 (Fig. 1). The occipitotemporal pathway, or “ventral stream,” is crucial for the identification of objects, whereas the occipitoparietal pathway, or “dorsal stream,” is crucial for spatial perception (7, 8) and for the visual guidance of movements toward objects in space (9, 10). A simple way to conceptualize the functions of the two processing streams is “what” vs. “where.”

The original evidence for separate processing streams for object vision and spatial vision was the contrasting behavioral effects of inferior temporal and posterior parietal lesions in monkeys (for review, see ref. 7). Lesions of inferior temporal cortex cause severe deficits in performance on a wide variety of visual discrimination tasks (e.g., pattern, object, and color), but not on visuospatial tasks (e.g., visually guided reaching and judging which of two identical objects is located closer to a visual landmark). By contrast, posterior parietal lesions do not affect visual discrimination performance, but instead cause severe deficits in visuospatial performance. Physiological evidence supports this distinction. Cells in areas within the ventral stream (V1, V2, V4, and inferior temporal areas TEO and TE) respond to visual features relevant to object identification, such as color, texture, and shape, whereas cells in areas within the dorsal stream (V1, V2, V3, middle temporal area, medial superior temporal area, and further stations in the inferior parietal and superior temporal sulcal cortex) respond selectively to spatial aspects of stimuli, such as direction of motion and velocity, as well as to tracking eye movements (for reviews, see refs. 5 and 11).

0027–8424/98/95883–8$0.00/0

PNAS is available online at http://www.pnas.org.

   

Abbreviations: BA, Brodmann area; fMRI, functional magnetic resonance imaging; PET, positron emission tomography; rCBF, regional cerebral blood flow; V1, primary visual cortex.

*  

To whom reprint requests should be addressed. e-mail: lgu@ln.nimh.nih.gov.

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

FIG. 1. Visual processing pathways in monkeys. Areas in the dorsal stream, having primarily visuospatial functions, are shown in green, and areas in the ventral stream, having primary object recognition functions, are shown in red. Lines connecting the areas indicate known anatomical connections, with heavy arrowheads indicating feed-forward connections from lower-order areas to higher-order ones and open arrowheads indicating feedback connections from higher-order areas to lower-order ones. Solid lines indicate connections from both central and peripheral visual field representations, and dotted lines indicate connections restricted to peripheral field representations. Shaded region on lateral view of the brain indicates extent of cortex included in the diagram. (From ref. 85; for further details of the visual areas, see ref. 86.)

Areas within both processing pathways appear to be organized hierarchically, in the sense that low-level inputs are transformed into progressively more integrated representations through successive stages of processing. Within the ventral stream, for example, the processing of object features begins with simple spatial filtering by cells in V1, but by the time the inferior temporal cortex (area TE) is activated, the cells respond to global object features, such as shape, and some cells are even specialized for the analysis of faces (5). Likewise, within the dorsal stream, the processing of moving stimuli begins with simple direction-of-motion selectivity by V1 cells, but in the higher-order areas of the parietal cortex (such as the lateral intraparietal and medial superior temporal areas) the cells respond selectively to complex patterns of motion, such as rotation, and to the optic flow patterns produced when one moves through the environment (1214).

Thus, much of the neural mechanism for both object vision and spatial vision can be viewed as a “bottom-up” process subserved by feed-forward projections within a pathway. Each of these feed-forward projections, however, is reciprocated by a feedback, or reentrant, projection (6, 15). Projections from higher-order processing stations to lower-order ones could mediate some “top-down” aspects of vision, such as the effects of selective attention on perceptual processing.

Monkey Prefrontal Cortex and Working Memory

Both processing streams, dorsal and ventral, have reciprocal connections with regions beyond the modality-specific visual system. One such set of connections includes those with the prefrontal cortex. Anatomical studies have shown that projections from areas in the dorsal stream terminate mainly in and around the principal sulcus [Brodmann area (BA) 46] of dorsolateral prefrontal cortex (1618), whereas projections from areas in the ventral stream terminate mainly on the inferior convexity (BA 12 and 45) in ventrolateral prefrontal cortex (1921). Thus, from an anatomical point of view, the domain specificity of spatial vs. object vision that is present in

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

the posterior processing areas of the monkey brain appears to extend forward into the prefrontal cortex.

There is abundant neurophysiological evidence that prefrontal cortex plays an important role in visual working memory (22, 23), the ability to actively maintain and use an item of information for only a short period of time before it is discarded. The concept of working memory, as originally proposed by Baddeley and Hitch (24), embraces several inter-related ideas, including the covert rehearsal of verbal material, a visuospatial “sketch pad” for visual and spatial information, and a “central executive” for attentional control.

In monkeys, working memory typically has been studied in either delayed response or delayed match-to-sample tasks. In both types of tasks, the monkey is given a brief cue at the start of the trial, which it must maintain in memory during a delay of several seconds. At the end of the delay, the monkey is required to make a choice or differential response based on the previous cue. Many studies have found cells whose response to the initial cue is sustained at some level during the delay period (for reviews, see refs. 23 and 25). Thus, the memory of the cue appears to endure by maintaining the activity of the cells that represent the cue. Depending on the type of cue, cells with such delay activity have been found in the inferior temporal cortex (visual pattern or color cues), the posterior parietal cortex (spatial cues), and the prefrontal cortex (all types of cues).

Wilson et al. (26) have shown that the dorsal prefrontal areas that are reciprocally connected with dorsal stream visual areas exhibit sustained delay activity that is primarily related to spatial information, whereas the ventral prefrontal areas that are reciprocally connected with ventral stream visual areas exhibit sustained delay activity that is primarily related to

FIG. 2. Location and response properties of the monkey prefrontal cortex. (A) Monkey prefrontal cortex is defined as the region extending forward from the fundus of the rostral bank of the arcuate sulcus to the frontal pole. Cortex within the principal sulcus is BA 46, whereas the cortex below it on the inferior prefrontal convexity (shaded region) is BA 12. (B) Responses of a single inferior convexity neuron with pattern-specific working memory activity. (Upper) Delay period activity specific to a pattern on pattern delayed response trials. (Lower) Lack of response on spatial delayed response trials. (C) Responses of a single neuron in the principal sulcus with spatial-specific working memory activity. (Upper) Lack of response on pattern delayed response trials. (Lower) Delay activity specific to right spatial cues on spatial delayed response trials. [Reproduced with permission from ref. 26 (Copyright 1993, The American Association for the Advancement of Science.)]

pattern, color, object, and face information (Fig. 2). Based on these physiological results, it seems reasonable to conclude that the visuospatial sketch pad in the working memory scheme of Baddeley and Hitch (24) can be divided at the neural level into two domains of processing within prefrontal cortex, a dorsolateral region for spatial-based information and a ventrolateral region for object-based information.

The preponderance of behavioral data in monkeys with cortical excisions bears out this distinction. Lesions of the cortex in the principal sulcus produce severe impairments on spatial memory tasks, such as delayed response and delayed alternation (2729). By contrast, lesions of lateral prefrontal cortex that are ventral to the principal sulcus impair performance on nonspatial memory tasks, such as delayed match-to-sample and object alternation (30, 31).

On the basis of anatomical, physiological, and behavioral evidence in monkeys, Goldman-Rakic (32, 33) has conceptualized the organizing principle for the lateral prefrontal cortex in terms of “domain specificity.” According to this model, working memory processes within the lateral prefrontal cortex are mediated by separate regions according to the type of information being processed. Dorsolateral prefrontal regions would be principally involved in working memory for spatial information, whereas ventrolateral regions would be principally involved in working memory for nonspatial material (e.g., object-related information).

Although most experiments in monkeys have supported the notion that working memory in prefrontal cortex is organized according to domain specificity, results from two recent studies have raised some questions. In a behavioral study, Rushworth et al. (34) found that monkeys with ventrolateral prefrontal lesions (including BA 12 and the anterior portion of BA 45), though profoundly impaired in relearning a simultaneous color-matching task, were nonetheless unimpaired when subsequent delays were introduced. This finding contrasts with previous findings in monkeys with lesions of the principal sulcus, in which there is no impairment unless the task has both delay (30, 35, 36) and spatial (30, 31) components. On this basis, the authors concluded that ventrolateral prefrontal cortex may be more important for stimulus selection than for working memory. However, because the lesions in the study by Rushworth et al. (34) did not include the posterior portion of BA 45, namely, the portion within the anterior bank of the descending limb of the arcuate sulcus, they spared part of the prefrontal projection zone of the inferior temporal cortex (21). Thus, the absence of a delay effect could be attributed to the fact that the lesions were incomplete.

A second recent study that does not appear to support the organizing principle of domain specificity is a physiological study by Rao et al. (37). This study did not question the idea that ventrolateral prefrontal cortex is involved in working memory, but rather suggested that object and spatial domains of processing within prefrontal cortex are not strictly segregated. Recordings made in monkeys during a task that required both “what” and “where” information revealed that more than half of the neurons with delay activity showed both what and where tuning. There are two possible explanations for these findings that are not incompatible with the notion of domain specificity. First, it may be that different parts of lateral prefrontal cortex emphasize different domains of processing, but the segregation of processing between domains is not absolute. Alternatively, it may be that prefrontal cortex is highly modifiable, such that behavioral training on a task requiring the integration of what and where information alters the properties of the neurons within the region.

Visual Processing Pathways in Humans

The differential visual impairments produced by focal lesions in clinical cases suggest that human visual cortex, like that of

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

the monkey, may be similarly organized into two anatomically distinct and functionally specialized ventral and dorsal processing pathways. The specific clinical syndromes produced by occipitotemporal lesions include visual object agnosia, prosopagnosia, and achromatopsia (ref. 38; for reviews, see refs. 39 and 40), whereas those produced by occipitoparietal lesions include optic ataxia, visuospatial neglect, constructional apraxia, gaze apraxia, akinotopsia, and disorders of spatial cognition (refs. 4143, for review, see ref. 44).

Functional brain imaging makes it possible to map the organization of the human cortex with far greater precision than is possible with human lesion studies, and without the confounding influence of compensatory responses to brain injury. In experiments designed to investigate the possible existence of separate object vision and spatial vision pathways in humans, we measured changes in regional cerebral blood flow (rCBF) by using positron emission tomography (PET) while subjects performed object identity and spatial location match-to-sample tasks (45). In the object identity task, the subject indicated which of two choice faces matched the sample face, whereas in the spatial location task, the subject indicated which of two choice stimuli contained a small square in the same location, relative to a double line, as the small square in the sample stimulus. Because these small squares contained faces, identical stimuli were used for both face and location matching, with only the task requirements changing.

Regions that were activated during performance of these tasks were identified by comparing rCBF during face or location matching to rCBF during the performance of a sensorimotor control task. As shown in Fig. 3, early visual areas in occipital cortex were activated during both tasks. The face matching task, but not the location matching task, also activated the ventral occipitotemporal cortex, centered on the fusiform gyrus, i.e., areas of the ventral stream. By contrast, the location matching task, but not the face matching task, also activated the dorsal occipital, superior parietal, and intraparietal sulcal cortex, i.e., areas of the dorsal stream. These results confirmed the hypothesis that human visual cortex is organized into two functionally specialized and anatomically segregated processing pathways. Moreover, insofar as early visual areas showed less functional specialization compared with later areas, the results support the view that visual cortex in humans, like that in monkeys, is hierarchically organized.

The locations of the occipitotemporal and occipitoparietal foci activated by face matching and spatial location matching, respectively, are in good agreement with other PET studies of face perception and visuospatial processing (46, 47). In addition, we have replicated the face matching results in a functional magnetic resonance imaging (fMRI) study using the same task and stimuli (48). Unlike PET studies, in which the results of a group of subjects typically are averaged to increase signal-to-noise ratios and sensitivity, with fMRI the regions of activation can be delineated in individual subjects. The cortical areas in the ventral stream activated by faces and identified with fMRI were consistently the same regions on the fusiform gyrus and within the occipitotemporal sulcus that had been identified by the group average PET results, but they were of smaller spatial extent, a finding supported by other fMRI studies of face perception (4951). These results indicate that, because of anatomical variability across subjects, group averaging in PET tends to overestimate the size of an activated region.

In the posterior cortex of the human brain, the locations of the ventral and dorsal streams for object and spatial vision, respectively, are similar to their locations in the monkey. However, some differences are apparent. The ventral stream in the human brain, unlike that in the monkey, is located almost entirely on the ventral surface of the temporal lobe; in addition, it does not extend as far anteriorly in the human compared with the monkey. The dorsal stream in the human brain has a more superior location in parietal cortex than in the monkey. Such differences suggest that these regions of cortex were displaced in the course of human brain evolution. The inferior displacement of ventral stream areas and the superior

FIG. 3. The dorsal and ventral visual processing streams in human cortex, as demonstrated in a PET study of location and face perception. Areas shown in green had significantly increased rCBF during the location matching but not during the face matching task, as compared with rCBF during a sensorimotor control task. Areas shown in red had significantly increased rCBF during the face matching but not during the location matching task, as compared with rCBF during the control task. Areas shown in yellow had significantly increased rCBF during both face and location matching tasks. Maximum location intensities are shown on the lateral views of the left and right hemispheres. Coronal sections are taken at the anterior-posterior levels indicated on the lateral views. (Adapted from ref. 45.)

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

displacement of dorsal stream areas both may be related to the emergence of phylogenetically newer areas specialized for language in the posterior perisylvian cortex. Similarly, the posterior displacement of ventral stream areas away from the temporal pole may be related to the role the latter cortex plays in semantic and lexical knowledge about objects (52, 53).

The results of our PET imaging study not only revealed the locations of object and spatial vision pathways in the human, but also suggested their further extensions into the frontal lobe. Face matching activated a region in the right inferior frontal cortex, whereas location matching activated a superior frontal region. Although this latter region initially appeared to be centered in premotor cortex, our results and those of others (47, 5458) suggest that activity within the region is not dependent on motor planning or motor preparedness. Using fMRI, we have localized this region to the depth of the superior frontal sulcus (59), and, as we shall see below, its functional properties are indeed characteristic of prefrontal rather than premotor cortex.

Localizing Visual Working Memory Systems in Humans

Finding separate prefrontal areas associated with object and spatial matching tasks, respectively, led us to ask whether the activations were caused by the working memory components of the matching tasks. To test explicitly whether these separate prefrontal regions play a predominant role in maintaining an active representation of object or spatial information “on line” after the relevant stimuli are removed from view, we designed a PET study of visual working memory in which the subjects had to retain either the identity or location of three faces presented sequentially in three different locations on the screen (56). As before, both the spatial location and face identity tasks used identical stimuli. Immediately after the presentation of the third face, a test stimulus was shown. The test stimulus was a face in a variable location. For the face working memory task, the subject indicated whether that face

FIG. 4. Selective activation of the dorsal and ventral visual processing pathways and of dorsal and ventral prefrontal areas, as demonstrated in a PET study of spatial location and face working memory. Areas shown in blue and green had significantly greater rCBF during the spatial, as compared with the face, working memory task. Areas shown in yellow and red had significantly greater rCBF during the face, as compared with the spatial, working memory task. The two working memory tasks used identical stimuli. (Adapted from ref. 56.)

was in the memory set, regardless of the location in which it was initially presented. For the location working memory task, the subject indicated whether the location of the test stimulus was one of the locations in the memory set, regardless of the identities of the faces used to mark those locations.

Comparison of rCBF during performance of the two tasks again revealed differences between the ventral and dorsal pathways (Fig. 4). The face working memory task differentially activated a large ventral occipitotemporal region, including the right hippocampus, which had not been differentially activated in the previous perceptual study of faces. The face working memory task also led to large activations in the right inferior and mid prefrontal cortex (BA 45/47 and 46/9). This region of activation overlapped with, but was more extensive than, the inferior prefrontal region that had been activated by faces in the perception study. The location working memory task differentially activated a large dorsal occipitoparietal region, which extended anteriorly to include, bilaterally, the same frontal area, located in the superior frontal sulcus, that had been activated in the perception study of spatial location. The results thus demonstrated that different types of working memory differentially activate separate prefrontal regions. Working memory for objects (for example, for faces) activates predominantly inferior prefrontal cortex, whereas working memory for spatial locations (for example, the spatial locations of faces) activates predominantly superior prefrontal cortex. These results therefore provide evidence that human prefrontal cortex, like that of the monkey, is functionally organized for working memory according to domain specificity, supporting the model of Goldman-Rakic (32, 33).

Smith, Jonides, and colleagues (57, 58) also have directly contrasted object vs. spatial working memory, and, as we did, they found evidence for domain specificity in prefrontal cortex. However, their evidence indicated that domain specificity is primarily a hemispheric laterality effect, with a left mid-frontal region specialized for object working memory and right inferior and superior frontal regions specialized for

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

spatial working memory. In our studies of face and spatial working memory, we found activations with similar coordinates, but the activations tended to be bilateral (5961). Smith and Jonides (57) attributed the left lateralization of object working memory to rehearsal of a symbolically or linguistically encoded representation of the object. We have seen left lateralization for face working memory under conditions that encouraged more symbolic or verbal encoding of the faces (60), but we also have seen right lateralization under conditions that allowed for more image-based encoding (56, 60). We would argue that laterality effects in visual memory may be influenced by a variety of factors, such as memory set size, retention interval length, and item familiarity, all of which may affect the extent to which subjects engage in symbolic or verbal encoding and rehearsal (see below).

Owen (62, 63) has argued that human prefrontal areas are not organized according to domain specificity but rather according to “processing specificity,” that is, the nature of processing required. According to this model, put forth by Petrides (64, 65), ventrolateral prefrontal regions are concerned with the organization of sequences of behavioral responses based on retrieval of information from posterior processing areas, whereas dorsolateral regions are recruited only when executive control is required, that is, when it is necessary to actively monitor and manipulate information within working memory. Owen’s main argument against domain specificity is that spatial, object, and verbal working memory tasks all appear to activate BA 46/9 in the mid-dorsolateral prefrontal cortex (e.g., refs. 55 and 6668) as well as BA 45/47 in inferior (ventrolateral) prefrontal cortex (e.g., refs. 5456, 58, and 66), the same general regions associated with, respectively, spatial and object working memory in monkeys. However, this kind of meta-analysis of the existing literature lumps together studies in which the comparison task varies widely. Thus, it fails to distinguish between activations of different strengths in prefrontal regions. The demonstration of a functional dissociation between working memory domains can be made only by contrasting, in a single study, activations during carefully matched working memory tasks, as we have done (56).

It is also important to note that Owen (62, 63) assumed, based on results in the monkey, that any segregation between spatial and object working would be between BA 46/9 and 45/47. However, as we described above, and as others also have shown, the performance of spatial working memory tasks activates, not only BA 46/9, but also a frontal area in and near the superior frontal sulcus (54, 55, 57, 58, 69). Activations in this area generally have been dismissed because of their presumed location within premotor cortex or the frontal eye field (refs. 54, 55, 66, and 68; but see ref. 56). However, we have demonstrated with fMRI that the region activated by spatial working memory lies just anterior to the human frontal eye field (59). Moreover, activations in this region during spatial working memory tasks are not related to either skeletomotor or oculomotor components of the task, as these variables were controlled in comparison tasks. Therefore, the evidence to support the notion of domain specificity in human prefrontal cortex comes from the superior frontal sulcus playing a dominant role in spatial working memory rather than BA 46/9 (see ref. 70).

To summarize, we find that areas specialized for object working memory in humans and monkeys are located within similar portions of ventrolateral prefrontal cortex. By contrast, areas specialized for spatial working memory differ in their locations between humans and monkeys, with the human area being displaced superiorly and posteriorly compared with that in the monkey. However, as in the monkey, in which the area specialized for spatial working memory is located just anterior to the frontal eye field, the same topological relationship exists in the human. The difference between monkey and human in the exact location of these dorsal frontal areas suggests that they were displaced by the expansion of more inferior and anterior portions of the prefrontal cortex. This expansion maybe related to the emergence of distinctively human cognitive abilities. Examples of these abilities include abstract reasoning, complex problem solving, and planning for the future, consistent with neuropsychological descriptions of patients with frontal lobe damage (71).

Following the Dynamics of Working Memory in Human Cortex

The time scale of working memory is typically on the order of seconds. It is therefore difficult with PET, which integrates activity over 1–2 min, to distinguish between activity related to perception and that related to memory in a working memory task. Our first approach to this problem was to design a task in which the delay between the presentation of a face to hold in working memory and the test stimulus was systematically varied (60). We predicted that different delay intervals would have differential effects on areas principally involved in the perceptual aspects of the task compared with areas principally involved in the mnemonic aspects. With shorter delay intervals, the subjects would spend a greater proportion of time viewing pictures of faces, whereas with longer delay intervals, they would spend a smaller proportion of time viewing faces but a greater proportion of time holding a face in memory. Therefore, we predicted that areas principally involved in perceptual analysis would show smaller increases in rCBF with increasing delays, whereas areas principally involved in working memory would show relatively constant levels of increased rCBF as a function of the delay interval. The latter prediction was based on the fact that, in the monkey, areas that participate in working memory contain neurons that respond both to stimulus presentation and during memory delays, and, therefore, should be active throughout the face memory task, regardless of delay length.

We found that regions showing a significant negative correlation between rCBF increases and delay were restricted to the ventral occipitotemporal cortex, in areas essentially identical to the ones identified in studies of face perception. Thus, these areas are mainly active during the presentation of stimuli, not during memory delays. They are, therefore, likely to be more involved in the perceptual analysis of faces during the task rather than in the working memory component per se. By contrast, areas showing a more constant rCBF increase across all delays were in the prefrontal cortex, indicating that they are active during memory delays as well as during stimulus presentation. Thus, prefrontal regions likely play a more important role in the maintenance of a representation during working memory. Interestingly, right prefrontal activity tended to diminish at longer delays, whereas left prefrontal activity showed the largest activity increases at longer delays. One possible explanation for this hemispheric difference is that right hemisphere activity is associated with a simple, icon-like image of a face that is difficult to maintain in working memory, whereas left hemisphere activity is associated with a face representation that is more symbolic, and, possibly, verbal. More recently, we have taken advantage of the temporal resolution afforded by fMRI to reinvestigate the functional roles played by visual and prefrontal areas in face working memory (61). Multiple regression analysts enabled us to decompose activations in a face working memory task into three functions, namely, a nonselective, transient response to visual stimuli, a selective, transient response to faces, and sustained activity over memory delays (Fig. 5). Posteriorly, in ventral occipital areas, we found mostly a transient, nonselective response to visual stimuli, which did not differ between faces and nonface control stimuli. More anteriorly, in ventral temporal regions, we found a more selective response to faces

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

compared with nonfaces, and also a small, but a significant, sustained response over memory delay intervals. Finally, three distinct prefrontal regions were identified that all showed greater levels of sustained activity over memory delays. Moreover, the relative contributions of perception-related activity and memory-related activity differed significantly for these three regions, suggesting they play different, functionally specialized roles in working memory. These results are a direct demonstration of memory-related sustained activity in human prefrontal cortex (see also ref. 72).

FIG. 5. Design and results of an fMRI study of working memory for faces (61). (Upper) Design of the task. For each series of fMRI scans, subjects performed 3 1/2 baseline-activation task cycles, each consisting of two sensorimotor control trials followed by two working memory trials. During the memory task, subjects saw a picture of a face, a delay, and then another picture of a face. Subjects were asked to hold an image of the first face in mind during the delay and to respond with a left or right button press to indicate whether the second face matched the first. During the control task, subjects simply looked at the scrambled pictures and then pressed both buttons when the second scrambled picture appeared. Three time series are shown that represent the different cognitive components of the task: a transient, nonselective response to visual stimuli; a transient, selective response to faces; and sustained activity during memory delays. These time series (smoothed and delayed by convolution with a model of the hemodynamic response) were used as regressors in a multiple regression analysis of the time course of activation in each area. (Lower) Results from a single subject overlaid onto that subject’s anatomical images. Activations are color-coded according to the relative sizes of the three regression coefficients described above. Areas that responded transiently and nonselectively to any visual stimulus, such as posterior occipital cortex (a), are shown in green. Areas that responded transiently and showed a selective response to faces over scrambled faces, such as fusiform gyrus (b), are shown in blue. Areas that showed sustained activation during the memory delay after the stimulus was removed from view, such as inferior frontal cortex (c), are shown in red. Areas that showed a combination of these types of responses are shown in a blend of colors. (From ref. 87.)

Conclusion

Our fMRI results have demonstrated the existence of areas in human cortex with response properties remarkably similar to those described in single-cell recordings in monkeys: (i) early visual areas in occipital cortex with relatively nonselective responses to complex or meaningful stimuli (5); (ii) later visual areas in temporal cortex with selective responses to meaningful stimuli, such as faces (73, 74); and (iii) areas in both temporal (75, 76) and prefrontal (25, 26, 37) cortices with sustained activity during memory delays. The degree of sustained activity in monkey prefrontal cortex during the delay period is typically greater than that observed in the inferior temporal cortex, and, unlike the activity in the temporal cortex, the prefrontal activity is not disrupted when the monkey processes other visual inputs during the delay period (7679). These results suggest that prefrontal cells may be the major originator of the delay activity and may activate perceptual representations in posterior visual areas during the delay via feedback projections to those areas. The idea that sustained activity in posterior visual areas reflect top-down influences from prefrontal cortex is supported by the results of deactivation studies. Fuster et al. (80) have found that delay activity for object information in the inferior temporal cortex, though not eliminated, becomes markedly less selective during reversible deactivation of prefrontal cortex by cooling. Similarly, Goldman-Rakic and Chafee (81) have found that delay activity for spatial information in the posterior parietal cortex is greatly diminished during prefrontal deactivation. Although these kinds of invasive studies are not possible in the human brain, it is possible in neuroimaging experiments to reveal modulatory influences from feedback projections by mathematical modeling of the data (82, 83). Mathematical modeling clearly adds a new dimension to the inferences one can make about functional interactions among cortical areas subserving specific cognitive operations, such as those engaged in working memory tasks (e.g., see ref. 84).

We thank Robert Desimone for critical comments on the manuscript and Christine Rey-Hipolito for help in manuscript preparation.

1. Fox, P.T. & Raichle, M.E. (1986) Proc. Natl. Acad. Sci. USA 83, 1140–1144.

2. Bandettini, P.A., Wong, E.C., Hinks, R.S., Tikofsky, R.S. & Hyde, J.S. (1992) Magn. Reson. Med. 25, 390–397.

3. Kwong, K.K., Belliveau, J.W., Chesler, D.A., Goldberg, I.E., Weisskoff, R.M., Poncelet, B.P., Kennedy, D.N., Hoppel, B.E., Cohen, M.S. & Turner, R. (1992) Proc. Natl. Acad. Sci. USA 89, 5675–5679.

4. Ogawa, S., Tank, D.W., Menon, R., Ellermann, J.M., Kim, S.G., Merkle, H. & Ugurbil, K. (1992) Proc. Natl. Acad. Sci. USA 89, 5951–5955.

5. Desimone, R. & Ungerleider, L.G. (1989) in Handbook of Neuropsychology, eds. Boller, F. & Grafman, J. (Elsevier, Amsterdam), Vol. 2, pp. 267–300.

6. Felleman, D.J. & Van Essen, D.C. (1991) Cereb. Cortex. 1, 1–47.

7. Ungerleider, L.G. & Mishkin, M. (1982) in Analysis of Visual Behavior, eds. Ingle, D.J., Goodale, M.A. & Mansfield, R.J.W. (MIT Press, Cambridge, MA), pp. 549–586.

8. Ungerleider, L.G. & Haxby, J.V. (1994) Curr. Opin. Neurobiol. 4, 157–165.

9. Goodale, M.A. & Milner, A.D. (1992) Trends Neurosci. 15, 20–25.

10. Jeannerod, M. & Rossetti, Y. (1993) Bailliere Clin. Neurol. 2, 439–460.

11. Maunsell, J.H. & Newsome, W.T. (1987) Annu. Rev. Neurosci. 10, 363–401.

12. Tanaka, K. & Saito, H. (1989) J. Neurophysiol. 62, 626–641.

13. Andersen, R.A., Bracewell, R.M., Banish, S., Gnadt, J.W. & Fogassi, L. (1990) J. Neurosci. 10, 1176–1196.

14. Duffy, C.J. & Wurtz, R.H. (1991) J. Neurophysiol. 65, 1329–1345.

15. Zeki, S. (1993) A Vision of the Brain (Blackwell Scientific, Oxford).

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×

16. Barbas, H. & Mesulam, M.M. (1985) Neuroscience 15, 619–637.

17. Ungerleider, L.G. & Desimone, R. (1986) J. Comp. Neurol. 248, 190–222.

18. Cavada, C. & Goldman-Rakic, P.S. (1989) J. Comp. Neurol. 287, 422–445.

19. Chavis, D.A. & Pandya, D.N. (1976) Brain Res. 117, 369–386.

20. Ungerleider, L.G., Gaffan, D. & Pelak, V.S. (1989) Exp. Brain Res. 76, 473–484.

21. Webster, M.J., Bachevalier, J. & Ungerleider, L.G. (1994) Cereb. Cortex 4, 470–483.

22. Fuster, J.M. (1990) in Progress in Brain Research, eds. Uylings, H.B.M., Eden, C.G.P., Bruin, J.P.C.D., Corner, M.A. & Feenstra, M.G.P. (Elsevier, Amsterdam), pp. 313–323.

23. Goldman-Rakic, P.S. (1990) in Progress in Brain Research, eds. Uylings, H.B.M., Eden, C.G.P., Bruin, J.P.C.D., Corner, M.A. & Feenstra, M.G.P. (Elsevier, Amsterdam), pp. 325–336.

24. Baddeley, A.D. & Hitch, G.J. (1974) in The Psychology of Learning and Motivation, ed. Bower, G. (Academic, San Diego), pp. 47–90.

25. Fuster, J.M. (1995) Memory in the Cerebral Cortex: An Empirical Approach to Neural Networks in the Human and Nonhuman Primate (MIT Press, Cambridge, MA).

26. Wilson, F.A., O’Scalaidhe, S.P. & Goldman-Rakic, P.S. (1993) Science 260, 1955–1958.

27. Mishkin, M. (1957) J. Neurophysiol. 220, 615–622.

28. Butters, N. & Pandya, D. (1969) Science 165, 1271–1273.

29. Passingham, R.E. (1985) Behav. Neurosci. 99, 3–21.

30. Passingham, R. (1975) Brain Res. 92, 89–102.

31. Mishkin, M. & Manning, F.J. (1978) Brain Res. 143, 313–323.

32. Goldmain-Rakic, P.S. (1994) in Motor and Cognitive Functions of the Prefrontal Cortex, eds. Theiry, A.M., Glowinski, J., Goldman-Rakic, P.S. & Christen, Y. (Springer, Berlin), pp. 112–122.

33. Goldman-Rakic, P.S. (1995) Ann. N.Y. Acad. Sci. 769, 71–83.

34. Rushworth, M.F., Nixon, P.D., Eacott, M.J. & Passingham, R.E. (1997) J. Neurosci. 17, 4829–4838.

35. Goldman, P.S. & Rosvold, H.E. (1970) Exp. Neurol. 27, 291–304.

36. Funahashi, S., Bruce, C.J. & Goldman-Rakic, P.S. (1993) J. Neurosci. 13, 1479–1497.

37. Rao, S.C., Rainer, G. & Miller, E.K. (1997) Science 276, 821–824.

38. Newcombe, F., Ratcliff, G. & Damasio, H. (1987) Neuropsychologia 25, 149–161.

39. Damasio, A.R., Tranel, D. & Damasio, H. (1989) in Handbook of Neuropsychology, eds. Boller, F. & Grafman, J. (Elsevier, Amsterdam). Vol. 2, pp. 317–332.

40. Farah, M.J. (1990) Visual Agnosia (MIT Press, Cambridge, MA).

41. Eglin, M., Robertson, L.C. & Knight, R.T. (1989) J. Cognit. Neurosci. 1, 372–385.

42. Vaina, L.M., Lemay, M., Bienfang, D.C., Choi, A.Y. & Nakayama, K. (1990) Visual Neurosci. 5, 353–369.

43. Zihl, J., von Cramon, D., Mai, N. & Schmid, C. (1991) Brain 114, 2235–2252.

44. Newcombe, F. & Ratcliff, G. (1989) in Handbook of Neuropsychology, eds. Boller, F. & Grafman, J. (Elsevier, Amsterdam), Vol. 2, pp. 333–356.

45. Haxby, J.V., Horwitz, B., Ungerleider, L.G., Maisog, J.M., Pietrini, P. & Grady, C.L. (1994) J. Neurosci. 14, 6336–6353.

46. Sergent, J., Ohta, S. & MacDonald, B. (1992) Brain 115, 15–36.

47. Corbetta, M., Miezin, F.M., Shulman, G.L. & Petersen, S.E. (1993) J. Neurosci. 13, 1202–1226.

48. Clark, V.P., Keil, K., Maisog, J.M., Courtney, S.M., Ungerleider, L.G. & Haxby, J.V. (1996) Neuroimage 4, 1–15.

49. Puce, A., Allison, T., Gore, J.C. & McCarthy, G. (1995) J. Neurophysiol. 74, 1192–1199.

50. Puce, A., Allison, T., Asgari, M., Gore, J.C. & McCarthy, G. (1996) J. Neurosci. 16, 5205–5215.

51. Kanwisher, N., McDermott, J. & Chun, M.M. (1997) J. Neurosci. 17, 4302–4311.

52. Damasio, H., Grabowski, T.J., Tranel, D., Hichwa, R.D. & Damasio, A.R. (1996) Nature (London) 380, 499–505.

53. Nobre, A.C., Allison, T. & McCarthy, G. (1994) Nature (London) 372, 260–263.

54. Jonides, J., Smith, E.E., Koeppe, R.A., Awh, E., Minoshima, S. & Mintun, M.A. (1993) Nature (London) 363, 623–625.

55. Owen, A.M., Evans, A.C. & Petrides, M. (1996) Cereb. Cortex 6, 31–38.

56. Courtney, S.M., Ungerleider, L.G., Keil, K. & Haxby, J.V. (1996) Cereb. Cortex 6, 39–49.

57. Smith, E. & Jonides, J. (1995) in The Cognitive Neurosciences, ed. Gazzaniga, M. (MIT Press, Cambridge), pp. 1009–1020.

58. Smith, E.E., Jonides, J.J., Koeppe, R.A., Awh, E., Schumacher, E.H. & Minoshima, S. (1995) J. Cognit. Neurosci. 7, 337–356.

59. Courtney, S.M., Maisog, J.M., Ungerleider, L.G. & Haxby, J.V. (1997) Neuroimage 5, S607 (abstr.).

60. Haxby, J.V., Ungerleider, L.G., Horwitz, B., Rapoport, S.I. & Grady, C.L. (1995) Hum. Brain Mapp. 3, 68–82.

61. Courtney, S.M., Ungerleider, L.G., Keil, K. & Haxby, J.V. (1997) Nature (London) 386, 608–611.

62. Owen, A.M. (1997) Trends Cognit. Neurosci. 4, 123–125.

63. Owen, A.M. (1997) Eur. J. Neurosci. 9, 1329–1339.

64. Petrides, M. (1994) in Handbook of Neuropsychology, eds. Boller, F. & Grafman, J. (Elsevier, Amsterdam), Vol. 9, pp. 59–82.

65. Petrides, M. (1995) Ann. N.Y. Acad. Sci. 769, 85–96.

66. Fiez, J.A., Raife, E.A., Balota, D.A., Schwarz, J.P., Raichle, M.E. & Petersen, S.E. (1996) J. Neurosci. 16, 808–822.

67. McCarthy, G., Puce, A., Constable, R.T., Krystal, J.H., Gore, J.C. & Goldman-Rakic, P.S. (1996) Cereb. Cortex 6, 600–611.

68. Smith, E.E., Jonides, J.J. & Koeppe, R.A. (1996) Cereb. Cortex 6, 11–20.

69. Courtney, S.M., Ungerleider, L.G., Keil, K. & Haxby, J.V. (1995) Soc. Neurosci. Abstr. 21, 1498.

70. Courtney, S.M., Ungerleider, L.G. & Haxby, J.V. (1997) Trends Cognit. Neurosci. 1, 125–126.

71. Stuss, D.T. & Benson, D.F. (1986) The Frontal Lobes (Raven, New York).

72. Cohen, J.D., Peristein, W.M., Braver, T.S., Nystrom, L.E., Noll, D.C., Jonides, J. & Smith, E.E. (1997) Nature (London) 386, 604–608.

73. Perrett, D.I., Rolls, E.T. & Caan, W. (1982) Exp. Brain Res. 47, 329–342.

74. Desimone, R., Albright, T.D., Gross, C.G. & Bruce, C. (1984) J. Neurosci. 4, 2051–2062.

75. Miyashita, Y. (1988) Nature (London) 335, 817–820.

76. Miller, E.K., Li, L. & Desimone, R. (1993) J. Neurosci. 13, 1460–1478.

77. Chelazzi, L., Miller, E.K., Duncan, J. & Desimone, R. (1993) Nature (London) 363, 345–347.

78. Miller, E.K., Erickson, C.A. & Desimone, R. (1996) J. Neurosci. 16, 5154–5167.

79. Desimone, R. (1996) Proc. Natl. Acad. Sci. USA 93, 13494–13499.

80. Fuster, J.M., Bauer, R.H. & Jervey, J.P. (1985) Brain Res. 330, 299–307.

81. Goldman-Rakic, P.S. & Chafee, M. (1994) Soc. Neurosci. Abstr. 20, 808.

82. Friston, K.J., Ungerleider, L.G., Jezzard, P. & Turner, R. (1995) Hum. Brain Mapp. 2, 211–224.

83. Büchel, C. & Friston, K.J. (1997) Cereb. Cortex 7, 768–778.

84. McIntosh, A.R., Grady, C.L., Haxby, J.V., Ungerleider, L.G. & Horwitz, B. (1996) Cereb. Cortex 6, 571–584.

85. Ungerleider, L.G. (1995) Science 270, 769–775.

86. Distler, C., Boussaoud, D., Desimone, R. & Ungerleider, L.G. (1993) J. Comp. Neurol. 334, 125–150.

87. Courtney, S.M. & Ungerleider, L.G. (1997) Curr. Opin. Neurobiol. 7, 554–561.

Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 121
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 122
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 123
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 124
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 125
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 126
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 127
Suggested Citation:"A neural system for human visual working memory." National Academy of Sciences. 1998. (NAS Colloquium) Neuroimaging of Human Brain Function. Washington, DC: The National Academies Press. doi: 10.17226/6236.
×
Page 128
Next: Functional neuroimaging studies of encoding, priming, and explicit memory retrieval »
(NAS Colloquium) Neuroimaging of Human Brain Function Get This Book
×
Buy Paperback | $70.00 Buy Ebook | $54.99
MyNAP members save 10% online.
Login or Register to save!

The colloquium on "Imaging of Cognitive Function" speaks to the many audiences whose interests relate to efforts to map cognitive processes in the human brain. There are things of great interest in this collection of papers for specialists in cognition and neuroscience and imaging science as well as in disciplines interested in human development through education and training and others with intrinsic interest in the latest information on how the human brain supports thought. The papers were presented at a meeting sponsored by the National Academy of Sciences in its western home the Beckman Center at the University of California, Irvine.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!