National Academies Press: OpenBook

Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science (1990)

Chapter: Information Processing in the Primate Visual System

« Previous: Introduction
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 5
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 6
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 7
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 8
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 9
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 10
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 11
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 12
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 13
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 14
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 15
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 16
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 17
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 18
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 19
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 20
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 21
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 22
Suggested Citation:"Information Processing in the Primate Visual System." National Research Council. 1990. Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/9557.
×
Page 23

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Information Processing in the Primate Visual System DAVID C. VAN ESSEN It is useful at the outset to recall how impressive our visual system is in analyzing and integrating information. A simple example can be found in Figure IN This is a black and white image that starts out as millions of bits of information about gray-level intensities that are encoded within the retina. What happens, literally in a split second, is that the brain processes that information and yields a rich and vind set of perceptions. In particular, we perceive a human face. Moreover, by automatically comparing this face with the enormous number of images viewed over a lifetime, we recognize it as a unique individual- Albert Einstein. What is equally impressive is that a vastly degraded and simplified image (Figure IB) is also immediately recognized as Einstein. This is some- thing that no computer vision system as yet can come even remotely close to achieving not because computers inherently lack the computational power to process the image, but because they have not been programmed with the right strategies for processing this information. The issue for this symposium, then, is the specific strategies used by the visual system to carry out such elegant analyses of the immense variety of images that we confront during normal vision. My remarks concentrate on the macaque monkey. Monkeys have been chosen because their sense of vision is very similar to that of the human in a variety of important and basic ways. RETINAL PROCESSING Images formed on the retina are picked up by approximately 100 million photoreceptors in both macaque monkeys and humans. About 95 percent of those are rods, which are used for night vision. In daytime vision, about 5 million cones feed information onto approximately 1 million 5

6 DAVID C. VAN ESSEN FIGURE 1A A photograph of a unique individual—Albert Einstein.

INFORMATION PROCESSING IN THE PRIMATE VISUAL SYSTEM 7 retinal ganglion cells. Thus, there is already a great deal of convergence within the retina. That convergence is handled very elegantly by putting a high density of cones and retinal ganglion cells to subserve the fovea. In the center of the fovea there are 10,000 or more ganglion cells subserving each square degree of the visual field. Out in the periphery of the retina, however, the image is analyzed at a much coarser level—on the order of just a handful of ganglion cells per square degree. Another important point about the output from the retina is that retinal ganglion cells can be subdivided into major cell classes. In primates, the subdivision is a basic dichotomy between approximately 10 percent of the cells that are very large neurons with large dendritic trees, termed the magnocellular population. These are distinct from the majority (90 percent) of the ganglion cells, which are of the smaller and termed the parvocellular subtype. It is the parvocellular neurons that send high-acuity information, in- cluding information about color. Parvocellular neurons are generally asso- ciated with sustained responses to illumination. The magnocellular system, by contrast, carries little chromatic information and responds transiently to illumination. These two pathways head out from the retina and relay through separate layers of the lateral geniculate nucleus (LGN). The magnocellular neurons of the retina terminate in the ventral-most pair of layers, while the majority of paIvocellular neurons terminate in the uppermost layers. This dichotomy is continued in the relay up to primary visual cortex—the striate cortex. VISUAL CORTEX It has been known for more than a century that the primary visual area, also known as striate cortex, V1, and area 17, is easily distinguishable from neighboring areas by its characteristic structure. It receives the direct inputs from the LGN, and it contains a very precise and orderly representation of the opposite half of the visual field. In addition, there is a much larger belt of tissue, the extrastriate visual cortex, much of which is buried in one or another of the assorted folds of the cortex. At one time, it was thought that the visual cortex occupied only the occipital lobe. From studies done in a number of laboratories, however, it is now clear that visual cortex extends well down into the temporal lobe (the inferior temporal region) and well up into the posterior parietal region. So, altogether, more than half of the macaque's cerebral cortex is largely or exclusively visual in function. How is this belt of tissue organized? Classical anatomists historically emphasized that there was just a small number of subdivisions associated

8 'a /~ - ~ ~~ ~~! i/ DAVID C. VAN ESSEN , if , ~i' fit - 1 - l t\ (~ ~ ~ is! _,< \ ~` \ 4'~ ! 'an' . \ \ FIGURE 1B A line drawing that is much reduced in overall information content but is still easily recognizable as Einstein. Drawing by R.A. Eatock. \

INFORMATION PROCESSING IN THE PRIMATE VISUAL SYSTEM P,P 7~J ~ r _ / 1 As I ~ IT Y \ LIT ~ / 9 a/ FIGURE 2 Visual areas in the cerebral cortex of the macaque monkey. The location of different areas are indicated on a drawing of the right hemisphere (upper left) and on an unfolded Dimensional cortical map (center). Areas that have been particularly well-studied are shown in stippled: area V1, V2, V4, the middle temporal area (Ml), the inferotemporal complex (IT), and the posterior panetal complex (PP). Source: Van Essen and Anderson (1990) with visual processing in the cortex. It is now clear this was a vast over- simplification. There are not merely a few, but rather a large number of distinct visual areas that have been identified in various laboratories over the past two decades (Van Essen, 1985~. In a two-dimensional unfolded map of the cerebral cortex in the right hemisphere of a macaque monkey, one can see more than two dozen different visual areas occupying the entire posterior (left) half of the hemisphere. For simplicity, only a few of these areas (the stippled ones) have been labeled in Figure 2. Many of these are well-defined areas that nearly all laboratories agree on, although not everyone uses exactly the same terminology. Although substantial number of regions are less well-defined, there is reasonable evidence that they represent distinct subdivisions.

10 DAVID C VAN ESSEN Delineating different areas of the cortex has turned out to be a major undertaking. It is difficult because the criteria for identifying these different subdivisions vary considerably. In general, visual areas in the cerebral cortex have been identified by a combination of criteria, the most important of which is that each area has a distinct pattern of inputs from other cortical areas and outputs to other target areas. Most of these areas have maps of the contralateral half of the visual field, which is accordingly represented over and over again. The maps are themselves an important part of the identification process. There is a lot of individual variability, however, in the detailed organization of these areas from one animal to the next. Although this is of interest in its own right, it also contributes to the difficulty of working out the arrangement of different areas. HIERARCHICAL PROCESSING Anatomical connections can also be used, to assess the way in which information flows from one place to another in a cortex. Studies in a number of laboratories have shown that connections within the visual cortex are nearly always reciprocal in nature. If area A projects to area B. then there is a reciprocal connection from B back to ~ In the great majority of such cases, these connections are asymmetrical. Several research groups have noted this asymmetry and have suggested that direction could be associated with forward or ascending information flow, whereas flow In the opposite direction is viewed as feedback. Based on these anatomical criteria, John Maunsell and I (Van Essen and Maunsell, 1983) suggested several years ago that the overall collection of visual areas could be grouped together into an anatomically based hierarchical scheme that starts with area V1 and then goes through a half dozen separate layers until one gets to higher-level processing centers in the temporal, frontal, and parietal lobes. An important question is how high into cortical processing centers this scheme remains valid. There is now enough evidence to trace the succession of processing centers all the way from the retina up through visual areas in the occipital, temporal, and parietal lobes, all the way up and out of visual cortex proper in fact, out of neocortex and into the hippocampus. In our current version Dan Felleman and I have proposed a dozen stages of hierarchical processing in the cortex, plus an additional pair of stages represented by the retina and LGN. This, then, gives us a sense of the degree to which information goes through successive hierarchical levels, and also of the rich degree of parallelism and reciprocity in terms of multiple outputs from any one area to targets at both higher and lower levels. Thus, we can think of the visual system as being divided into a large number of discrete, higher,

INFORMATION PROCESSING IN THE PRIMATE VISUAL SYSTEM 11 interconnected modules: individual cortical areas containing anywhere from just a few million cells in small areas like MT to hundreds of millions of cells in areas like V1. PROCESSING STREAMS Another aspect of modularity comes from looking in more detail at the way in which some visual areas, in particular areas V1 and V2, can be broken up into discrete compartments. (V1 is the largest of all visual areas; V2 is nearly as large and adjoins V1.) The cortex, when sliced parallel to the cortical surface and stained with the mitochondrial enzyme cytochrome oxidase, reveals a distinctive pattern. This pattern was first discovered for V1 by Margaret Wong-Riley (Carroll and Wong-Riley, 1984) and for V2 by Roger Motels and colleagues (Tootell et al., 1983~. Within V1, there is a remarkable arrangement of little patches or so-called blobs, particularly in the superficial layers of the cortex, that stain densely for cytochrome oxidase (Livingstone and Hubel, 1984~. These patches are separated by so-called interblobs which stain less densely for the same enzyme. At the border between V1 and V2, a rather different configuration is evident orthogonal to the boundary between the areas. The darkly stained regions have a different connectional pattern than the more lightly stained region, and both of these differ from a third compartment of pale interstripes, all within area V2. Not only are there distinct patterns of connections between V1 and V2 associated with this compartmental organization, but there are also distinctive connectional patterns between the compartments in V2 and higher-order targets in particular, areas MT and V4 (Figure 3~. Evidence for these connections comes from experiments done in England (Shipp and Zeki, 1985) and in my laboratory (DeYoe and Van Essen, 1985, 1988~. Experiments we have conducted show that when fluorescent tracers were injected into the target areas MT and V4, we were able to identify cells projecting to MT located primarily in the thick stripes, with a much lower number and some cells in the thin stripes as well. We also identified cells projecting off to area V4 that are concentrated in both the thin stripe and the interstripe regions. There appears to be a dichotomy in the retinal and geniculate or- ganization. The magnocellular and parvocellular subdivisions project to separate portions of area V1. The magnocellular projects to its own layer, area layer 4C, which in turn projects to layer 4B. That is one discrete com- partment. Then the pa~vocellular system, perhaps including an additional subset of so-called interlaminer cells in the LGN, projects indirectly to the blobs and the interblobs associated with the superficial layers. This tripartite

12 V2 V 1 I Blob UGN r: RGC ~ ~ DAVID G HAN ESSEN Temporal i-- I T ---LI I |-- P P ~ Parietal , ~ ~ , 1 . ~ ~ I T I V4 ~ AT r . . Thin Inter- Thick stripe stripe stripe Lam = _ ~ l bloc: T M I G Occipital Cortex FIGURE 3 Processing streams and hierarchical organization of the primate visual system. At the two lowest levels, retinal ganglion cells (RGC) and lateral geniculate nucleus (LGN), there is a dichotomy between the small pa~vocellular (P) cells and the large magnocellular (M) cells. In areas V1 and V2 there is a tripartite compartmentalization. Layer 4B of V1 and the thick stripes of V2 are dominated By magnocellular inputs, and they project most strongly to area MT and from there to the posterior parietal complex. The blobs and interblobs of V1 and the thin stripes and interstripes of V2 are dominated by pawocellular inputs, and information flows from them to area V4 and then to the inferotemporal complex. All lines except that from the retina to the LGN represent reciprocal connections. Note that there is significant cross-talk between processing streams at several levels of the hierarchy. Source: Van Essen and Anderson (1990~. arrangement is preserved by the projections to the tripartite scheme in area V2: interblobs to interstripes, blobs to thin stripes, and layer 4B to thick stripes. There is further segregation of these streams with the thick stripes as well as layer 4B projecting both to area MT and V4 receiving inputs not just from one stream but from both of these together.

INFORMATION PROCESSING IN THE PRIMATE V75UAL SYSTEM NEURONAL RESPONSE PROPERTIES 13 What are the cells in these different compartments actually doing terms of processing information? The standard approach inspired by the work of Hubel and Wiesel is to use simple stimuli such as bars and edges of light, and to ask what turns on a cell in any given area. Figure 4 shows an example of a cell from area VP, which happened to be highly selective for stimulus wavelength. The cell responds to long wavelengths, i.e., red, not at all to short wavelengths, and not at all to white. There is now quite a rich catalog of information of the basic selectivity of cells in the visual pathway, based on studies in many laboratories. We know, for example, that color selectivity is very common in V4, but is very rare in MT. Interestingly, the same distinction applies to the different compartments of area V2. Those compartments projecting off to MT have very low color selectivity, whereas both of the thin and interstripe compartments that project off to V4 are rich in color selectivity by our criteria. The opposite is evident when looking at direction selectivity. Here there is a high incidence of direction selectivity in area MT and very low in area V4. A similar bias occurs in that there is a very low incidence of direction selectivity in the V2 compartments projecting to V4 and a somewhat higher incidence in the thick stripes that project off to MT. But it is not a perfect match, in that the percentage of direction selective cells in the thick stripes of V2 is not nearly as high as the actual percentage for MT. One has to wonder what is going on in this compartment other than a simple analysis of stimulus direction. Using this kind of information, again gleaned from a number of different studies, Ted DeYoe put together an illustration to give a qualitative impression of the kinds of information processing represented within the different channels that we have seen. This is illustrated in Figure 5 with a set of icons representing different types of selectivity (prism = wavelength selectivity; spectacles = binocular disparity selectivity; pointing hand = direction selectivity; and angle = orientation selectivity); these are placed in areas and compartments in which a high incidence at such selectivity is encountered. In the magnocellular stream, for example, projecting through V1 and V2 into MT, there is a substantial incidence of direction selectivity, suggesting an involvement with motion analysis. There is also information about stimulus orientation and binocular disparity represented at all these levels. So it is not just a single kind of selectivity. Multiple cues appear to be analyzed within this stream. Within the parvocellular stream, the compartments associated with the blobs and the thin stripes are dominated by an analysis of wavelength; one suspects that it is involved in the analysis of stimulus color. The compartments the one associated with the interblobs contain a combination

14 DAVID C. VAN ESSEN Response (impuIses/sec) 30 - 20 10 S pontaneous activity ; blue green yellow orange red white ,....1,- ..,, 1. ,n,..1.~..4 . BLUE I. . .1 1. -..1 .1 Aid .. -..IL, t-", RED 1.~t ... L. t.1a ~ ....1, 1.111. ORANGE I ..~11._1l_ 1.,.. ....` ...,~.L W H I T E 1 200 I impulses/ sec sec FIGURE 4 Selectivity of a cell in area VP to stimuli of different wavelengths. The cell responds well to long wavelengths (red) but not to shorter wavelengths or to white light. Source: ~ Burkhalter and D. Van Essen, unpublished. Of color-selective cells and orientation selective cells, many cells showing selectivity along both dimensions. This raises a question of whether there is a real difference in the way in which color (wavelength) information is used in these two streams. We really do not know the answer, but it seems likely that color information

INFORMATION PROCESSING IN THE PLATE SUM SYSTEM Infero tempera Areas U4 At UU! Th i n . SItnr~'epre '1 .' 11 _ - Blob LON In for Blob ~ | Parvo 15 Par fetal Areas 1 11 ~ r I :---------T (I MT ~ l Am. U3 Th icE 11 ALL 4B in o FIGURE 5 Representation of various response selectivities in different areas and compart- ments in the visual hierarchy. Icons are placed in each compartment to symbolize a high incidence of cells showing selectivity for stimulus wavelength (prisms), orientation (angle symbols), direction (pointing hands), or binocular disparity (spectacles). Each processing stream has a distinctive physiological profile, but most types of selectivity are representative in more than one stream. Source: Adapted from DeYoe and Van Essen (1988~.

16 DAVID C. VAN ESSEN might sometimes be used lo encode the presence of interesting features, such as chromatic borders. Once the information about the presence of a border has been generated, however, information about the colors used to define the border might be discarded higher in the system. Thus, the presence of wavelength selectivity per se does not in and of itself imply that it is part of the stream explicitly involved in color analysis. We need to be aware of those subtleties in order to fathom the relationship between the properties of individual neurons and the functions of the circuits in which they are involved (see DeYoe and Van Essen, 1988~. As one traces the visual pathway through fairly high levels, it is striking that not only can one activate these higher-level cells with rather simple stimuli bars and edges—but also the sharpness of tuning that one sees is not dramatically different at these high levels than it is in V-1 (the first level at which such information becomes explicit at the single neuron level). That raises a fundamental question about what these higher-order areas are actually doing. Neurons in these areas are not simply relaying information; they must also be processing the information in an interesting way. We suspect that the answer does not lie in the way in which simple stimuli are analyzed. More complicated stimuli are needed to understand the role of these cells. The type of stimulus complexity that is needed should be linked to the tasks of the visual system in mediating perception. We should use psychophysics, then, as a guide for studying higher-level processing. A good example is an experiment inspired by the perception of texture. It is notable that we are able to get the percept very quickly that we are looking at a photo of a rocky beach (Figure 6~. We don't need to scrutinize individual rocks, one after another and sort of assemble them sequentially into the percept of a beach. It is the texture of that pattern that we process very quickly to get the impression not only that this is a particular rocky beach, but that its surface is receding in depth. How do we begin to analyze the visual processing of texture? Psy- chophysical experiments (e.g., Julesz, 1984; Beck, 1976) have made consid- erable progress on this issue in significant part by reducing the problem to one in which textures are defined using a set of distinct texture elements, or sextons. Using their work as a general guide, we have made recordings from areas V-1 and V-2 using texture patterns of a rather simple type. Using computer-generated texture patterns we have done the following kind of experiment (Figure 7~. By recording from a cell in area V-2, we found that the cell preferred near-vertical stimuli when tested with a single texture element. When this element was surrounded by a texture pattern of the same element orientation (a uniform texture field), the response of the cell was almost completely suppressed. When the surround elements were of the orthogonal orientation (orientation contrast), the responses were still

INFORMATION PROCESSING IN THE PRIMATE VISUAL SYSTEM FIGURE 6 A natural scene that is rich in textural information.

18 `,, 40 Y 30 0 - a) ~ O 20 10 Response Histograms ........ ` L l~., ~ ~ ~ _ ~1 DAVID C VAN ESSEN + / \ / \ \ A. I I .. L I 1, _ d' "4 ' - a`'/.` ~ C C=S CIS S C C=S C'S S" Stimulus Configurations FIGURE 7 Responses of a neuron in are a V1 to orientation contrast. The cell responds well to a near-vertical bar presented within the classical receptive field (C) and to the same bar when it is surrounded by a texture pattern containing bars of orthogonal orientation (C = S). Responses are suppressed when the bar is part of a uniform texture field (C = S). Source: Van Essen et al. (1989~. quite vigorous. Thus, the responses of this particular cell correlate well with the perceptual salience of the central texture element. COMPUTATIONAL APPROACHES I would like now to address the role of computational approaches to vision in understanding basic aspects of visual processing. It is popular these days to consider the possibility that computational approaches will provide strong insights for understanding visual processing, but the field is still in its infancy, especially in terms of making a correlation between abstract computational theories and actual physiological processes occurring within the brain. Although computational models are very interesting in their own

INFORMATION PROCESSING IN THE PLATE SUM SYSTEM 19 right, it is hard to see exactly what their implications are at the-level of single neurons in visual cortex. One example of a computational strategy that we suspect may have firm roots in the underlying anatomy and physiology involves the understanding of depth perception by stereopsis, an idea generated by Charles Anderson, now working at the Jet Propulsion Laboratory in Pasadena. The standard concept of stereopsis begins with the eyes fixating on a given point. That point is imaged on the center of the fovea in the right and left eyes. Images lying on the horopter, or fixation plane, fall on precisely corresponding portions of the two retinas. Images in depth relative to the horopter, however, fall on disparate or noncorresponding portions of the two retinas. Our sense of stereoacuity is exquisitely good. We can pick up disparities on the order of a few seconds of arc a fraction of a photoreceptor diameter. Although that is certainly impressive by any standard, it is even more striking when one takes into account the fact that our ability to fixate an object and hold it on the center of the two foveae is really not all that great. Psychophysical observations have demonstrated that there is actually a fair amount of jitter in the precise localization of the image and that the jitter is not concordant in the two eyes. From moment to moment, there are fluctuations on the order of at least a few minutes of arc, and up to 10 or 20 minutes of arc during a time when we can see a well-defined spot that appears to be stable in depth. The implication is that our stereoacuity is a couple of orders of magnitude sharper than the binocular vergence errors that are part of our normal visual processing. How do we do this? This is a serious computational problem for the visual system to cope with, yet we obviously succeed. We suggest that one needs some kind of dynamic shifting or remapping process between the retina and the first stage of binocular integration in the cortex (Anderson and Van Essen, 1987~. This is termed the reg~straiion problem in stereopsis. Imagine the left and right eyes are looking at very similar patterns except that they are physically offset. This peak falls on disparate parts of the two retinas because of the vergence errors (Figure 8~. What we need is a process in which the projection from the retinal ganglion cells up to the cortex can be adjusted independently for the right and left eyes, and adjusted by sufficient magnitude that there would be alignment of the image representations. If that could be achieved in the manner illustrated schematically in the figure, then the registration problem could be cleverly solved by the visual system. Figure 9A shows a type of circuit that could do this job. In this succession of relay stages from the retinal ganglion cells up through several successive layers, connections ascend, but they branch a little bit at the first level, a little more at the second, even more at the highest level. As they

20 BINOCULAR REGISTRATION INDEX l r SHIFT ~ DAVID C VAN ESSEN BINOCULAR I NTEGRDT I ON REGISTRATION O O O O O O O O O O O ~ O O O O O On R STAGE ~ O O O O O O O O O O O e 0 0 0 0 0 0 0 0 ~ L / / ~ ~ / I CONTROL | ~ \ ~ \ ROC LUMINANCE _/~-^-~-` PATTERNS LEFT EYE RIGHT EYE FIGURE 8 Schematic diagram of how a dynamic shifting process could provide a binocular registration at the cortical level (V1) despite misregistration of luminance patterns for the two eyes. Note that the sharp luminance peak, which activates noncorresponding RGCs (hatched circles) maps onto corresponding cells at the registration stage. Source: Anderson and Van Essen (1987). branch, one would like only one set of branches active at any one moment. So if, for example, the right side of the pathway were active, information would go up and get shifted to the right, and then get shifted to the left, register, and shift back to the right. If one had control over which branches were being affected, one could get a dynamic remapping of the cells at the beginning onto any set of contiguous cells up at highest level. We propose, in the slightly simpler set of layers and cells shown in Figure 9B, that one could achieve this by means of a set of inhibitory neurons that could actually shunt or veto the signals going through one set of dendrites. The other inhibitory neuron would be silent, thereby letting the pathway going to the other set of dendrites send its signal through. The activity of these inhibitory neurons could determine whether information goes to the right or the left at each successive level. Is there any shred of evidence that something like that might be go- ing on in the visual system? To know, one would need several successive seemingly simple relay stages, inhibitory neurons, and some kind of feed- back mechanism. All those features are in fact present in area V1, but their significance has heretofore been puzzling. The inputs from the lateral geniculate nucleus have been known for a long time to terminate within the layer four complex, in which there is an enormous increase in the number of cells available. Yet these cells have been described as simple relay stages without having orientation selectivity or any other kind or processing. So

INFORMATION PROCESSING IN THE PRIMATE VISUAL SYSTEM 21 0~ ~ B SHIFT CONTROL Ail_ 1 FIGURE 9 A simple shifter circuit. A: Ascending components for a four-level circuit with eight cells at the bottom. Cells at each level have bifurcating axons that contact a pair of target cells at the next level. B: A complete shifter circuit for a three-level network, starting with four cells at the bottom level. Specific dendntic innervation patterns are shown for both ascending inputs and inhibitory neurons involved in the control of the shifting process. Heavy lines in A and B represent an activity pattern involving successive shifts to the right, left, and (in A) again to the right. Source: Anderson and Van Essen (1987~. there are successive relay stages available, inhibitory neurons in these lay- ers, and also massive feedback known to come from layer six to all of these intermediate layers. There is also physiological evidence (Poggio, 1984) that the disparity tuning of cells in the superficial layers of the cortex is actually sharper than the vergence errors known to exist in the monkey as well as in humans.

22 DAVID C VAN ESSEN Thus, there is physiological evidence that the registration problem has been solved very early in the visual cortex. Our specific hypothesis, then, is that it may be solved by means of a dynamic shifting process of predictable magnitude that would exist in these earlier and heretofore mysterious set of relay layers within primary visual cortex. This is something we are exploring at the present time. Altogether, we have learned a fair amount about what is going on in the monkey brain. The extent to which this is relevant to the human brain, with its tenfold or more greater size, is an issue that I will leave to other investigators. I think, though, it is fair to conclude that we are continuing to make progress in understanding how the detailed microcircuitry of the cortex can explain interesting aspects of visual processing. REFERENCES Anderson, C.H., and D.C. Van Essen 1987 Shifter circuits: A computational strategy for dynamic aspects of visual processing. Proc. Nate Acad. Sci. 84:6297~301. Beck, JJ. 1976 Effect of orientation and of shape similarity on perceptual grouping. Per- cepiion & Psychophysics 1. Carroll, E.W., and M. Wong-Riley 1984 Quantitative light and electron microscopic analysis of cytochrome oxidase- rich zones in VII prestriate cortex of the squirrel monkey. l Comp. Neurol. 222:1-17. DeYoe, E.^, and D.C. Van Essen 1985 Segregation of efferent connections and receptive field properties in visual area V2 of the macaque. Nature 317:58~1. 1988 Concurrent processing streams in monkey visual cortex. Trends in Neurosci. 11:219-226. Julesz, B. 1984 ldward an axiomatic theory of preattentive vision. Pp. 585~12 in G.M. Edelman, WE. Gall, and W.M. Cowan, eds., Dynamic Aspects of Neocoriical Funcuan. New York: Wiley. Livingstone, M.S., and D.H. Hubel 1984 Anatomy and physiology of a color system in the primate visual cortex. 1 Neurosci. 4:309-356. Poggio, G.F. 1984 Processing of stereoscopic information in primate visual cortex. Pp. 613~34 in G.M. Edelman, WE. Gall, and W.Nl. Cowan, eds., Dynamic Aspects of Neocortical Funcuen. New York: Wiley. Ship, S., and S. Zeki 1985 Segregation of pathways leading from area V2 to areas V4 and V5 of macaque monkey visual cortex. Nature 315:32:2-325. Tootell, R.B.H, M.S. Silverman, R.L" DeValois, and G.H. Jacobs 1983 Functional organization of the second cortical visual area in primates. Science 220:737-739.

INFORMATION PROCESSINGIN THE PRIMATE VISUALSYSTEM 23 Van Essen, D.C 1985 Functional organization of primate visual cortex. Pp. 259-329 in E.G. Jones and A. Peters, eds., Cerebral Cortex, Vol. 3. New York: Plenum Press. Van Essen, D.C, and J.H.R Maunsell 19B3 Hierarchical organization and functional streams in the visual oozes. Mends us Neurosci. 6:37~375. Van Essen, D.C and C.H. Anderson 1990 Information processing strategies and pathways in the primate retina and visual oorta`. In S.F. Zornetzer, J.L Davis, and C Law, eds., Introduction to Neural and Ekc~onic Networks. Orlando, Fla.: Academic Press. Van Essen, D.C, EN DeYoe, J.F. Olavarna, JJ. Knierim, D. Sagi, J.M. Fox, and B. Jules 1989 Neural responses to static and moving texture patterns in visual cortex of the macaque monkey. Pp. 49~7 in D.M.K. Lam and C. Gilbert, eds., Neural Mechanisms of Visual Perception. Woodland, lilac.: Portfolio Publishing.

Next: Areas and Modules in Visual Cortex »
Advances in the Modularity of Vision: Selections From a Symposium on Frontiers of Visual Science Get This Book
×
MyNAP members save 10% online.
Login or Register to save!
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!