This paper was presented at a colloquium entitled “Vision: From Photon to Perception,” organized by John Dowling, Lubert Stryer (chair), and Torsten Wiesel, held May 20–22, 1995, at the National Academy of Sciences in Irvine, CA.

Spatial integration and cortical dynamics

CHARLES D. GILBERT, ANIRUDDHA DAS, MINAMI ITO, MITESH KAPADIA, AND GERALD WESTHEIMER

The Rockefeller University, 1230 York Avenue, New York, NY 10021-6399

ABSTRACT Cells in adult primary visual cortex are capable of integrating information over much larger portions of the visual field than was originally thought. Moreover, their receptive field properties can be altered by the context within which local features are presented and by changes in visual experience. The substrate for both spatial integration and cortical plasticity is likely to be found in a plexus of long-range horizontal connections, formed by cortical pyramidal cells, which link cells within each cortical area over distances of 6–8 mm. The relationship between horizontal connections and cortical functional architecture suggests a role in visual segmentation and spatial integration. The distribution of lateral interactions within striate cortex was visualized with optical recording, and their functional consequences were explored by using comparable stimuli in human psychophysical experiments and in recordings from alert monkeys. They may represent the substrate for perceptual phenomena such as illusory contours, surface fill-in, and contour saliency. The dynamic nature of receptive field properties and cortical architecture has been seen over time scales ranging from seconds to months. One can induce a remapping of the topography of visual cortex by making focal binocular retinal lesions. Shorter-term plasticity of cortical receptive fields was observed following brief periods of visual stimulation. The mechanisms involved entailed, for the short-term changes, altering the effectiveness of existing cortical connections, and for the long-term changes, sprouting of axon collaterals and synaptogenesis. The mutability of cortical function implies a continual process of calibration and normalization of the perception of visual attributes that is dependent on sensory experience throughout adulthood and might further represent the mechanism of perceptual learning.

The visual cortex analyzes information coming from the retina, transforms it in a way that makes it possible to identify objects within the visual image, and stores a representation of these objects for later recall. The visual cortex as a whole consists of multiple areas. It has been suggested that each area is specialized for analyzing a particular aspect of the visual image, though the relative contribution of each of these areas for image analysis is only beginning to be understood. Even so, the conceptual framework by which we view the cortical mechanisms of vision is undergoing a radical change. It had been thought that information about object identity was analyzed progressively, through a series of areas, culminating in the areas situated in the temporal lobe. The nature of functional deficits following lesions of the temporal lobe, including prosopagnosia, the inability to recognize faces, suggests that these areas are responsible for the categorization and identification of objects. This led to the supposition that the plasticity of circuits required to store this information would be a property of these higher-order areas and that at the earliest stages of visual processing, especially the primary visual cortex (V1), the properties of cells would be considerably simpler and fixed in adulthood. In support of this idea, work on the development of the functional properties of cells in V1 and of its cortical circuits showed cortical plasticity to be limited to a critical period in the first few months or years of life. During this period, abnormal visual experience can produce a change in the balance of input from the two eyes, which is mediated by a change in the pattern of input to the cortex from the thalamus (1). After the end of that period, these properties and connections become fixed. This led to the assumption that in adulthood, all of the properties and connections in V1 would be fixed and that this area performs a fixed, stereotyped operation on the retinal input regardless of the characteristics of the visual stimulus. The emerging view, however, is that even in V1, and even in adulthood, many functional properties of cells, the strength and density of connections, and the functional architecture are highly dynamic. The dynamic nature of the functional properties of cortical cells is seen at several levels: the response specificity of a cell is dependent on the context within which local features are presented, and is dependent on the previous history of stimulation over time scales ranging from seconds to months.

Another change in the view of cortical processing, which is likely to be related to the dynamic properties of visual cortex, concerns the ability to link the components of a visual image into a unified percept. One of the fundamental requirements of the visual system is to integrate the line segments of which object boundaries are composed, and the surfaces of which three-dimensional structures are made, into identifiable objects. It had long been believed that the sizes of receptive fields (RFs) for cells at early stages in the visual pathway were very tiny and discrete, providing a minute window on an object's contour. This rather atomistic view of visual processing left a major issue unresolved: how the line segments of which an object is composed are integrated in a way that enables us to deal with the object as a whole. The traditional viewpoint would have held that the process of spatial integration is left to high-order stages in the visual pathway, such as those found in the temporal lobe. Several lines of evidence indicate that, quite to the contrary, spatial integration begins at the earliest stages, even in primary visual cortex (V1), and builds progressively at consecutive stages as the signal is processed from one area to the next. For several reasons one can view the process of object recognition as being distributed across all visual cortical areas, rather than taking place at one final stage, so that a strictly hierarchical view of processing is an oversimplification. The integrative, dynamic nature of even the first visual cortical area points toward a considerable degree of complexity in the properties of V1 cells. Visual information is present at the finest spatial grain within V1, so the perceptual attributes involving the most detailed spatial information are likely to be based in V1. The

The publication costs of this article were defrayed in part by page charge payment. This article must therfore be hereby marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact.

Abbreviations: PS, point spread; RF, receptive field.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION This paper was presented at a colloquium entitled “Vision: From Photon to Perception,” organized by John Dowling, Lubert Stryer (chair), and Torsten Wiesel, held May 20–22, 1995, at the National Academy of Sciences in Irvine, CA. Spatial integration and cortical dynamics CHARLES D. GILBERT, ANIRUDDHA DAS, MINAMI ITO, MITESH KAPADIA, AND GERALD WESTHEIMER The Rockefeller University, 1230 York Avenue, New York, NY 10021-6399 ABSTRACT Cells in adult primary visual cortex are capable of integrating information over much larger portions of the visual field than was originally thought. Moreover, their receptive field properties can be altered by the context within which local features are presented and by changes in visual experience. The substrate for both spatial integration and cortical plasticity is likely to be found in a plexus of long-range horizontal connections, formed by cortical pyramidal cells, which link cells within each cortical area over distances of 6–8 mm. The relationship between horizontal connections and cortical functional architecture suggests a role in visual segmentation and spatial integration. The distribution of lateral interactions within striate cortex was visualized with optical recording, and their functional consequences were explored by using comparable stimuli in human psychophysical experiments and in recordings from alert monkeys. They may represent the substrate for perceptual phenomena such as illusory contours, surface fill-in, and contour saliency. The dynamic nature of receptive field properties and cortical architecture has been seen over time scales ranging from seconds to months. One can induce a remapping of the topography of visual cortex by making focal binocular retinal lesions. Shorter-term plasticity of cortical receptive fields was observed following brief periods of visual stimulation. The mechanisms involved entailed, for the short-term changes, altering the effectiveness of existing cortical connections, and for the long-term changes, sprouting of axon collaterals and synaptogenesis. The mutability of cortical function implies a continual process of calibration and normalization of the perception of visual attributes that is dependent on sensory experience throughout adulthood and might further represent the mechanism of perceptual learning. The visual cortex analyzes information coming from the retina, transforms it in a way that makes it possible to identify objects within the visual image, and stores a representation of these objects for later recall. The visual cortex as a whole consists of multiple areas. It has been suggested that each area is specialized for analyzing a particular aspect of the visual image, though the relative contribution of each of these areas for image analysis is only beginning to be understood. Even so, the conceptual framework by which we view the cortical mechanisms of vision is undergoing a radical change. It had been thought that information about object identity was analyzed progressively, through a series of areas, culminating in the areas situated in the temporal lobe. The nature of functional deficits following lesions of the temporal lobe, including prosopagnosia, the inability to recognize faces, suggests that these areas are responsible for the categorization and identification of objects. This led to the supposition that the plasticity of circuits required to store this information would be a property of these higher-order areas and that at the earliest stages of visual processing, especially the primary visual cortex (V1), the properties of cells would be considerably simpler and fixed in adulthood. In support of this idea, work on the development of the functional properties of cells in V1 and of its cortical circuits showed cortical plasticity to be limited to a critical period in the first few months or years of life. During this period, abnormal visual experience can produce a change in the balance of input from the two eyes, which is mediated by a change in the pattern of input to the cortex from the thalamus (1). After the end of that period, these properties and connections become fixed. This led to the assumption that in adulthood, all of the properties and connections in V1 would be fixed and that this area performs a fixed, stereotyped operation on the retinal input regardless of the characteristics of the visual stimulus. The emerging view, however, is that even in V1, and even in adulthood, many functional properties of cells, the strength and density of connections, and the functional architecture are highly dynamic. The dynamic nature of the functional properties of cortical cells is seen at several levels: the response specificity of a cell is dependent on the context within which local features are presented, and is dependent on the previous history of stimulation over time scales ranging from seconds to months. Another change in the view of cortical processing, which is likely to be related to the dynamic properties of visual cortex, concerns the ability to link the components of a visual image into a unified percept. One of the fundamental requirements of the visual system is to integrate the line segments of which object boundaries are composed, and the surfaces of which three-dimensional structures are made, into identifiable objects. It had long been believed that the sizes of receptive fields (RFs) for cells at early stages in the visual pathway were very tiny and discrete, providing a minute window on an object's contour. This rather atomistic view of visual processing left a major issue unresolved: how the line segments of which an object is composed are integrated in a way that enables us to deal with the object as a whole. The traditional viewpoint would have held that the process of spatial integration is left to high-order stages in the visual pathway, such as those found in the temporal lobe. Several lines of evidence indicate that, quite to the contrary, spatial integration begins at the earliest stages, even in primary visual cortex (V1), and builds progressively at consecutive stages as the signal is processed from one area to the next. For several reasons one can view the process of object recognition as being distributed across all visual cortical areas, rather than taking place at one final stage, so that a strictly hierarchical view of processing is an oversimplification. The integrative, dynamic nature of even the first visual cortical area points toward a considerable degree of complexity in the properties of V1 cells. Visual information is present at the finest spatial grain within V1, so the perceptual attributes involving the most detailed spatial information are likely to be based in V1. The The publication costs of this article were defrayed in part by page charge payment. This article must therfore be hereby marked “advertisement” in accordance with 18 U.S.C. §1734 solely to indicate this fact. Abbreviations: PS, point spread; RF, receptive field.

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION presence of feedback loops to V1 from numerous areas along the visual cortical pathway allows the analysis of complex properties of the visual image, such as surface segmentation, to be referred back to earlier stages. Anatomical evidence shows that at the early cortical levels the substrate exists for providing cells with input from relatively large parts of the visual field. From physiological studies, it now appears that the response properties of cells are modulated by stimuli lying outside of the “classical” RF. The structure and specificity of RFs and of cortical functional architecture are increasingly seen as context dependent. This may represent the cellular mechanism of perceptual studies showing that the visual system is capable of linking contours and surfaces in a process of perceptual fill-in (2–6), that our perception of the attributes of local features is influenced by context (7–14), and that simple contours can be picked out of a noisy background (15–18). The perceptual phenomena obey rules that are consonant with the patterns of connectivity in primary visual cortex, supporting the idea that a major component of the process of spatial integration occurs there. One can characterize a host of processes, referred to as intermediate-level vision, that occur between the discrimination of simple visual attributes, such as orientation, and the identification of complex objects. The neural mechanisms may be found at earlier stages than previously believed. Long-Range Cortical Connections Our initial evidence of a cellular substrate for spatial integration was a pattern of long-range horizontal connections formed by the axons of pyramidal cells in V1 (19–23). The extensive plexus of horizontal connections was revealed in experiments which mapped the intrinsic cortical circuit and related cortical connections to RF properties by labeling the full axonal arbor of functionally characterized cells (Fig. 1). The findings were quite surprising, since they seemed to violate the principles of RF structure and cortical topography. Because of the extensive spread of the horizontal connections, their cellular targets are capable of integrating input over an area of cortex that represents an extent of visual field roughly an order of magnitude larger in area than the cells' own RFs. This finding contrasted with the belief that all the connections in the cortex are vertical, between cells with overlapping RFs and similar orientation preference, with relatively little lateral transfer of information. In effect it posed a contradiction in the definition of the RF, in that it suggested that cells should be sensitive to stimuli lying outside of the RF. The resolution to this conflict lies in the way the RF is defined, which is highly dependent on the stimulus used. When one uses a simple stimulus, such as a single short line segment of the appropriate orientation, one can activate a cell to suprathreshold levels over a very limited area, the classical RF. If, in addition to a line lying within the classical RF, one places additional stimuli outside the RF, the response of the cell changes. The nature of the change depends on the precise geometric relationship between the stimuli lying within and outside the RF, and it correlates well with the influence of context on the perception of local features. The rules governing contextual influences are mirrored by the registration between the long-range horizontal connections and cortical functional architecture. The first relationship is the extent of visual space that is represented by the area of cortex over which these connections spread. The horizontal connections spread laterally up to 6–8 mm (19,20,25). As was shown by Hubel and Wiesel (26), a distance of two hypercolumn diameters (a hypercolumn is defined as a complete set of orientation columns), or roughly 1.5 mm, is the minimal cortical distance between cells with nonoverlapping RFs. The distance covered by the longest-range horizontal connections, therefore, separates cells with RFs that are several RF diameters apart. The second principle of organization of the horizontal connections is revealed by the clustered nature of their axon collaterals (Fig. 1). These clusters are separated by 0.5–1 mm, approximating the width of an individual hypercolumn. The relationship between the clusters and the columnar functional architecture was shown in several ways. Crosscorrelation analysis, which is a statistical technique relating the time of occurrence of action potentials in pairs of neurons and which provides a measure of effective connection strength between the neurons, showed that cells with correlated firing had similar orientation preference (27,28). Correlated firing was found even for cell pairs with nonoverlapping RFs, as would be expected for the distances spanned by the horizontal connections. The registration between the clustering of the horizontal connections and orientation columns was revealed anatomically by labeling the orientation columns by 2-deoxyglucose autoradiography and marking the horizontal connections with extracellularly applied tracers (25). This showed that the horizontal connections ran between columns of similar orientation specificity. FIG. 1. Examples of horizontally projecting pyramidal cells in primate area V1, labeled by intracellular injection of horseradish peroxidase. The cell in A is seen in a transverse view of cortex, with the cell and dendritic field located in the superficial layers. Its axon spread for >4 mm parallel to the cortical surface, and within this area the axon's collaterals were distributed in discrete clusters. The cell in B is seen in tangential view, with only the axon shown. The axonal field is shown in relation to the cytochrome oxidase blobs, a feature of histochemical staining in V1 that correlates with the ocular dominance columns, giving a sense of the scale of the horizontal connections. This axon traversed 6 mm of cortex, mediating communication between cells with widely separated RFs. (Bars = 1 mm.) [Reprinted with permission from ref.24 (copyright 1992, Cell Press).] Visualization of Lateral Cortical Interactions with Optical Recording. More recently we have used optical recording to obtain a functional measure of the extent and specificity of the horizontal connections. Optical recording reveals in vivo the pattern of activity, projected onto the cortical surface, elicited by a given stimulus. It relies on changes in surface reflectance, referred to as the “intrinsic signals,” that are linked to metabolic changes resulting from local neural activity. Optical recordings of this signal have been used to visualize the cortical functional architecture, in particular the arrangement of orientation columns (29–32). We used optical recording to

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION visualize the area of cortex activated by a minimal visual stimulus, known as the cortical point spread (PS; ref.33). The PS was a useful tool to analyze the lateral interactions in cortex. In addition to the optically measured PS, we determined the area of cellular spiking activity, the spiking PS, by recording with extracellular electrodes. The striking result in these studies was that the optical PS was much larger than the spiking PS (Fig. 2; refs.34 and 35). The diameter of the optical PS averaged 4 mm, while that of the spiking PS averaged 750 µm. This finding was consistent with the idea that the optical PS revealed areas of cortex where cells were activated to subthreshold levels, and that the area of subthreshold activation represented >95% of the total area covered by the optical PS. The visual field representation of the optical PS was determined electrophysiologically by measuring RF size and positions of cells at its boundary. For the example shown in Fig. 2, this corresponded to an area of visual field 4° in diameter, as compared to the 0.5° size of the stimulus used and the similar size of RFs in the region. This supported the idea that the lateral interactions within V1 provide an order-of-magnitude-larger area of visual integration than that covered by the classical RF. The discrepancy between the area of visual field over which cells receive input and the area of the classical RF emphasizes the stimulus dependency of the RF definition. Ordinarily, the RF boundaries are determined by using a short line segment of a particular orientation as the visual stimulus. When using multiple stimuli, however, one can see evidence for modulatory regions surrounding the RF. This is shown again in Fig. 2, where a conditioning stimulus was placed within the classical RF, and a second test bar was placed in various positions outside the RF. This revealed the existence of a large inhibitory “moat” surrounding the RF, which includes regions mediating end- or side-band inhibition (36,37). The diameter of the central excitatory core and the surrounding inhibitory region for this cell was ≈4°, equivalent to the visual field representation of the optical PS. If we assumed that connections between the spiking region and the surrounding area of subthreshold activation are reciprocal, the correspondence between the area of the RF surround and the representation of the area of cortical activation suggested that lateral cortical interactions play a part in the extended RF measured with complex visual stimuli. Further evidence for the relationship between the horizontal connections and the cortical columnar architecture is also revealed by the comparison between the optical PS and the optical image of orientation columns (Fig. 2). The close match between the two, as seen in the uniform blue color in the difference image, showed that the distant patches of subthreshold activation had the same orientation specificity as the central zone of spiking activity. That is, when a stimulus of a particular orientation activates a small patch of spiking cells, these cells have axons projecting outside the area of spiking activity, which leads to subthreshold activation of satellite patches of the same orientation preference. The subthreshold signal could arise from inhibitory as well as excitatory activity, either of which could lead to the metabolic signal which induces the change in surface reflectance. Synaptic Physiology of Horizontal Connections. The suggestion from the RF map obtained by using paired stimuli is that much of the extended RF surround is inhibitory. Is it plausible that some of this inhibition could be mediated by the horizontal connections? Though the longest-range connections in the cortex are formed by pyramidal cells, which produce direct excitation, the effect of the horizontal plexus can involve both excitation and inhibition. The reason for this is seen in both anatomical studies and physiological studies of the horizontally evoked synaptic potentials in cortical slices. Roughly 80% of the targets of the horizontal connections are other spiny, pyramidal cells and 20% are inhibitory interneurons (38). Though the inhibitory limb of this circuit represents the minority of connections, it can have a disproportionate effect in terms of the resultant synaptic potentials. When the horizontal connections are weakly activated, the synaptic potential is exclusively excitatory, but as additional horizontal inputs are recruited by increasing the strength of stimulation, the excitatory postsynaptic potential (epsp) initially is truncated and reversed by a disynaptic inhibitory postsynaptic potential (ipsp) and, at the highest stimulus strengths, becomes strongly inhibitory (39). Another important feature of the horizontally evoked epsp is that it increases substantially as the cell is depolarized. This increases the strength of the horizontal inputs when the cell is simultaneously depolarized by other inputs, and suggests that the effect of horizontal inputs is state dependent (39). These nonlinear properties at the synaptic level may account in part for the nonlinear RF properties described below. Contextual Influences in Psychophysics and Physiology The interactions revealed by the anatomy of the horizontal connections and their relationship with orientation columns suggest a possible role in visual spatial integration. The process of integration was evaluated in terms of the threshold level of contrast required to detect a target line. In psychophysical experiments, the contrast level required to detect a target line is influenced by the presence of a second line adjacent to the target. When the flanking line is located near the target line, is colinear with it, and has the same orientation, the required contrast level for detection of the target is reduced by ≈40%. Shifting the flanking line away from the target along the colinear axis reduces its influence. Similarly, shifting it laterally, along an axis orthogonal to its orientation, also abolishes the effect. In addition to the influence of changing position, there is also a dependency on orientation, such that changing the relative orientation of the two lines from parallel to >30° abolishes the threshold reduction (11,13,14). Analogous effects are seen at the level of single cells in V1. The long-range horizontal connections in the cortex raise the possibility for extensive facilitatory influences from outside the classical RF, and these have been seen in a number of studies (40–47). Also, contextual alteration of a cell's response specificity, such as orientation preference, has been suggested to play a role in the tilt illusion (45,48). To determine whether facilitation from outside the RF, in area V1, might account for contextual influences on perception, we explored whether they showed similar dependency on position and orientation. These experiments were done in alert, fixating monkeys, with the idea that making a comparison between human and monkey experiments requires using animals that are at levels of alertness comparable to those of the human subjects. In these studies, >40% of the complex cells in the superficial layers of cortex showed facilitation when, in addition to the stimulus within the RF, a second nearby, colinear, iso-oriented line was placed outside the excitatory core of the RF (Fig. 3; ref.14). The median level of facilitation was 2.3-fold. For most of the cells showing facilitation, the greatest effect occurred when the flanking line was near the boundary of the RF, and the effect decreased as the line was separated from the target line in directions along the orientation axis or orthogonal to it (Fig. 3A and B). In addition, the effect was maximal when the lines were parallel, and it decreased as the flanking line was tilted relative to the target line (Fig. 3C). The physiological results showed not only a similar dependency on position and angle as the psychophysical studies, but the effects operated over comparable spatial scales. Thus the substrate for the psychophysical effects of context may be present as early as V1. How might the facilitatory effect of a single line outside the classical RF relate to the process of binding the components of a contour along its length, and to the segmentation of a

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION FIG. 2. Functional interactions mediated by horizontal connections visualized in cat visual cortex with optical recording of cortical PS. (a) Optical image of orientation columns, based on stimulation with a global stimuli of gratings of different orientations. The different colors represent the orientation specificity of the underlying columns (key above), blue representing 0°, yellow 90°, and so on. Areas of strong orientation preference are seen as bright colors, areas of poor orientation preference or quickly changing orientation are seen as dark spots or lines. (b) Optical PS, the area of cortical activation in response to a small, 0.5° light bar, is 3.2 mm in diameter. (c) Optical PS subtracted from image of orientation columns, showing close correspondence over the entire area of the PS, consistent with a pattern of lateral interactions between columns of similar orientation specificity. This image is also multiplied by the strength of the signal, so that only sites of strong orientation preference and good visual responses are seen in the subtracted pattern. (d) Visual field representation of optical PS, obtained by electrophysiological recordings around its perimeter. The area of activation represents an area of visual field that is much larger than the stimulus used to generate the optical PS. (e) RF profile, showing close correspondence to the hand-mapped RFs, and reinforcing the discrepancy between RF size and the visual representation of the area of cortical activation. (f) Interaction profile of RF, using a conditioning stimulus to elevate the level of firing of the cell, and a test stimulus placed in and around the RF. This procedure brings out the subthreshold inhibitory influences surrounding the RF. The overall profile, including both excitation and inhibition, is equivalent in size to the area of cortical activation measured optically. (g) Measurement of area of spiking activity by closely spaced electrode penetrations. The area of cortex activated to spiking levels by the test stimulus (bold red square) is 0.5 mm × 1.0 mm, much smaller than the area measured with optical recording, which includes regions of subthreshold and suprathreshold activation. (Adapted from ref. 34.)

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION contour from its background? One way to interrupt a contour is to place an orthogonal line across one of the component line segments. When, instead of a flanking line, one placed a T-shaped figure outside the RF, with the crossbar of the T lying between the two lines, the psychophysical effect was greatly reduced. Similarly, placing the T outside the classical RF often abolished the facilitation seen with a simple line (Fig. 3D). In more complex visual environments, the presence of stimuli outside the RF often inhibited cells' responses, due to the flanking inhibitory regions surrounding the core excitatory region of the RF. When the classical RF was surrounded by a background of randomly positioned and oriented lines, the effect of one or more lines of the appropriate position and orientation outside the RF counteracted the inhibition caused by the random background. These results suggest that with the appropriate configuration of contours surrounding the RF, the cell is lifted from a rather profound level of inhibition and its excitatory inputs are unmasked, allowing it to respond to the stimulus. The push-pull nature of the surround effects, operating over the cortical sheet, would promote the activation of cells whose RFs are superimposed on a salient contour, and would suppress the activity of cells whose RFs cover a random background. These findings emphasize the nonlinear nature of complex cells in primary visual cortex: the response of these cells in a complex visual environment cannot be predicted from their responses to a single line, presented at different positions and orientations. In this sense the response specificity of cells is dynamic, changing with alterations in context, but not necessarily requiring changes in synaptic weights. The mechanism underlying the contextual sensitivity is likely to involve nonlinearities in the integration ascending interlaminar inputs, which carry information about the more local stimuli, and the horizontal inputs, which carry information about contextual stimuli lying over a larger area. Long-Term Cortical Plasticity An expanding body of evidence indicates that even in adulthood some fundamental properties of cortical cells, and cortical functional architecture itself, are mutable and subject to alteration by experience. The initial evidence came from studies in the somatosensory system, which, like the visual system, has a map of the sensory surface on the cortical surface. Amputation of a finger leads to an alteration of this map, such that the area of cortex originally receiving input from the amputated finger changes its representation to the adjacent fingers (49–51). We performed similar studies in the visual system, driven by our knowledge of the existence of the long-range horizontal connections, which seemed a likely substrate for the map alterations. To study experience-dependent plasticity in the visual system, we made focal retinal lesions at homologous positions in the two eyes, thereby removing visual input from a focal area of visual cortex. Over a period of several months, the silenced area of cortex, or cortical scotoma, recovered functioning visual input. The RFs of cells that recovered visual responses shifted from the lesioned part of the retina to positions immediately surrounding it. Effectively, the cortical topography had reorganized, expanding the representation of the perilesion retina and shrinking the representation of the lesioned part of the retina (Fig. 4; refs.52–59). The substrate for the reorganization was explored by recording at various stages along the visual pathway. At a time when the cortex had been remapped, the lateral geniculate nucleus (LGN, the major source of input to primary visual cortex) still sustained a large silent area. Hence one could conclude that the reorganization was due to processes intrinsic to the cortex and not to changes at antecedent levels of visual processing (53,55). The extent of reorganization, roughly 6–8 mm in diameter, could not be explained by the lateral spread of thalamic afferents, which are up to about 1.5–2 mm wide, unless they had increased their projection pattern into the center of the reorganized region. On the other hand, the extent of the long-range horizontal connections did approximate that of the area of reorganization, so these were likely candidates for the cellular substrate of topographic plasticity. We were able to rule out a significant role of thalamocortical projections, leaving the long-range horizontal connections as the most likely substrate (53,55). The idea that the horizontal connections were responsible for the reorganization was supported by the finding that after recovery, the pattern of orientation columns was similar to that seen before the lesion was made, despite the fact that the RFs of the cells in this region had shifted considerably in visual space (34). Given that the horizontal connections run between columns of similar specificity, the involvement of a preexisting framework of horizontal connections would cause the reorganized cortex to recover its original pattern of orientation columns. In addition, the recovery was associated with an increase in the size of the spiking PS, with a larger area of cortex representing a particular part of the visual field. The spiking PS expanded to a size similar to the size of the optical PS in normal cortex, indicating that the reorganization occurred by strengthening existing lateral interactions from subthreshold to suprathreshold levels. We next attempted to determine the mechanism accounting for the strengthening of the connection from cells lying outside the cortical scotoma to those within the scotoma. The length of time required to see the full extent of reorganization, 2 months or more, raised the possibility of a morphological change. To explore this, we placed injections of biocytin, an anterograde label, at several sites immediately outside the boundary of the original scotoma and compared the pattern of projection into the scotoma with projections into normal (unreorganized) cortex. After about a year, the density of the horizontal projection into the reorganized region had doubled, indicating that the strengthening was mediated by a process of sprouting of axon collaterals and synaptogenesis (54). The change observed here did not entail an increase in the extent of the horizontal arbor, but an increase in the density of collateral arborization within the existing clusters of axon collaterals. To see sprouting in the adult brain as a result of alteration in visual experience was quite a contrast with the limitations on alterations in connectivity to the critical period (60). Clearly, even in adulthood, brain plasticity results from a continuing process of experience-dependent synaptogenesis. Rapid Cortical Dynamics The kind of plasticity discussed to this point involves changes occurring over a period of months. These same experiments revealed a much faster plasticity, occurring within minutes after the retinal lesions were made. Although we saw the expected silencing of activity within the center of the cortical

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION representation of the lesioned part of the retina, there was still visually driven activity for cells whose RFs were originally located within what was later to become the boundary of the retinal lesion. Within minutes of making the lesion, the RFs of these cells had expanded an order of magnitude in area and had shifted to positions outside the lesion (53,55,59). Thus, substantial changes in RF size and position can be induced, as a result of alteration in visual experience, over a time course of minutes. That one could see these changes so quickly—and that they could be generated without a cutting of the connections to the cortex, but merely by destroying the retinal photoreceptor layer —suggested that one did not need to make lesions in order to induce changes in RF properties. To test this idea, we generated an “artificial scotoma,” a masked part of the visual field including and surrounding the RFs of cells isolated in electrophysiological recordings. The stimulus consisted of a pattern of moving lines or dynamic (twinkling) random dots, within which a blank area or occluder was located. The occluder was sized and positioned to lie over the RF of a cell isolated with a recording electrode, and was roughly three times the diameter of the classical RF. The RF boundary was measured before stimulation with this pattern, during stimulation with the occluder present, and after stimulation of the RF center. The effect after stimulation with this pattern for a few minutes was to expand the size of the RF severalfold, and stimulation within the RF caused it to collapse back down to its original size (Fig. 5; refs.61 and 62). Fig. 5 shows that the RF expanded into parts of the visual field where no response had been elicited previously, and demonstrates that the effect is a true RF expansion and not simply gain control. This effect reveals that the structure of the RF is dependent on the history of previous visual stimulation of the FIG. 3. Contextual effects on perception of local features and on RF properties of cells in superficial layers of primary visual cortex. The effects illustrated here show the effect of context on the perception of the visual attributes of local features and the underlying facilitatory effect of contours lying outside the classical RF. The psychophysical studies were done in human observers, and the physiological studies were done on superficial layer complex cells in primary visual cortex of alert, fixating monkeys. (A) When two lines were placed in close proximity and were colinear and similarly oriented, there was a reduction in the contrast level needed to detect a target line, and this effect diminished as the lines were separated along the colinear axis (Left). Individual cells could show a 2-fold or greater increase in their responses to lines outside the classical RF (Center), and over the population the facilitatory effect dropped off as the lines were separated (Right). (B) There was a loss of the thresholdlowering effect as the lines were shifted laterally (Left), and a corresponding loss of facilitation at the level of single cells (Center), which was seen for the overall population (Right). (C) The effect was also dependent on orientation, as seen in both the psychophysical (Left) and physiological (Center and Right) experiments, with the strongest effect seen when the lines were parallel. (D) Breaking the continuity between the lines by introducing a cross bar between them caused a loss of the perceptual effect (Left) and an elimination of the facilitation seen with individual cells (Right). (E) The effects described above with pairs of lines might be related to the ability for salient contours to emerge from a noisy background. The effect of a background of randomly placed and oriented lines was to inhibit the response of the cell, but when elements of the background were shifted to positions that were near, colinear and parallel with the line that lay within the RF, the cell's response was lifted from inhibition. Thus with the appropriate configuration of contours lying within and outside the RF, cells could respond in a visual context that would otherwise be strongly inhibitory. (Adapted from ref.14.)

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION cell, and reinforces the fact that stimuli in one part of the visual field influence the response properties of cells with RFs located some distance away. It points out the existence of a problem in neurobiology analogous to the uncertainty principle in physics, that whatever one does to measure the response properties of a cell may change them. The perceptual consequences of the short-term RF plasticity were explored using psychophysical techniques. It is plausible to think, given the nature of these changes, that the expansion of RFs could explain the phenomenon of perceptual fill-in, in which an occluder appears to fill-in with the color or texture pattern surrounding it (6). Cells represent “line labels” indicating, when the cell is active, the presence of a stimulus somewhere within their RFs. When the RFs inside an artificial scotoma expand, the cells become activated by the stimuli lying outside the scotoma, erroneously signaling the presence of the stimuli within the boundary of the scotoma. In addition to perceptual fill-in, the phenomenon of RF expansion might be expected to produce a distortion in the spatial position sense in the vicinity of the scotoma. Due to the imbalance in RF size, a line located near the boundary of the scotoma would activate more cells with RFs within the scotoma than outside, and it would, therefore, be perceived as being shifted inward toward the center of the scotoma. This was shown in human observers, where viewing an artificial scotoma causes the perceived position of objects located near its boundary to be pulled in towards its center (63). Thus, in human visual perception one can observe the effects of RF plasticity, within a short period of time. These experiments provided insight into the time course of the plasticity, in that viewing the artificial scotoma for a period as short as 1 sec caused a significant shift in perceived position. In addition, the task showed a learning effect, with subjects showing an increased accuracy in the determination of position. Both the short-term plasticity evidenced by the perceptual shift and the longer-term learning effects may be associated with the RF expansion and the associated increase in the cortical representation of the trained portion of visual field. FIG. 4. Schematic representation of pattern of recovery following binocular retinal lesions. Initially a region was silenced, but over a period of a few months it recovered visually driven activity. This recovery involved a shrinkage in the representation of the lesioned part of the retina and an expansion in the representation of the part of the retina surrounding the lesion. The recovery was due to mechanisms that were intrinsic to the cortex, involving intracortical connections, and over the time course of the full effect involved sprouting of axonal collaterals and synaptogenesis. Much shorter-term changes, over a smaller cortical scale, were seen within minutes. [Reprinted with permission from ref.66 (copyright 1992, Cell Press).] Given the extent of the long-range horizontal connections, the mechanism of the shortest-term plasticity is likely to involve a change in the effectiveness of existing connections, rather than a formation of new synapses. Rather, one can think of the long-range connections in a context similar to that seen in development, where the connections are exuberant, having a broader functional potential than that expressed at any one time. By varying the effective strength of a subset of connections formed by a cell, the functional properties of the target cells can be shifted around within a larger domain. The changing strength of the connections was measured by using the technique of cross-correlation analysis referred to above. When two cells are isolated and their RFs are placed within an artificial scotoma and caused to expand after a period of stimulation, the peak in correlated firing increases in size (62). The increase in the peak in the correlogram indicates that there was an associated increase in the effective connection strength between the neurons. The observed plasticity is likely to involve intrinsic cortical connections, since the effect shows interocular transfer (64). Here again it is tempting to attribute the change to the horizontal connections. FIG. 5. Short-term plasticity of RF size induced by an artificial scotoma, an occluded area of visual field surrounded by a pattern of lines or random dots. When placed within the scotoma, the RF expands severalfold in area, and when stimulated in its center, it collapses toward its original size. The RF profile was mapped by placing a small bar at different positions in an 8 × 8 grid. The responses are shown as three-dimensional contour plots on the left, with response rate represented on the vertical axis, and as two-dimensional plots on the right, with response rate represented by the brightness of each pixel. The scale markers represent 1° of visual angle. (A) The original size of the RF. The dashed white square on the twodimensional plot indicates the size and position of the boundary of the artificial scotoma during subsequent conditioning with a dynamic random dot display that surrounded the scotoma. Peak response rate was 40 spikes/sec. (B) After a few minutes of conditioning, the RF expanded, as indicated in both the three-dimensional and twodimensional plots. Peak response rate was 24 spikes/sec. (C) After visual stimulation within the RF center, the RF collapsed down to its preconditioned size. Peak response rate was 20 spikes/sec. (D) Placing the RF again within the artificial scotoma resulted in its reexpansion. Peak response rate was 60 spikes/sec.

OCR for page 59
COLLOQUIUM ON VISION: FROM PHOTON TO PERCEPTION This increase in effectiveness may be achieved in various ways at the synaptic level, including a strengthening of excitatory connections and an adaptation of inhibitory connections (65). Since the horizontal connections, as described above, involve push-pull interactions between excitation and inhibition, a reduction in the inhibition would unmask the connections, boosting their strength from a subthreshold influence to a suprathreshold, driving influence. Moreover, since the ability to strengthen these connections is itself under inhibitory control, where with less inhibition there is an increased probability of producing a use-dependent change in the excitatory connection (66), one might produce an increase in the strength of the horizontal connections by a cascade of mechanisms. The precise synaptic mechanisms governing plasticity in this system, however, remain to be worked out. Summary The response properties of cells in primary visual cortex are considerably more complex than was previously believed. The complexity is manifest as both a context dependency and a dependency on the prior history of stimulation. As a result of these findings it is clear that the primary visual cortex carries information about higher-order characteristics of the visual stimulus rather than a mere representation of the line segments of which it is composed. Instead, it provides information about the character of the conjunctions between contours and surfaces in the visual image. The perceptual consequences of the dynamic changes in RF structure and cortical functional architecture depend on the time scale of the plasticity. Changes occurring over the longest time periods may play a role in recovery of function after lesions of the central nervous system, but under normal circumstances may be involved in perceptual learning. Over shorter time scales, the effect may represent a continuing process of normalization and calibration of the visual system, as well as the linkage of contours and fill-in of surfaces common to a single object. Several characteristics of the phenomena described above bear emphasizing: cells in area V1 are increasingly being seen as being involved in complex perceptual tasks, mediating the process of linkage of contours and integrating visual information over visual space. These processes are likely to involve a differential strengthening and weakening of subsets of connections within extensive axonal fields, the long-range horizontal connections representing a likely substrate for many of the observed effects. Because of these connections any cortical cell has a wider range of potential properties it can potentially express than is manifest at any given time. An important question to be addressed is to differentiate those contextual effects and dynamic changes in RFs that are due to the intrinsic horizontal connections, hence reflecting bottom-up processes, from those that arise from feedback connections, reflecting top-down influences. Though the precise synaptic mechanisms remain to be worked out, the fact that the effects have been observed in primary visual cortex, where much of the detailed functional architecture, connectivity, and RF properties have been worked out in considerable detail, makes accessible an understanding of the mechanisms of higher-order perceptual phenomena. 1. Hubel, D. H. & Wiesel, T. N. ( 1970) J. Physiol (London) 206, 419–436. 2. Kanizsa, G. ( 1979) Organization in Vision. Essays on Gestalt Perception (Praeger, New York). 3. Yarbus, A. L. ( 1957) Biophysics 2, 683–690. 4. Krauskopf, J. ( 1961) Am. J. Psychol. 80, 632–637. 5. Crane, H. D. & Piantanida, T. P. ( 1983) Science 221, 1078–1079. 6. Ramachandran, V. S. & Gregory, T. L. ( 1991) Nature (London) 350, 699–702. 7. Gibson, J. J. & Radner, M. ( 1937) J. Exp. Psychol. 20, 453–467. 8. Badcock, D. R. & Westheimer, G. ( 1985) Vision Res. 25, 1259–1269. 9. Westheimer, G., Shimamura, K. & McKee, S. P. ( 1976) J. Opt. Soc. Am. 66, 332–338. 10. Westheimer, G. ( 1986) J. Physiol. (London) 370, 619–629. 11. Polat, U. & Sagi, D. ( 1993) Vision Res. 33, 993–999. 12. Polat, U. & Sagi, D. ( 1994) Vision Res. 28, 115–132. 13. Dresp, B. ( 1993) Spatial Vision 7, 213–225. 14. Kapadia, M. K., Ito, M., Gilbert, C. D. & Westheimer, G. ( 1995) Neuron 15, 843–856. 15. Wertheimer, M. ( 1938) Laws of Organization in Perceptual Forms (Harcourt, Brace & Jovanovich, London). 16. Grossberg, S. & Mingolla, E. ( 1985) Percept. Psychophys. 38, 141–171. 17. Ullman, S. ( 1990) Cold Spring Harbor Symp. Quant. Biol. 55, 889–898. 18. Field, D. J., Hayes, A. & Hess, R. F. ( 1993) Vision Res. 33, 173–193. 19. Gilbert, C. D. & Wiesel, T. N. ( 1979) Nature (London) 280, 120–125. 20. Gilbert, C. D. & Wiesel, T. N. ( 1983) J. Neurosci. 3, 1116–1133. 21. Rockland, K. S. & Lund, J. S. ( 1982) Brain Res. 169, 19–40. 22. Rockland, K. S. & Lund, J. S. ( 1983) J. Comp. Neurol. 216, 303–318. 23. Martin, K. A. C. & Whitteridge, D. ( 1984) J. Physiol. (London) 353, 463–504. 24. Gilbert, C. D. ( 1992) Neuron 9, 1–20. 25. Gilbert, C. D. & Wiesel, T. N. ( 1989) J. Neurosci. 9, 2432–2442. 26. Hubel, D. H. & Wiesel, T. N. ( 1974) J. Comp. Neurol. 158, 295–306. 27. Ts'o, D. Y., Gilbert, C. D. & Wiesel, T. N. ( 1986) J. Neurosci. 6, 1160–1170. 28. Ts'o, D. Y. & Gilbert, C. D. ( 1988) J. Neurosci. 8, 1712–1727. 29. Grinvald, A., Lieke, E., Frostig, R. D., Gilbert, C. D. & Wiesel, T. N. ( 1986) Nature (London) 324, 361–364. 30. Frostig, R. D., Lieke, E. E., Ts'o, D. Y. & Grinvald, A. ( 1990) Proc. Natl. Acad. Sci. USA 87, 6082–6086. 31. Ts'o, D. Y., Frostig, R. D., Lieke, E. E. & Grinvald, A. ( 1990) Science 249, 417–420. 32. Bonhoeffer, T. & Grinvald, A. ( 1991) Nature (London) 353, 429–431. 33. McIlwain, J. T. ( 1975) J. Neurophysiol. 38, 219–230. 34. Das, A. & Gilbert, C. D. ( 1995) Nature (London) 375, 780–784. 35. Grinvald, A., Lieke, E., Frostig, R. D. & Hildesheim, R. ( 1994) J. Neurosci. 14, 2545–2568. 36. Hubel, D. H. & Wiesel, T. N. ( 1962) J. Physiol. (London) 160, 106–154. 37. Bishop, P. O., Coombs, J. S. & Henry, G. H. ( 1971) J. Physiol. (London) 219, 659–687. 38. McGuire, B. A., Gilbert, C. D., Rivlin, P. & Wiesel, T. N. ( 1991) J. Comp. Neurol. 305, 370–392. 39. Hirsch, J. A. & Gilbert, C. D. ( 1991) J. Neurosci. 11, 1800–1809. 40. Maffei, L. & Fiorentini, A. ( 1976) Vision Res. 16, 1131–1139. 41. Nelson, J. I. & Frost, B. ( 1985) Exp. Brain Res. 61, 54–61. 42. Allman, J. M., Miezin, F. & McGuinnes, E. ( 1985) Perception 14, 105–126. 43. Tanaka, K., Hikosaka, K., Saito, H., Yukiem, M., Fukada, Y. & Iwai, E. ( 1986) J. Neurosci. 6, 134–144. 44. Gulyas, B., Orban, G. A., Duysens, J. & Maes, H. ( 1987) J. Physiol. (London) 57, 1767–1791. 45. Gilbert, C. D. & Wiesel, T. N. ( 1990) Vision Res. 30, 1689–1701. 46. Knierim, J. J. & Van Essen, D. C. ( 1992) J. Neurophysiol. 67, 961–980. 47. Lamme, V. A. F. ( 1995) J. Neurosci. 15, 1605–1615. 48. Westheimer, G. ( 1990) Vision Res. 30, 1913–1921. 49. Merzenich, M. M., Kaas, J. H., Wall, J. T., Nelson, R. J., Sur, M. & Felleman, D. ( 1983) J. Neurosci. 8, 33–55. 50. Merzenich, M. M., Kaas, J. H., Wall, J. T., Sur, M., Nelson, R. J. & Fellemen, D. ( 1983) J. Neurosci. 10, 639–665. 51. Merzenich, M. M., Nelson, R. J., Stryker, M. P., Cynader, M.S., Schoppmann, A. & Zook, J. M. ( 1984) J. Comp. Neurol. 224, 591–605. 52. Gilbert, C. D., Hirsch, J. A. & Wiesel, T. N. ( 1990) Cold Spring Harbor Symp. Quant. Biol. 55, 663–677. 53. Gilbert, C. D. & Wiesel, T. N. ( 1992) Nature (London) 356, 150–152. 54. Darian-Smith, C. & Gilbert, C. D. ( 1994) Nature (London) 368, 737–740. 55. Darian-Smith, C. & Gilbert, C. D. ( 1995) J. Neurosci. 15, 1631–1647. 56. Heinen, S. J. & Skavenski, A. A. ( 1991) Exp. Brain Res. 83, 670–674. 57. Kaas, J. H., Krubitzer, L. A., Chino, Y. M., Langston, A. L., Polley, E. H. & Blair, N. ( 1990) Science 248, 229–231. 58. Chino, Y. M., Smith III, E. L., Wada, H., Ridder, W. L., III, Langston, A. L. & Lesher, G. A. ( 1991) J. Neurophysiol. 65, 841–859. 59. Chino, Y. M., Kaas, J. H., Smith, E. L., III, Langston, A. L. & Cheng, H. ( 1992) Vision Res. 32, 789–796. 60. Hubel, D. H., Wiesel, T. N. & LeVay, S. ( 1977) Philos. Trans. R. Soc. London B 278, 377–409. 61. Pettet, M. W. & Gilbert, C. D. ( 1992) Proc. Natl. Acad. Sci. USA 89, 8366–8370. 62. Das, A. & Gilbert, C. D. ( 1995) J. Neurophysiol. 74, 779–792. 63. Kapadia, M. K., Gilbert, C. D. & Westheimer, G. ( 1994) J. Neurosci. 14, 451–457. 64. Volchan, E. & Gilbert, C. D. ( 1995) Vision Res. 35, 1–6. 65. Xing, J. & Gerstein, G. L. ( 1994) Vision Res. 34, 1901–1911. 66. Hirsch, J. A. & Gilbert, C. D. ( 1993) J. Physiol. (London) 461, 247–262.