Cover Image

PAPERBACK
$51.75



View/Hide Left Panel

This paper was presented at a colloquium entitled “Neuroimaging of Human Brain Function,” organized by Michael Posner and Marcus E.Raichle, held May 29–31, 1997, sponsored by the National Academy of Sciences at the Arnold and Mabel Beckman Center in Irvine, CA.

Rapidly induced auditory plasticity: The ventriloquism aftereffect

GREGG H.RECANZONE*

Center for Neuroscience and Section of Neurobiology, Physiology and Behavior, University of California at Davis, 1544 Newton Ct., Davis, CA 95616

ABSTRACT Cortical representational plasticity has been well documented after peripheral and central injuries or improvements in perceptual and motor abilities. This has led to inferences that the changes in cortical representations parallel and account for the improvement in performance during the period of skill acquisition. There have also been several examples of rapidly induced changes in cortical neuronal response properties, for example, by intracortical microstimulation or by classical conditioning paradigms. This report describes similar rapidly induced changes in a cortically mediated perception in human subjects, the ventriloquism aftereffect, which presumably reflects a corresponding change in the cortical representation of acoustic space. The ventriloquism aftereffect describes an enduring shift in the perception of the spatial location of acoustic stimuli after a period of exposure of spatially disparate and simultaneously presented acoustic and visual stimuli. Exposure of a mismatch of 8° for 20–30 min is sufficient to shift the perception of acoustic space by approximately the same amount across subjects and acoustic frequencies. Given that the cerebral cortex is necessary for the perception of acoustic space, it is likely that the ventriloquism aftereffect reflects a change in the cortical representation of acoustic space. Comparisons between the responses of single cortical neurons in the behaving macaque monkey and the stimulus parameters that give rise to the ventriloquism aftereffect suggest that the changes in the cortical representation of acoustic space may begin as early as the primary auditory cortex.

Studies of cerebral cortical organization have demonstrated a capacity for representational reorganization after peripheral and central lesions (15) and learning paradigms (611). Implicit in these observations is that the alterations in cortical representations form the basis of the changes in perceptual and motor abilities. The bulk of the studies to date have been dedicated to comparing the posttraining electrophysiological or functional imaging data with either the pretraining data of the same subjects or data from different, naive subjects. These studies also have largely concentrated on using training paradigms that require several sessions of performing the particular task before changes in perception are demonstrated.

There also have been several descriptions of rapidly induced changes in cortical neuronal response properties, for example, by intracortical microstimulation (1214), classical conditioning paradigms (15, 16), or by artificially induced visual scotomas (17). The ability to rapidly change cortical receptive fields suggests that there is a capacity for rapidly induced changes in cortically mediated perceptions. One such demonstration of this is the ventriloquism aftereffect (18, 19), in which the perception of the location of an acoustic stimulus can be systematically altered after a period of exposure to a consistent spatial disparity between visual and auditory stimuli.

The localization of auditory stimuli in space is known to be affected by concurrent visual stimulation, which is demonstrated by the ventriloquism effect (20, 21), where the location of an auditory stimulus is perceived to originate from the same location as a simultaneously presented visual stimulus. The salience of the visual stimulus and the temporal correlation of the visual and auditory stimuli are critical for this ventriloquism effect to occur. The ventriloquism aftereffect is demonstrated when the same shift in the perception of an acoustic stimulus endures beyond the training period. This is different from the motion aftereffect, or “waterfall effect,” in two important ways. First, the ventriloquism aftereffect has a much longer duration (tens of minutes) than does the motion aftereffect (several seconds). Second, the ventriloquism aftereffect is in the same direction as during the disparity exposure, whereas the motion aftereffect results in the perception of stationary stimuli moving in the direction opposite to the direction of motion during the exposure period.

Initial studies on the ventriloquism aftereffect used either displacing prisms or a “pseudophone” to shift the relative locations of visual and auditory stimuli. Subjects localized acoustic targets before and after a period of “training” in which there was a mismatch between the spatial locations of acoustic and visual stimuli. These reports described a shift of 1–3° in the predicted direction for most subjects (18, 19).

Although the phenomenon is robust, the underlying physiological mechanisms and the potential neuronal structures that change their representation of acoustic space remain unclear. There are several possible candidates of neuronal structures, including the superior colliculus, where the superficial layers respond to visual stimuli (22) and the deeper layers respond to acoustic stimuli (2326), the auditory cortex, which is known to be necessary for assigning a spatial location to an acoustic stimulus (2731), and the parietal cortex, known for its role in spatial localization (32, 33). Measures of neuronal activity, such as functional imaging and single electrode physiology, will be necessary to investigate this question directly; however, psychophysical studies can also illuminate the most likely candidates and define how such physiological studies can be designed. This report describes a more detailed account of the ventriloquism aftereffect in human subjects, and compares the stimulus parameters creating the aftereffect with the response properties of single auditory cortical neurons recorded in a macaque monkey performing a sound localization task, to begin to address the question of which neuronal structures are likely to underlie this rapidly induced change in perception.

Demonstration of the Ventriloquism Aftereffect

Three human subjects with no known hearing deficits (aged 24–35 years at the time of testing, two males and one female,

© 1998 by The National Academy of Sciences 0027–8424/98/95869–7$2.00/0

PNAS is available online at http://www.pnas.org.

   

Abbreviations: AI, primary auditory cortex; CM, caudal-medial field.

*  

To whom reprint requests should be addressed. e-mail: ghrecanzone@ucdavis.edu.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function This paper was presented at a colloquium entitled “Neuroimaging of Human Brain Function,” organized by Michael Posner and Marcus E.Raichle, held May 29–31, 1997, sponsored by the National Academy of Sciences at the Arnold and Mabel Beckman Center in Irvine, CA. Rapidly induced auditory plasticity: The ventriloquism aftereffect GREGG H.RECANZONE* Center for Neuroscience and Section of Neurobiology, Physiology and Behavior, University of California at Davis, 1544 Newton Ct., Davis, CA 95616 ABSTRACT Cortical representational plasticity has been well documented after peripheral and central injuries or improvements in perceptual and motor abilities. This has led to inferences that the changes in cortical representations parallel and account for the improvement in performance during the period of skill acquisition. There have also been several examples of rapidly induced changes in cortical neuronal response properties, for example, by intracortical microstimulation or by classical conditioning paradigms. This report describes similar rapidly induced changes in a cortically mediated perception in human subjects, the ventriloquism aftereffect, which presumably reflects a corresponding change in the cortical representation of acoustic space. The ventriloquism aftereffect describes an enduring shift in the perception of the spatial location of acoustic stimuli after a period of exposure of spatially disparate and simultaneously presented acoustic and visual stimuli. Exposure of a mismatch of 8° for 20–30 min is sufficient to shift the perception of acoustic space by approximately the same amount across subjects and acoustic frequencies. Given that the cerebral cortex is necessary for the perception of acoustic space, it is likely that the ventriloquism aftereffect reflects a change in the cortical representation of acoustic space. Comparisons between the responses of single cortical neurons in the behaving macaque monkey and the stimulus parameters that give rise to the ventriloquism aftereffect suggest that the changes in the cortical representation of acoustic space may begin as early as the primary auditory cortex. Studies of cerebral cortical organization have demonstrated a capacity for representational reorganization after peripheral and central lesions (1–5) and learning paradigms (6–11). Implicit in these observations is that the alterations in cortical representations form the basis of the changes in perceptual and motor abilities. The bulk of the studies to date have been dedicated to comparing the posttraining electrophysiological or functional imaging data with either the pretraining data of the same subjects or data from different, naive subjects. These studies also have largely concentrated on using training paradigms that require several sessions of performing the particular task before changes in perception are demonstrated. There also have been several descriptions of rapidly induced changes in cortical neuronal response properties, for example, by intracortical microstimulation (12–14), classical conditioning paradigms (15, 16), or by artificially induced visual scotomas (17). The ability to rapidly change cortical receptive fields suggests that there is a capacity for rapidly induced changes in cortically mediated perceptions. One such demonstration of this is the ventriloquism aftereffect (18, 19), in which the perception of the location of an acoustic stimulus can be systematically altered after a period of exposure to a consistent spatial disparity between visual and auditory stimuli. The localization of auditory stimuli in space is known to be affected by concurrent visual stimulation, which is demonstrated by the ventriloquism effect (20, 21), where the location of an auditory stimulus is perceived to originate from the same location as a simultaneously presented visual stimulus. The salience of the visual stimulus and the temporal correlation of the visual and auditory stimuli are critical for this ventriloquism effect to occur. The ventriloquism aftereffect is demonstrated when the same shift in the perception of an acoustic stimulus endures beyond the training period. This is different from the motion aftereffect, or “waterfall effect,” in two important ways. First, the ventriloquism aftereffect has a much longer duration (tens of minutes) than does the motion aftereffect (several seconds). Second, the ventriloquism aftereffect is in the same direction as during the disparity exposure, whereas the motion aftereffect results in the perception of stationary stimuli moving in the direction opposite to the direction of motion during the exposure period. Initial studies on the ventriloquism aftereffect used either displacing prisms or a “pseudophone” to shift the relative locations of visual and auditory stimuli. Subjects localized acoustic targets before and after a period of “training” in which there was a mismatch between the spatial locations of acoustic and visual stimuli. These reports described a shift of 1–3° in the predicted direction for most subjects (18, 19). Although the phenomenon is robust, the underlying physiological mechanisms and the potential neuronal structures that change their representation of acoustic space remain unclear. There are several possible candidates of neuronal structures, including the superior colliculus, where the superficial layers respond to visual stimuli (22) and the deeper layers respond to acoustic stimuli (23–26), the auditory cortex, which is known to be necessary for assigning a spatial location to an acoustic stimulus (27–31), and the parietal cortex, known for its role in spatial localization (32, 33). Measures of neuronal activity, such as functional imaging and single electrode physiology, will be necessary to investigate this question directly; however, psychophysical studies can also illuminate the most likely candidates and define how such physiological studies can be designed. This report describes a more detailed account of the ventriloquism aftereffect in human subjects, and compares the stimulus parameters creating the aftereffect with the response properties of single auditory cortical neurons recorded in a macaque monkey performing a sound localization task, to begin to address the question of which neuronal structures are likely to underlie this rapidly induced change in perception. Demonstration of the Ventriloquism Aftereffect Three human subjects with no known hearing deficits (aged 24–35 years at the time of testing, two males and one female, © 1998 by The National Academy of Sciences 0027–8424/98/95869–7$2.00/0 PNAS is available online at http://www.pnas.org.     Abbreviations: AI, primary auditory cortex; CM, caudal-medial field. *   To whom reprint requests should be addressed. e-mail: ghrecanzone@ucdavis.edu.

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function subject 2) completed all of the psychophysical studies reported here. The results from two other subjects (one male, one female) that completed part of these experiments are consistent with the other three. All experiments were performed in a darkened sound- and echo-attenuated sound booth (inner dimensions: 6.5′×8.5′, Industrial Acoustics, Bronx, NY; lined with 3″ Sonex foam) with the same acoustic apparatus, stimulus delivery, and data acquisition systems as described in detail elsewhere (34). Briefly, subjects faced the center of a 15-speaker array spanning ±28° in 4° increments along the horizontal meridian. Acoustic stimuli were generated by using a Tucker-Davis Technologies system, and all parameters of the experiment were controlled by a personal computer. Acoustic stimuli consisted of 200-msec (5-msec rise/fall) 750 Hz tones and 3,000 Hz tones. Stimulus intensity (45 dB SPL) was varied over a 4 dB range for every stimulus presentation to reduce detectable differences in the speaker transformation functions. Visual stimuli consisted of dim red or green LEDs located 1° above each of the 15 speakers. Fig. 1A shows the experimental strategy for documenting the ventriloquism aftereffect. First, the ability of each subject to localize these stimuli in absolute space was measured. Second, a period of training with an eight-degree disparity between the auditory and visual stimuli was presented for approximately 20–30 min. Third, the absolute localization ability was measured again. The only visual stimuli presented during the course of the experiment were from the LEDs, and the only other acoustic stimuli were from brief instructions from the experimenter presented from behind the subject between the first and second parts of the experiment. To define the absolute localization ability (Fig. 1B), the subject’s head position was measured by using a headband mounted to a variable potentiometer that measured the head orientation. Subjects were in complete darkness except for a dim-fixation LED located at the central position (12° up) that would blink on/off until the subjects oriented their head directly toward zero degrees in azimuth and elevation (±1°). The fixation light was then extinguished, and 500–1500 msec later a tone or noise was presented from 1 of the 15 speakers of the array. The subjects turned their head to face the perceived location of the stimulus. Single trial estimates were taken as the head position measured 1,950–2,000 msec after the onset of the tone stimulus. This time period was well after minor adjustments by the subjects and before the subjects’ returned their head to the center position. Each session consisted of 15 trials at each of 9 locations, which typically lasted 15–20 min. After collection of these baseline data, subjects were exposed to a 20- to 30-min “training” period (Fig. 1C). Training stimuli consisted of paired 200-msec duration light stimuli and 200-msec acoustic stimuli identical to the stimuli used to collect the baseline data. The light and the sound were always at the same relative spatial locations, either the same (0°) or with the light offset by 8° to the right of the sound (+8). These stimuli were presented at a rate of two per second in sets of five stimuli from the same location, then a different location was randomly selected. To ensure that the subjects attended to these stimuli, either the third, fourth, or fifth stimulus of the set of five for any given location was randomly presented at 35 dB SPL (10 dB quieter) on approximately one of every five locations. The subjects pressed a button to initiate the stimulus presentations and released the button when they detected a change in the intensity of the stimulus. The instructions given to the subjects were to attend to the intensity of the acoustic stimuli and to release the button when the stimulus intensity decreased. The training period presented 2,500 stimuli, corresponding to approximately 20–30 min. Immediately after this training period, the subjects again performed the same task as for the baseline data, in which they oriented their head toward the FIG. 1. Psychophysical paradigms used to demonstrate the ventriloquism aftereffect. (A) The experimental strategy. Subjects first had their absolute localization ability measured, then were exposed for a period of 20–30 min to paired visual and auditory stimuli, then had their absolute localization ability remeasured. (B) The absolute localization paradigm used pre- and posttraining for each subject. (C) Training paradigm for the ventriloquism aftereffect. The 15-speaker array with a corresponding LED for each speaker is schematically illustrated. The subject pressed a button to initiate a trial. Throughout the course of the session a light and sound were simultaneously presented in a consistent spatial relationship. In this example the light is located two positions (8°) to the right of the sound. The subject was asked to attend to the intensity of the acoustic stimuli and to release the button on detection of a decrease in intensity. same acoustic stimulus in complete darkness. If the exposure to a mismatch in the acoustic and visual stimuli was effective in driving representational changes, there should be a shift in the localization estimates in the session immediately posttrain-

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function ing when compared with the pretraining, baseline data. The shift should be in the direction of the difference between the light and the acoustic stimulus (e.g., a rightward shift for the +8° condition). Fig. 2 shows an example of such an experiment. This figure shows the single trial estimates of Subject 1 localizing a 750 Hz tone before (long, vertical lines) and after (short, vertical lines) a training period in which the light was 8° to the right of the speaker (+8° training). The stippled vertical line shows the target location for five of the nine different target locations tested, offset in the vertical direction for clarity. These data are representative for all subjects tested in this paradigm, in that there was a clear shift toward the right of the estimates of the sound source location after the training period. A summary of the data across subjects for the two different tonal stimuli is shown in Fig. 3. This figure plots the regression of the mean estimates for each speaker used in the task before training (x axis) and after training (y axis). The dashed line shows perfect correlation (no difference) whereas the heavy solid line shows the regression line. Regardless of whether a 750 Hz tone (open symbols) or 3,000 Hz tone (solid symbols) was used, there was a significant shift of almost 8° following the +8° training condition for all subjects (different symbols of Fig. 3A). By contrast, if there was no disparity between the location of the light and the tone (0° training, Fig. 3B), there was no systematic shift of the localization estimates for either tone across subjects. The regression equations shown in the Inset of each graph also indicate that the estimates were linear across the frontal area of space tested (slopes of the regression line near 1.0). These data indicate that there is a shift in the representation of acoustic space after a relatively brief exposure to a mismatch in location between the auditory and visual modalities and no apparent shift in the absence of such mismatches. Potential Subcortical Substrates for the Ventriloquism Aftereffect Although this report concentrates on the cerebral cortical contributions to this effect, it is likely that other brain regions also will alter their representations, for example, the superior colliculus. The superior colliculus contains a spatial represen FIG. 2. Single trial estimates from subject 1 localizing a 750 Hz tone before (long vertical lines) and after (short vertical lines) 2,500 trials of the training paradigm as described in Fig. 1C. This figure shows five of the nine tested locations, offset vertically for clarity. Target location is indicated by the thick, shaded bar. FIG. 3. Summary of the aftereffect experiments across subjects (symbol type) for the 750 Hz stimulus (open symbols) and 3,000 Hz stimulus (solid symbols; see Inset). (A) Results from the +8° disparity during the training period. The mean estimates for each speaker location were taken pre- and posttraining and subjected to regression analysis. These data were significantly correlated, and the resulting regression line had a y-axis offset of +7.08°. The regression equation, r, and P values are given in the Inset. Dashed line, perfect correlation; heavy line, regression line. (B) Results from the 0° disparity during the training period. Conventions are as A. The dashed line of perfect correlation is occluded by the regression line. tation of acoustic space that is aligned with the representation of visual space, and saccadic eye movements to auditory targets is well represented in the primate superior colliculus (22–26). In spite of the clear involvement of the superior colliculus in localization and orientation behaviors, it is unlikely that the superior colliculus is the seat of the perceptual change. It is possible, however, that a realignment of the motor maps of the auditory and visual worlds would occur under these conditions. One line of evidence to suggest that the superior colliculus is not required for the aftereffect to occur is that the same change in perception can be demonstrated when there is no orienting motor response. To demonstrate the aftereffect by using a nonspatial response, subjects were tested by using a lever-response task before and after the same training paradigms. In this lever-response task, an LED was flashed for 200 msec at one of three locations (–12, 0, or +12°) randomly interleaved, followed by a 300-msec pause, followed by a 200-msec tone using the same stimulus parameters as described above (Fig. 4A). The subjects’ task was to maintain the lever in the down position and to release the lever within 500 msec if they detected that the tone and light originated from different locations. If there was no response the next trial was initiated. The probability that the light and tone would be presented from the same location on any given trial was 0.5. Subjects were tested at this task, followed by the same training paradigm as described above, and then tested immediately afterward.

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function FIG. 4. Demonstration of the ventriloquism aftereffect by using a nonspatial response paradigm. (A) The psychophysical task differed from the absolute localization paradigm in that the subjects kept their head stationary throughout the session and initiated each trial with a lever press. A single LED was illuminated for 200 msec, followed by a 300-msec pause, and then a 200-msec tone was presented. The subjects responded to a perceived difference in the spatial location between the tone and the light by releasing the lever within 500 msec, or a new trial was initiated. No feedback was given. A single session consisted of three different LED locations (–12, 0, and +12°) and nine speaker locations spanning a region ±16° of each LED. A complete session consisted of 120 trials for each matching LED and speaker location, and 15 trials at each nonmatching LED and speaker location (720 trials total). (B) Representative psychometric functions from Subject 3. The percent of trials in which the subject released the lever is plotted as a function of the distance between the light and tone stimulus locations. The heavy line shows the percent of responses before training, whereas the thin line shows the data from the same subject immediately after training. Only trials in which the LED was located at position 0 are shown for clarity. A typical example of this type of experiment is shown for Subject 3 in Fig. 4B, where the pretraining data are shown as the heavy line and the posttraining data are shown as the thin line. Using the nonspatial paradigm, we still found a measurable shift in the perception of the acoustic stimulus after the training period. These data indicate that, although a task requiring an orienting response may change the motor output of the superior colliculus, it is probably not necessary to alter the representation of space in the superior colliculus for the perceptual aftereffect to occur. Potential Cortical Substrates for the Ventriloquism Aftereffect Three cortical regions that are likely candidates to underlie this change in perception include the primary auditory cortex (AI), the secondary auditory cortical areas such as the caudal-medial field (CM), and the multisensory cortical areas such as the parietal lobe. Each of these potential cortical regions will be discussed in turn. Auditory cortical lesions result in a deficit in spatial localization ability in a variety of mammalian species (28–30). including primates (27, 31). The lesion studies in primates included at least some if not all of the adjacent auditory cortical areas as well as AI. In cats, lesions restricted to the physiologically defined boundaries of AI do produce sound localization deficits (30). In spite of the involvement of auditory cortex in sound localization behavior, there is little direct evidence of how acoustic space is represented in auditory cortex (35–40). Recordings from single AI neurons in an awake macaque monkey actively localizing these sounds from my laboratory indicate that this cortical area could contribute to this effect. The responses from a population of single auditory cortical neurons have been recorded from an awake macaque monkey (Macaca mulatta) during performance at a sound localization task. In this task, a series of 7–12 acoustic stimuli were presented from a speaker located directly opposite (90°) the contralateral ear, then the acoustic stimulus was presented at a location within 30° of the midline. The monkey’s task was to release a lever when it detected a change in the location of the stimulus. The monkey performed approximately 90% correct at this task. The activity of single neurons were recorded by using standard electrophysiological techniques, and the response during the 200-msec tone stimulus at the characteristic frequency of the studied neuron was measured for each of nine locations across the central 60° of space (±30, 15, 10, and 0°). as well as at 90° contralateral. The range of locations along the horizontal meridian that elicited at least 75% of the maximal response for each neuron was used as a measure of the spatial selectivity. Throughout the sample of recorded neurons, less than 5% (n=78) have shown spatial selectivity within 15°, and as yet none have been recorded with a spatial selectivity of less than 10°. These data are similar to those reported by others in monkeys performing a different, but similarly simple localization task (39) as well as in the anesthetized cat (36–40). Thus, the vast majority of primary auditory cortical neurons have spatial tuning selectivity greater than the 8° demonstrated to give rise to the ventriloquism aftereffect. One experimental test of the involvement of the auditory cortex is to determine the transference of the effect across frequencies. Typical primary auditory (AI) cortical neurons respond to about an octave range or less in frequency at the intensities used in the human ventriloquism aftereffect experiments. Typical examples of the tuning functions recorded from two different single neurons in the physiologically defined primary auditory cortex of an awake monkey are shown in Fig. 5. To obtain these tuning functions, 50-msec (5-msec rise/fall) tonal stimuli at 31 different frequencies and 16 different intensities (496 total stimuli) were presented at a rate of 1/sec from a speaker located 90° contralateral to the recording site. Each stimulus was presented at least two times, and the response was averaged across stimuli. The monkey was trained to depress a lever throughout the presentation of 7–12 of these stimuli, and then to release the lever when the acoustic

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function FIG. 5. Representative tuning function from two single neurons recorded in physiologically defined primary auditory cortex in a macaque monkey trained to perform a sound localization task. A total of 496 different stimuli (50 msec; 5 msec rise/fall) were presented, spanning a range of 31 frequencies (25–5,000 Hz in equal octave steps) and 16 intensities (5–75 dB SPL). Each stimulus was presented twice, and the average response of the neuron is shown as the shaded region. Regions of the plots that are not shaded represent responses that were not greater than two times the spontaneous activity. Activity was recorded for 100 msec from stimulus onset. Data for both panels are taken from single neurons recorded in the left hemisphere of one monkey in regions physiologically corresponding to cortical area AI (41–43) of auditory cortex. Arrows indicate the responses at 45 dB SPL. stimulus was presented from a location within 30° of directly in front of the monkey. The sample of AI neurons we have recorded to date with characteristic frequencies within a half-octave of 750 Hz and 3,000 Hz rarely respond to both frequencies. This implies that different and largely nonoverlapping populations of neurons are activated during the training phases of the ventriloquism aftereffect experiments, depending on the frequency used. If AI were contributing to the change in perception, there should be little if any transference between the two frequencies. We tested this directly in human subjects, asking whether there was a transference of the ventriloquism aftereffect when the testing was done at one frequency and the training was done at the other. Fig. 6 shows a summary of this experiment for all three subjects. The open symbols show the pre- and posttraining data when the subject first was tested at 750 Hz, then performed the 2,500 trials where the LED was flashed 8° to the right of the 3,000 Hz stimulus, then retested at 750 Hz. Similarly, the solid symbols show the results of the experiment where the 3,000 Hz tone was localized first, and the training consisted of the LED presented 8° to the right of a 750 Hz tone. Across subjects and frequencies, there was no clear shift in the perception of the location of the stimulus after the training period by using stimuli at a different frequency (compare Fig. 6 with Fig. 3A). The results of these experiments suggest, but of course are far from conclusive, that auditory cortical areas, even the primary auditory cortex, could contribute to the ventriloquism aftereffect. Although the spatial resolution of auditory cortical neurons is not prohibitive for this effect, it is also important that the timing of the response is appropriate so that the visual information can be incorporated. The activity of single auditory cortical neurons typically consists of a large, short-latency component followed by a much smaller, and sometimes nonexistent, sustained component. Fig. 7 shows the latency to the response as a cumulative percentage of recorded neurons. Approximately one-half of the recorded neurons had a latency to the peak onset response of less than 50 msec from stimulus onset. However, approximately one-third of the neurons maintained a significant response (25% of peak) throughout the duration of the stimulus. The timing of these responses is an important consideration because the visual input to the cerebral cortex has a much longer initial latency, generally on the order of 25–50 msec (44), than does the auditory input. These data would indicate that the peak response in auditory cortical neurons is either occurring or just completed at about the time that the visual input is just reaching the cortex. However, there is still a significant response of approximately one-third of the neurons FIG. 6. No transference of the aftereffect across frequencies. This plot shows the data collected across subjects and frequencies similar to that shown in Fig. 3A. The training period was always the frequency not used in the testing paradigms. The regression analysis indicates that there is no difference between the pre- and posttraining periods.

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function FIG. 7. Cumulative percent of neurons with the latencies of different portions of the response. The dashed line shows the cumulative percent of neurons with the latency of the first response represented on the x axis. The thin line shows the latency of the peak response. The thick line represents the cumulative percent of neurons that have decreased their activity to below 25% of the peak response by the latency indicated on the x axis. Stimuli consisted of 200-msec tone bursts (50±10 dB SPL) at the characteristic frequency of the neuron. Data were taken as the averaged response over 10–12 trials for stimuli presented at 0° azimuth. The monkey was performing a go/no-go sound localization task in which 7–12 stimuli were presented at +90°, and the monkey had to make a lever response when it detected a change in location to one of nine speakers located within ±30°. Data were analyzed only for hit trials. well after the onset response that could be modulated by visual input through projections from either extrastriate cortical areas, the superior colliculus, or the anterior and Pulvinar regions of the thalamus. It remains to be demonstrated directly, however, that the spatial tuning of auditory cortical neurons is influenced by visual stimuli. Other auditory cortical areas are also potentially contributing to the ventriloquism aftereffect. Cortical areas caudal and medial to AI have been described as having broadly tuned neurons (41–43), and it has been suggested that these cortical regions, particularly the caudal-medial region (CM), is processing sound location information (41). There is also anatomical and lesion evidence that CM receives input from AI and the frequency selectivity of neurons in CM are dependent on AI input (41, 42). Currently, it is not well understood how the perception of sound location, which clearly resides in AI in cats (30), is processed throughout the auditory cortex of primates. It is possible that the location information in AI is further processed by area CM, which in turn projects this information to higher cortical areas. Although the transference experiments suggest that the ventriloquism effect is consistent with changes in regions with sharply tuned responses, secondary and tertiary auditory cortical areas are undoubtedly contributing to the effect. To summarize the current understanding of primate auditory cortical neuron response properties, the spatial selectivity of these neurons are easily broad enough to accommodate the small disparities that give rise to the ventriloquism effect. Previous ventriloquist aftereffect studies have shown some aftereffect with disparities of 15° (18, 19). Preliminary results in my laboratory suggest that there is a distance limit to the effect, with a compelling perception that the sounds and lights are coming from different locations at disparities greater than 16°. The transference data indicate that the frequency tuning of AI neurons is in line with the lack of a transference across frequency of the ventriloquism aftereffect. However, the latency of the responses, and the general phasic nature of the responses of AI neurons, make it less likely to be directly modified by visual input. The secondary auditory cortical areas, such as CM, have not been extensively studied with regard to the spatial selectivity of the neurons, and it remains to be seen whether the spatial tuning functions of these neurons are within or beyond the range of the ventriloquism aftereffect. An alternative and equally likely explanation of the data is that the frequency-specific regions of cortex, such as AI, are providing selective spatial information that is then compared with the visual spatial information of the stimulus at some higher cortical area. The parietal areas of the cortex are likely candidates for this cross-modality processing. Although auditory responses are not as well studied as visual responses in parietal cortex, such responses do exist and parietal lobe lesions in humans produce contralesional neglect for both visual and auditory stimuli (45, 46). Recent experiments using an illusion similar to the ventriloquism effect, known as the McGurk effect (47), indicate that the parietal lobe is an important contributor to the perception of acoustic space. The McGurk effect describes an illusion derived from a mismatch between the auditory and visual inputs. For example, the presentation of the phoneme/ba/while viewing a videotape of a person speaking/ga/results in the perception of an in-between stimulus, in this example,/da/. By presenting an acoustic phoneme to the contralesional ear and the appropriate visual information to the ipsilesional visual field, patients with hemineglect show a significant improvement in their ability to identify the phoneme accurately (45, 46). In addition, recent experiments have demonstrated that there is overlap between auditory and visual spatial receptive fields in area LIP (33) of the awake macaque monkey performing a localization task. These neurons show similar latencies between the auditory and visual input, on the order of approximately 125–150 msec. Area LIP and other parietal regions with such intermodality processing are likely candidates for the plastic alignment of auditory and visual space. Cortical Contributions to Sound Localization The ventriloquism aftereffect demonstrates a visually driven change in the perception of acoustic space. The transference data suggest that this change in perception is restricted to the frequency of the training, implicating the narrowly tuned cortical regions such as AI. To date, however, there is no evidence of a topographic representation of acoustic space in the auditory cortex. Alternatives to a point-by-point representation of acoustic space include population responses (e.g., 40) or a temporally based coding mechanism (35). The ability to rapidly alter the perception of acoustic space may provide a valuable tool in understanding how the cerebral cortex encodes auditory space. The psychophysical, lesion, and physiological data combined suggest that acoustic space is represented in both auditory and parietal cortical areas. The ventriloquism aftereffect may be initiated in auditory subcortical areas or primary auditory cortex and simply reflected throughout the cortical hierarchy. Alternatively, a population or temporal coding scheme in AI may remain unaltered, with neurons in secondary or tertiary cortical areas modifying their responses to give rise to the altered perception. A third alternative is that higher-order cortical areas processing multimodal information is the neuronal locus of the changed perception. If changes in cortical neuron responses ultimately are demonstrated in primary and/or secondary auditory cortical areas,

OCR for page 107
Colloquium on Neuroimaging of Human Brain Function it will provide strong evidence that in this instance, cortical plasticity is mediated through a “top-down” approach. There are no examples in the literature, to my knowledge, that visual stimuli effect the response of auditory cortical neurons, although such inputs may exist via projections from the anterior thalamus or Pulvinar. This class of studies therefore could represent a powerful animal model of cortico-cortical interactions and provide insights into how the activity of the cerebral cortex gives rise to perception. Summary These data, taken together, suggest that normal adult human subjects are able to alter their cortical representations over a relatively brief time course and effectively alter their perception of at least acoustic space. The transference data suggest that the neuronal representation of acoustic space is frequency-specific. This further suggests that either the frequency-selective regions of auditory cortex, such as AI, are contributing to the effect, or that higher-order cortical areas, such as the secondary auditory cortical area CM and the parietal lobe, are adjusting the inputs from AI to better coincide with the visual spatial information. It remains to be seen whether the representation of space is modifiable in primary auditory cortex of the primate, similar to the modifiability of the representation of frequency (6). I thank M.Phan, D.Guard, and M.Sutter for their contributions to this report. This work was funded by National Institutes of Health Grant DC02371–02, the Sloan Foundation, and the Klingenstein Foundation. 1. Kaas, J.H. (1991) Annu. Rev. Neurosci. 14, 137–167. 2. Merzenich, M.M., Kaas, J.H., Wall, J.T., Nelson, R.H. & Felleman, D.H. (1983) Neuroscience 10, 639–665. 3. Nudo, R.J., Wise, B.M., SiFuentes, F. & Milliken, G.W. (1996) Science 272, 1791–1794. 4. Pons, T.P., Garraghty, P.E. & Mishkin, M. (1988) Proc. Natl. Acad. Sci. USA 85, 5279–5281. 5. Ramachandran, V.S., Rogers-Ramachandran, D. & Stewart, M. (1992) Science 258, 1159–1160. 6. Recanzone, G.H., Schreiner, C.E. & Merzenich, M.M. (1993) J. Neurosci. 13, 87–103. 7. Recanzone, G.H., Merzenich, M.M., Jenkins, W.M., Grajski, K.A. & Dinse, H.A. (1992a) J. Neurophysiol. 67, 1031–1056. 8. Recanzone, G.H., Merzenich, M.M. & Jenkins, W.M. (1992) J. Neurophysiol. 67, 1057–1070. 9. Recanzone, G.H., Merzenich, M.M. & Schreiner, C.E. (1992) J. Neurophysiol. 67, 1071–1091. 10. Nudo, R.J., Milliken, G.W., Jenkins, W.M. & Merzenich, M.M. (1996) J. Neurosci. 16, 785–807. 11. Karni, A., Meyer, G., Jezzard, P., Adams, M.M., Turner, R. & Ungerleider, L.G. (1995) Nature (London) 377, 155–158. 12. Nudo, R.J., Jenkins, W.M. & Merzenich, M.M. (1990) Somatosens. Motor Res, 7, 463–483. 13. Recanzone, G.H., Merzenich, M.M. & Dinse, H.R. (1992) Cerebral Cortex 2, 181–196. 14. Dinse, H.R., Recanzone, G.H. & Merzenich, M.M. (1993) Neuroreport 18, 173–176. 15. Bakin, J.S. & Weinberger, N.M. (1990) Brain Res. 536, 271–286. 16. Weinberger, N.M. (1995) Annu. Rev. Neurosci. 18, 129–158. 17. Pettet, M.W. & Gilbert, C.D. (1992) Proc. Natl. Acad. Sci. USA 89, 8366–8370. 18. Canon, L.K. (1970) J. Exp. Psychol. 84, 141–147. 19. Kelso, J.A.S., Cook, E., Olson, M.E. & Epstein, W. (1975) J. Exp. Psychol. Human Percept. Perf. 1, 237–245. 20. Howard, I.P. & Templeton, W.B. (1966) Human Spatial Orientation. (Wiley, New York), p. 361. 21. Pick, H.L., Warren, D.H. & Hay, J.C. (1969) Percept. Psychophys. 6, 203–205. 22. Goldberg, M.E. & Wurtz, R.H. (1972) J. Neurophysiol. 35, 542–559. 23. Middlebrooks, J.C. & Knudsen, E.I. (1984) J. Neurosci. 4, 2621–2634. 24. Jay, M.F. & Sparks, D.L. (1984) Nature (London) 309, 345–347. 25. Sparks, D.L. & Nelson, J.S. (1987) Trends Neurosci. 10, 312–317. 26. Wallace, M.T. & Stein, B.E. (1996) Prog. Brain Res. 112, 301–311. 27. Sanchez-Longo, L.P. & Forster, F.M. (1958) Neurology 8, 119–125. 28. Neff, W.D., Fisher, J.F., Diamond, I.T. & Yela, M. (1956) J. Neurophysiol. 19, 500–512. 29. Kavanagh, G.L. & Kelly, J.B. (1987) J. Neurophysiol. 57, 1746–1766. 30. Jenkins, W.M. & Merzenich, M.M. (1984) J. Neurophysiol. 52, 819–847. 31. Heffner, H.E. & Heffner, R.S. (1990) J. Neurophysiol. 64, 915–931. 32. Motter, B.C. & Mountcastle, V.B. (1981) J. Neurosci. 1, 3–26. 33. Mazzoni, P. Bracewell, R.M., Barash, S. & Andersen, R.A. (1996) J. Neurophysiol. 75, 1233–1241. 34. Recanzone, G.H., Makhamra, S.D.D.R. & Guard, D.C. (1997) J. Acoust. Soc. Amer., in press. 35. Middlebrooks, J.C., Clock, A.E., Xu, L. & Green, D.M. (1994) Science 264, 842–844. 36. Imig, T.J., Irons, W.A. & Samson, F.R. (1990) J. Neurophysiol. 63, 1448–1466. 37. Rajan, R., Aitkin, L.M. & Irvine, D.R.F. (1990) J. Neurophysiol. 64, 888–902. 38. Middlebrooks, J.C. & Pettigrew, J.D. (1981) J. Neurosci. 1, 107–120. 39. Benson, D.A., Hienz, R.D. & Goldstein, M.H., Jr. (1981) Brain Res. 219, 249–267. 40. Eisenman, L.M. (1974) Brain Res. 75, 203–214. 41. Rauschecker, J.P., Tian, B., Pons, T. & Mishkin, M. (1997) J. Comp. Neurol. 382, 89–103. 42. Morel, A., Garraghty, P.E. & Kaas, J.H. (1993) J. Comp. Neurol. 335, 437–459. 43. Merzenich, M.M. & Brugge, J.F. (1973) Brain Res. 50, 275–296. 44. Maunsell, J.H. & Gibson, J.R. (1992) J. Neurophysiol. 68, 1332–1344. 45. Soroker, N., Calamaro, N. & Myslobodsky, M.S. (1995) J. Clin. Exp. Neuropsychol. 17, 243–255. 46. Driver, J. (1996) Nature (London) 381, 66–68. 47. McGurk, H. & MacDonald, J. (1976) Nature (London) 263, 747–748.