near neighbors in the cortex). The behavior of the model neurons at each stage is determined by the properties of the input neurons and the way these are weighted by the initial linear combination.
A model V1 neuron sums image intensities over a local spatial region and recently past time. The linear weighting of these neurons is designed so that they respond selectively to a particular image velocity (i.e., speed and direction).
In this paper we do not attempt to make the models of V1 and MT responses biologically realistic; they are presented as mathematical abstractions, whose goal is to describe informational transformations rather than the details of the neuronal mechanisms that perform those transformations. The models can, however, be implemented with biologically reasonable mechanisms (15). Complete mathematical details are provided elsewhere (refs.12–15; E.P.S. and D.J.H., unpublished data).
Examples of the Behavior of the Model of V1 Responses. Many aspects of simple cell responses are consistent with the linear model. However, there also are important violations of linearity. One major fault with the linear model is the fact that simple cell responses saturate (level off) at high contrasts, as in Fig. 2A (20,21). The responses of a truly linear neuron would increase in proportion to stimulus contrast over the entire range of contrasts.
A second fault with the linear model is revealed by testing linear superposition. A typical simple cell responds vigorously to stimuli at the preferred orientation and direction of motion (e.g., a vertical grating moving rightward), but not at all to the perpendicular orientation/direction (e.g., a horizontal grating moving upward). Superposition is tested by displaying both stimuli at once, the upward moving grating superimposed on the rightward moving grating. According to the linear model, the response to the superimposed pair of stimuli (preferred plus perpendicular) should equal the response to the preferred stimulus presented alone (since there is no response to the upward grating alone). Surprisingly, this prediction is wrong; the response to the superimposed pair of gratings is typically about half the response to the rightward grating alone. This phenomenon is known as cross-orientation inhibition, and is an example of a variety of phenomena that can collectively be described as “nonspecific suppression.” Fig. 2C shows that adding a “masking” grating of a different orientation reduces the response elicited by an optimal grating presented alone (horizontal line) (22). The reduction in response is maximal for near-orthogonal stimuli but is evident for stimuli of other orientations.
It is the normalization stage of the normalization model that allows it to account for these data. Each neuron's linear response to the stimulus is divided by a quantity proportional to the pooled activity of a number of other neurons from the nearby cortical “neighborhood. ” Activity in this large pool of neurons partially suppresses the response of each individual neuron. Normalization is a nonlinear operation: one input (a neuron's underlying linear response) is divided by another input (the pooled activity of a large number of neurons). The effect of this divisive suppression is that the response of each neuron is normalized (rescaled) with respect to stimulus contrast. The normalization model exhibits amplitude saturation (Fig. 2B) because the divisive suppression increases with stimulus contrast. The model exhibits nonspecific suppression (Fig. 2D) because the normalization signal is pooled over many other neurons with a wide variety of tuning properties, including many that respond to orthogonal gratings.
Examples of the Behavior of the Model of MT Responses. Because of the structure of the linear portion of their receptive fields, V1 neurons can only signal the component of motion that is perpendicular to their preferred orientation. When stimulated with a complex stimulus containing multiple orientation components, a V1 neuron responds vigorously when any one of the oriented components is aligned with the neuron's preferred orientation (17,23). Fig. 3A and B show polar plots of direction tuning for V1 neurons using two stimuli: (i) drifting sinusoidal gratings and (ii) drifting plaid patterns composed of two gratings. For both real neurons and model neurons there is a unimodal response, a single preferred direction, for drifting grating stimuli. The direction tuning curves for plaids, however, are very different, with two distinct lobes. Each lobe is due to responses elicited by one of the plaid 's component gratings. The normalization model of V1 cells correctly predicts this behavior (Fig. 3 C and D).
A recombination of motion signals is required to compute and represent stimulus velocity independently of the stimulus' spatial pattern. This second stage appears to exist in area MT. For some MT neurons, the direction tuning curves are uni-