National Academies Press: OpenBook

Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science (1990)

Chapter: The Photoreceptor Mosaic as an Image Sampling Device

« Previous: The Distribution of Cones in the Primate Retina
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 117
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 118
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 119
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 120
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 121
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 122
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 123
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 124
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 125
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 126
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 127
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 128
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 129
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 130
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 131
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 132
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 133
Suggested Citation:"The Photoreceptor Mosaic as an Image Sampling Device." National Research Council. 1990. Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science. Washington, DC: The National Academies Press. doi: 10.17226/1570.
×
Page 134

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

The Photoreceptor Mosaic as an Image Sampling Device JOH N I. YEI,I,OTT The fact that vision begins with a spatial sampling of the retinal image by discrete photoreceptors was first recognized in the 1850s, when Heinrich Muller combined psychophysics and anatomy to show that the light-sensitive elements of the retina must be the rods and cones (Brindley, 1970~. That recognition touched off a debate about the perceptual implications of "mo- saic vision," in particular its implications for spatial resolution. Apparently Muller himself initially argued that for two points to be seen as separate it was sufficient for them simply to stimulate two different cones (L.N. Thibos, personal communication, 1988~. But his rival, Bergmann (1858), pointed out that in that case a continuous line would be indistinguishable from a row of dots! ~ Bergmann, and to Helmholtz (1860), it was obvious that for two points to be resolved there must be a third unstimulated cone between two stimulated ones. And on that basis Helmholtz was satisfied that the limits of visual acuity were consistent with the anatomical dimensions of the receptor mosaic: at his own resolution limit for gratings, 60 cycles/degree, alternate light and dark bars would stimulate alternate rows of cones. Once this point was settled, the visual consequences of receptor sam- pling seem to have attracted surprisingly little attention for quite a long time. But in this century the general topic of signal sampling has as- sumed great practical importance, and mathematicians and engineers have developed sophisticated tools for analyzing the information transmission properties of arbitrary sampling schemes. Recently visual scientists have begun to apply these tools to the photoreceptor mosaic to analyze its prop- erties as an image sampling device. This paper reviews some of what has been learned from this analysis. For an engineer setting out to build an image sampling device, the natural starting point is Shannon's (1949) sampling theorem. I will start by 117

118 JOHN I. YELLOIT reviewing the assumptions and consequences of that theorem and then will describe the image sampling parameters of the cones and point out how they violate Shannon's assumptions. Work by anatomists and psychophysicists over the past decade has shown that the cone mosaic is quite unlike the kind of sampling device envisioned by Shannon from his perspective its construction seems much too sloppy. I think this suggests that the receptor mosaic is not really designed with Shannon's theorem in mind (so to speak), but instead follows a different blueprint based on some other approach to recovering continuous images from discrete samples. Part of this paper deals with image recovery algorithms that do not require the kind of precise architecture assumed by Shannon's theorem and therefore seem better suited to the kind of image sampling found in the primate retina. SHANNON'S THEOREM We are concerned with the sampling of two-dimensional signals- retinal images but to illustrate the basic ideas of sampling theory it is convenient to begin with one-dimensional signals, such as temporal wave- forms (see Figure 1~. Shannon (1949) originally stated his theorem for such signals: "If a function fate contains no frequencies higher than W cycles/sec it is completely determined by giving its ordinates at a series of points spaced 1/2W seconds apart." In other words, no information is lost by discrete sampling, provided we sample periodically (as in Figure 1A) and the highest frequency in the signal is at most one-half the sampling rate. That frequency cutoff, half the sampling rate, is called the Nyquist limit of the sampling array. The Nyquist limit is a key concept in a moment we will see what it means to violate it. "Completely determined" in the theorem means that the continuous input signal can be exactly reconstructed from its sample values. Shannon showed that this can be achieved by convolving the sample values with a function of the form sin 2 or Wt~ ~ Wt the so-called sine function. I will refer to this process as sine interpolation. The sine function arises here because it produces perfect low-pass filtering in the frequency domain: sine interpolation kills all frequencies above the Nyquist limit and passes all lower frequencies intact. So another way to describe the reconstruction process is to say that the input signal is recovered by low-pass filtering of the sample values. ~ visualize the process of image reconstruction by sine interpolation, one can imagine that each sample point creates a rippling point-spread function whose value equals the signal at that point and zero at all the other sample points, as illustrated in Figure 1B. If we add up all these point-spread functions, we get the original signal.

c B PHOTORECEPTOR MOSAIC AS AN WAGE SAPLING DEVICE 119 R - it 1.l'~:'i'l!'''~ _~. ~~ ~° L: :.t~tt/: >~` \'4 ~0 .~: ~ ~ FIGURE 1 Shannon's sampling theorem in one dimension. NOTE: (A) Continuous signal sampled at equally spaced points. (B) Sinc interpolation functions around the sample points. (For clarity only one function is shown as a continuous curve.) (C3 Postsampling reconstn~ction of a sinusoid at half the Nyquist frequency. The input signal is shown as a continuous curve; its reconstruction is shown as a dotted curve. (D) Aliasing: the input signal is a sinusoid whose frequency is 1.5 times the Nyquist limit of the sampling array. Sinc interpolation yields a reconstruction (dotted curve) whose frequency is one-half the Nyquist limit. Shannon's theorem deals strictly with bandlimited signals and infinite sampling arrays. In the real world, of course, we never encounter either one. (In principle, a band-limited signal must have infinite duration.) So it is natural to wonder how well sine interpolation works for finite-duration signals sampled by finite sets of points. Figure 1C shows a sinusoidal signal at half the Nyquist frequency that has been sampled at 11 points and reconstructed by sine interpolation. The results here are essentially perfect,

120 JOHN I. YEl l OlT and that is quite characteristic: Shannon's reconstruction algorithm works very well on a local scale for signals below the Nyquist limit. Figure ID shows what happens when the input signal exceeds the Nyquist limit. Here a sinusoid at 1.5 times the Nyquist frequency has been sampled and reconstructed by low-pass filtering. The result is a sinusoid whose frequency is half the Nyquist limit. This is an example of aliasing distortion: an input frequency above the Nyquist limit creates a spurious low frequency in the postsampling reconstruction. Here it is obvious from the figure why this occurs: the supra-Nyquist sinusoid and its sub-Nyquist alias create exactly the same sample values, so low-pass filtering of those values must produce the same result in both cases. The same would be true for any operation we performed on the samples. Shannon's theorem generalizes readily to two dimensions in other words, to image sampling (e.g., Goodman, 1968~. There it assumes that the sampling array is a perfect lattice and tells us that any image can be reconstructed from its sample values provided it contains no spatial frequencies higher than a cutoff a Nyquist limit that depend on the spacing of the sample points. Roughly speaking, that limit again is half the sampling rate. And here again the input signal is reconstructed by low-pass filtering. In this case that filtering is accomplished by convolving the sample values with a two-dimensional analog of the sine function a Mexican sombrero with multiple ripples in its brim. And just as in one dimension, if we sample a spatial frequency higher than the Nyquist limit, we get sample values identical to those-of some lower-frequency alias, when we reconstruct by low-pass filtering it is the alias that survives. The only new wrinkle in two dimensions is that the alias generally has a different orientation than the input as well as a different frequency. Figure 2 illustrates this by showing a grating seen through a hexagonal lattice of holes whose Nyquist limit is less than the grating frequency. A point to note for future reference is the perfect regularity of the alias pattern. Our concern is with violations of Shannon's assumptions by the cone mosaic. 1b understand the effects of those violations, one needs a general idea of the proof of the theorem, especially of the role played in that proof by the Fourier spectrum of the sampling array. Figure 3 illustrates the key points. Panel A represents a two-dimensional array of sample points a forest of delta functions. Image sampling is a matter of multiplying the image times such an array, and in the spectral domain that means we are convolving the spectrum of the image (Figure 3B) with that of the array. When the sampling array is a perfect lattice, its spectrum is also a lattice, and the spacing of the spectral lattice points is inversely proportional to the spacing of the sample points (i.e., it is proportional to the sampling rate). Convolution creates multiple replicas of the image spectrum, one centered at each point of the array spectrum (Figure 3C). When the sampling rate

PHOTORECEPTOR MOSAIC AS AN EDGE SAPLING DEICE 121 FIGURE 2 Aliasing in two dimensions. NOTE: A hexagonal array (seen alone on the left) samples a grating (seen alone on the right) whose frequency exceeds its Nyquist limit. After sampling (middle) the grating appears to have a lower frequency and a different orientation. (gibe figure is panted in reverse contrast.) is sufficiently high these replicas do not overlap, and the one centered on zero frequency is a perfect copy of the image spectrum. So we can isolate it by low-pass filtering and recover the image by Fourier inversion. But if the sampling rate is too low, the replicas do overlap, and no clean copy can be isolated by spatial filtering. So the critical assumption of Shannon's theorem is really the spatial regularity of the sampling arrays, equivalently, the lattice-like nature of its spectrum. It is this regularity that creates the discrete spectral replicas needed for perfect image reconstruction by low-pass filtering. SAMPLING PARAMETERS OF THE FOVEAL CONE MOSAIC Now we turn to the parameters of retinal image sampling by the cones. I will deal first with the fovea and then with the rest of the retina. Our concerns are (1) the spatial frequency bandwidth of the image, (2) the Nyquist limit implied by cone density, and (3) the spatial regularity of the mosaic.

122 JOHN I. YELLOIT A - G(J`. fy) it- A--''-- ,~ -1 >~ , >_. ~ , ~ ~ ,y \ ,'1) / /~- \~--' ~ I ~ ~ 'A 7-~--T , FIGURE 3 Key concepts in the proof of Shannon's theorem for two-dimensional sampling. NOTE: (A) Two-dimensional lattice of sample points represented as delta functions. Here the lattice is rectangular, with By spacings MY. The Fourier spectrum of this lattice is another lattice with reciprocal spacings 1/X, 1/Y. (B) G(f=, fly) is the spectrum of some two-dimensional signal grays. (C) The spectrum of g(~;y) after sampling consists of multiple replicas of G. one centered at each point in the spectrum of the sampling array. SOURCE: Adapted from Goodman (1968~.

PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE 123 For the fovea, Campbell and Gubisch's (1966) line-spread measure- ments indicate that the spatial bandwidth of normal retinal images is on the order of 60 cycles/degree. Osterberg's (1935) classic measurements of cone density in a single human eye implied a Nyquist limit for the foveal center of almost exactly 60 cycles/degree. Recently, Curcio and her colleagues examined several eyes and found foveal Nyquist limits ranging from 50 cycles all the way up to 85 (Curcio et al., 1987~! But their average value is 65 cycles, so we can still say that the Nyquist limit of the foveal cones is on the order of 60 cycles/degree. Now if the foveal cones formed a perfect spatial lattice, Shannon's theorem would allow us to say that in principle no information is lost there by receptor sampling. Figure 4 shows that foveal cones in the primate retina can achieve a high degree of spatial regularity over distances on the order of one-tenth of a degree. The top panel shows the center-point positions of roughly 100 cones in the center of the fovea of a macaque monkey, carefully measured by Hirsch and Hylton (1984~. The cones were sectioned at the inner-segment level, near their optical entrance aperture (Miller and Bernard, 1983), so the picture accurately represents the effective sampling regularity of the mosaic. The bottom panel of Figure 4 shows the Fourier spectrum of this array of cones (Ahumada and Yellott, 1985~. We see that the spectrum is quite lattice-like out to very high frequencies. So over small regions containing a 100 or so cones, the lattice assumption of Shannon's theorem seems to be well satisfied by the foveal receptor mosaic. But this high degree o~ sparest Tory us nor ma~n~a~neu over larger distances. Figure 5 (panels C and D) shows the aliasing patterns created when a 1-de" section of monkey foveal cones is made to sample gratings at frequencies above its nominal Nyquist limit (Williams, 1985~. The section itself (shown in Figure SA) is the same one from which Hirsch and Hylton selected the 0.1-de" patch of cones whose positions are shown in Figure 4. We see that there is very conspicuous aliasing here, but it does not have the long-range periodic character that would be produced by a perfect lattice (cf. Figure 2~. And when the normal 60-cycle bandwidth of the eye is bypassed by interferometry, as Williams (1985) has done, the aliasing patterns one perceives have the same ragged quality as those we see here. So both anatomy and psychophysics indicate that spatial regularity in the human foveal cone mosaic is preserved only over distances on the order of one-tenth of a degree. In the fovea then the spatial bandwidth of the retinal image matches the Nyquist limit implied by overall cone density, and the cone mosaic is locally regular enough to create narrowband aliasing of super-Nyquist . . . ~

124 JOHN I. YELLOIT . · . . . ~ . At. By.: .6 - :~ As. :~ . . :~; :. ~i.s`: I. i :~ At. - ~o ~ . ~ UP i: I. s .. 6: Hirsch & Hylton Data Fourier Amplitude Transform · · . Hirsch & Hylton Data Cone Positions is. A ' -I Us - _ - :. id.. . : . .. as, .> *s . FIGURE 4 Spatial regularity of the central foveal receptor lattice. NOTE: Top: dots mark the positions (centerpoints) of cone inner segments in the central fovea of a monkey (Macaca fasiculans) as measured lay Hirsch and Hylton (1984~. The mean interpoint distance is 3 ,um. Section width corresponds to about 0.1 deg on the human retina. Bottom: Fourier amplitude spectrum of the point array on the left.

PHOTORECEPTOR MOSAIC AS AN IM4GE SAMPLING DEVICE B 125 c tD' FIGURE 5 Gratings sampled By the cones in a monkey fovea. NOI,E: (A) Cone inner- segment positions (at the external limiting membrane) in the fovea of Macaca fasiculans. Section width corresponds to 1 deg on the human retina. (B) Foveal cones sampling at 40 pycles/degree (i.e., sub-Nyquist) square wave. (C) 80 pycles/degree. (D) 110 pyclesldegree. SOURCE: Adapted from Williams (1985~. gratings. But the mosaic overall certainly does not have the kind of perfect lattice structure assumed by Shannon's theorem. IMAGE RECOVERY FROM IRREGULAR SAMPLES: YEN'S THEOREM Does this lack of spatial regularity imply that foveal receptor sampling necessarily loses information? In principle, the answer is no—provided the actual positions of the sample points are known. Perhaps the best-known mathematical result in this connection is Yen's theorem (1956, Theorem A, which provides an explicit reconstruction algorithm for the case in which any finite number of sample points have been arbitrarily displaced away from their lattice positions. Yen's theorem assumes that the input signal contains no frequencies higher than the nominal Nyquist limit (i.e., the Nyquist limit implied by the average sampling rate).

126 R B D JOHN L YELLOIT ~ ~ C t! ~ Hi. ./ I I I I ~ 1 1 1 1 1~ 11 ~ '1 i: "A i.,,,,,.t , ~ r _ ttttit 9754 t let I ' ~ ,It I ', t I ~. 1, t I t '. ' I I ~ I ~ , t I Jew tt~tt I flit i. 1 1` t I t 11 , . ~ FIGURE 6 Yen's theorem in one dimension. NOTE: (A) Continuous signal sampled at irregularly spaced points. (B) Yen interpolation functions for sample points 3, 6, and 9. (The curve for point 3 is dotted for visibility.) (C) Yen reconstruction of a sinusoid at one-half the nominal Nyquist frequency. (D) Yen reconstruction of a sinusoid at 1.5 times the Nyquist frequency.

PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE 127 Yen's sampling theorem, like Shannon's, assumes an infinite number of sample points and strictly bandlimited signals, so we need assurance that his algorithm actually works under realistic conditions. Figure 6 demonstrates that it does and also illustrates the mechanics of Yen reconstruction. Panel A shows 11 irregularly spaced sample points. Like Shannon reconstruction, Yen reconstruction is based on interpolation functions centered at the sample points. But instead of using the same sine function for every point, Yen's algorithm requires each point to have its own idiosyncratic interpolation function one that depends on the distances between that point and all the others. Panel B shows the interpolation functions for three of the points from panel ~ One can see that Yen interpolation functions look like distorted sine functions drawn by a careless draftsman: they are often distinctly asymmetric, and no two are exactly alike. Like Shannon's algorithm, Yen reconstruction consists of weighting each sample point's interpolation function by the sample value (i.e., the input signal value) at that point and then summing all the interpolation functions. Figure 6C shows the results for a sinusoid whose frequency is one-half the nominal Nyquist limit. The results are as good as the comparable Shannon reconstruction shown in Figure 1C (i.e., essentially perfect). Figure 6D shows what happens when Yen's algorithm is applied to a si- nusoid whose frequency exceeds the nominal Nyquist limit. Here the results are quite unlike those produced by regularly spaced sample points (e.g., in Figure 1D): instead of aliasing creating a perfect low-frequency (sub- Nyquist) sinusoid, the reconstructed signal looks like a sinusoid corrupted by low-frequency noise. This is a general feature of Yen reconstruction: as the spacing of the sample points becomes more irregular, the less the aliases of supra-Nyquist sinusoids look like perfect low-frequency sinusoids and the more they look like broadband noise. This property of Yen re- construction is at least qualitatively consistent with what one sees when supra-Nyquist gratings are imaged on the fovea by interferometry. In the vision literature Yen's algorithm has sometimes been treated as one that could not be readily implemented by a visual system, and this has been used as an argument that irregular image sampling must have very deleterious consequences for vision (French et al., 1977~. This idea may have been prompted in part by the fact that in print the analytic expressions for Yen interpolation functions look horribly complicated. But in reality the theorem is based on a simple idea, and the computations involved are all straightforward linear operations. The idea is that if we had sampled an appropriately bandlimited signal at regularly spaced points, we could have used sine interpolation to find its values at any other set of points. So given the signal values at the points where we actually did sample, we can interpolate backward to find the

128 JOHN I. YELL07T values that must have occurred at the lattice points and then use them to reconstruct the entire signal by Shannon's method. This process turns out to be equivalent to erecting a tailor-made interpolation function around each of the actual sample points a function that is a weighted sum of sine functions. The only trick is to find the proper weights, and these can be readily calculated (by a matrix inversion) if the actual positions of the sample points are known. It is important to note that these weights need to be computed only once for any given sampling array. Thereafter, Yen reconstruction is com- putationally just as simple as Shannon reconstruction. So Yen's algorithm seems quite compatible with a visual system whose sampling elements do not change position from moment to moment, like blades of grass in the wind, but that form a stable irregular mosaic that lasts a lifetime. EXTRAFOVEAL IMAGE SAMPLING Yen's theorem allows us to say that if the visual system can learn the actual positions of its foveal cones, then despite spatial disorder in the cone mosaic, receptor sampling of normal foveal images need not impose any loss of information. The critical point is that in the fovea the average sampling rate matches the spatial frequency bandwidth imposed by the optics of the eye. Outside the fovea, however, there is a significant mismatch: the cones undersample the retinal image. Osterberg (1935) found that cone density decreased very rapidly with eccentricity in the single retina he examined, and the recent work of Curcio et al. (1987) indicates that Osterberg's density versus eccentricity curve is generally valid across individuals for eccentricities beyond about 1 deg. Thus, in a typical human retina the nominal Nyquist limit of the cones at 4-de" eccentricity has dropped from its foveal value of 60 cycles/degree to about 20. But the available evidence indicates that the spatial bandwidth of the retinal image has not decreased by anything like the same amount. In fact the line-spread measurements of Jennings and Charman (1981) suggest that at 4 deg the bandwidth is still around 60 cycles/degree, so in that region the cones undersample the retinal image by a factor of 3! That estimate may be somewhat extreme, because increased retinal thickness outside the fovea complicates the problem of estimating the true spatial bandwidth of the image at the level of the receptors. But there seems to be little doubt that image bandwidth decreases more slowly than cone density, so that the retinal image is significantly undersampled outside the fovea. Recent psychophysical results support this conclusion: Still and Thibos (1987) found that at 20 deg in the periphery the gratings on a CRT screen can be discriminated from uniform fields up to 22 cycles/degree,

PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE 129 while the nominal Nyquist limit at that eccentricity is about 10 cycles. In other words, the retinal image bandwidth at 20 deg must be at least twice the local Nyquist limit of the cones. Unlike cone density, which decreases continuously with retinal ec- centricity, the spatial regularity of the cones declines only over the first two to three degrees and then appears to reach a stable state that pre- vails throughout the rest of the retina. This fact is easy to appreciate by examining the Fourier spectra of small sections of cones from differ- ent eccentricities (Yellott, 1983a) or from comparable statistical analyses (Hirsch and Miller, 1987~. Figure 7 shows, on the left, the cone positions in a small section of monkey retina at 2.5~eg eccentricity (from Hirsch and Miller, 1987) and, on the right, the Fourier spectrum of this cone array. Clearly this spectrum looks nothing like the perfect lattice of deltas required by Shannon's theorem. Instead it contains a single delta at the origin, surrounded by a circular island of empty space that ends abruptly in a sea of noise. This kind of "desert island" spectrum characterizes local sections of the cone mosaic at all eccentricities beyond about 2.5 deg. As cone density decreases beyond that point, the radius of the desert island decreases, but it always has a value that is approximately twice the nominal Nyquist frequency implied by the local density. [A desert island spectrum indicates that the extrafoveal cones are packed essentially at random but are subject to a constraint on the minimum cone-cone distance (Yellott, 1983b).] Figure 8 illustrates the spectral consequences of desert island sampling for a sinusoidal grating. The top represents the spectrum of the sampling array and the spectrum of the grating and simply reminds us that the spec- trum of the grating after sampling will be the convolution of the two. On the bottom we see the postsampling spectrum for cases in which the grating frequency falls below or above the nominal Nyquist limit. Three points can be observed here: (1) For sub-Nyquist frequencies the postsampling spectrum contains deltas that fall in a relatively noise-free region of fre- quency space. In other words, sub-Nyquist images do not mask themselves. (2) For frequencies above the Nyquist limit there is no concentration of aliased energy at any single low frequency. Instead the aliased energy is scattered out into broadband noise at all orientations. Consequently, alias- ing here will not take the form of periodic moire patterns but instead will look like broadband noise. (3) The postsampling spectrum retains a strong concentration of spectral power at the original input frequency, even when that frequency falls above the Nyquist limit. This means that an irregular sampling array offers the potential for vision beyond the Nyquist frequency. Figure 9 illustrates the last two points using an actual array of ex- trafoveal cones. At the top the cones are shown sampling horizontal and vertical gratings at 1.25 times the Nyquist limit. Below are the power

9' * * * ~ 2.53 130 a e (Deg.) 2 7R JOHN I. YELLOIT Section Spectrum Hirsch & Miller Data Bar = 38 cvales/dec P. r .~ FIGURE 7 Spatial disorder in the extrafoveal cone mosaic. NOISE: Left side: dots mark the positions of extrafoveal cone inner segments in a monkey retina at Z5 deg eccentricity (as measured By Hirsch and Miller, 1987~. Right side: Fourier spectrum of the cone array (computed by A.J. Ahumada). The horizontal bar across the desert island portion of the spectn~m marks a distance equal to twice the Nyquist frequency implied by local cone density. spectra of the sampled images. We see that there is no hint of the kind of narrowband aliasing (i.e., moire patterns) produced by the foveal cones (cf. Figure 5~. Aliasing here takes the form of broadband noise. And we see in the postsampling spectra that there is easily enough energy concentrated at the input frequencies to allow us to determine which spectrum is which. Of course for the visual system to make such a discrimination, it must have orientation-sensitive mechanisms tuned to spatial frequencies that exceed the local Nyquist limits of its cones. Apparently our visual system does have such mechanisms, because Williams and Coletta (1987) have recently shown that observers can discriminate grating orientation at frequencies up to 1.5 times the nominal Nyquist limit out to lO~eg eccentricity. (At higher frequencies grating orientation can no longer be

PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE - R ~<~'2 ~ itch I_ , ~ o i' -K -~i +~t OR 131 1 1 1 -id o F~ FIGURE 8 Schematic illustration of the effects of sampling a grating with a point array whose spectrum has the desert island form found in the primate extrafoveal cone mosaic. NOTE: Top left: idealized desert island spectrum whose noise-free island has radius R (]V2 = nominal Nyquist frequency). Top right: spectrum of a sinusoid with frequency F. Bottom left: spectrum of the sampled grating when F < RL2. Here the grating energy at ~rF escapes masking by the sampling noise, all of which falls outside the Nyquist region. Bottom right: postsampling spectrum when F > R/2. Aliased energy in the form of broadband noise is widely scattered over the Nyquist region, with no concentration at any single sub-Nyquist frequency, and sharp spikes remain at the input frequency points OF. identified, but spatial contrast can still be detected in the form of broadband noise.) How can the visual system make effective use of the information pro- vided by a highly disordered receptor mosaic that undersamples the retinal image by a factor of 2 or more? Yen's theorem provides no guidance here, but recently Chen and Allebach (1987) showed that for bandlimited images undersampled by irregular point arrays, the least-squares reconstruction al- gorithm under quite general assumptions is very similar to Yen's. Like Yen reconstruction, Chen-Allebach reconstruction is based on tailor-made in- terpolation functions for each sample point (quite like those in Figure 6B),

32 JOlIN I. YELLOlT FIGURE 9 Visibility of spatial frequencies beyond the Nyquist limit in extrafoveal retina. NOTE: Top: one degree section of extrafoveal rhesus monkey cones (3.8~eg eccentricity sampling horizontal and vertical gratings whose frequent is 1.25 times the nominal Nyquist frequency of the cone array. Bottom: Fourier power spectra (optical transforms) of the sampled gratings. SOURCE: Adapted from Williams and Coletta (1987~. and the construction of those functions requires knowledge of the actual sampling position. But here again, once the interpolation functions have been created, they are good for life, and the reconstruction of any given input image is computationally just as simple as it would be for spatially regular sampling below the Nyquist limit. It is too early to say how well the Chen-Allebach algorithm will mesh with the facts of extrafoveal vision, but I think it is a promising direction for exploration. J. Ahumada and I are currently studying mechanisms by which higher visual centers could learn

PHOTORECEPTOR MOSAIC AS AN IMAGE SAMPLING DEVICE 133 the receptor positions and compute the appropriate interpolation functions (Ahumada and Yellott, 1988). ACKNOWLEDGMENTS I thank L.N. Thibos for sharing information on the work of Bergmann (1858), M. D'Zmura for providing his English translation of Bergmann's paper, D.R. Williams for permission to reproduce Figures 5 and 9, and J. Ahumada for his long-term collaboration. This research was supported in part by the National Aeronautics and Space Administration under joint research interchange NCA2-5. REFERENCES Ahumada, A.J., and J.I. Yellott 1985 A model for foveal photoreceptor placing. Investigative Ophthalmology and Visual Science 26 (Suppl.): 11 (abstract). 1988 A connectionist model for learning receptor positions. Investigative Ophthal- mology and Visual Science 29 (Suppl.~:58 (abstract). Bergmann, C. 1858 Anatomisches und physiologisches uber die netzhaut des auges. Zeitschnit fir rationelle Medicine 2:83-108. Brindley, G.S. 1970 Physiology of the Retina and Visual Pathway. Baltimore, Md.: Williams & Wilkins. Campbell, F.W., and R.W. Gubisch 1966 Optical quality of the human eye. Joumal of Physiology (London) 186:55 578. Chen, D.S., and J.P. Allebach 1987 Analysis of error in reconstruction of two-dimensional signals from irregularly spaced points. IEEE Transactions on Acoustics, Speech, and Signal Processing 35: 173-180. Curcio, C.A., K.R. Sloan, O. Packer, A.E. Hendrickson, and R. E. Kalina 1987 Distribution of cones in human and monkey retina: individual variability and radial asymmetry. Science 236:579-582. French, AS., A.W. Snyder, and D.G. Stavenga 1977 Image degradation by an irregular retinal mosaic. Biological Cybemetics 27:229-233. Goodman, J.W. 1968 Introduction to Fourier Optics. New York: McGraw-Hill. Helmholtz, H. 1860 Handbuch den Physiologichen Optik, vol. II. (English edition republished by Dover Publications, New York, 1962~. Hirsch, J., and R. Hylton 1984 Quality of the primate photoreceptor lattice and limits of spatial vision. Vision Research 24:347-356.

134 JOHN L YELLOlT Hirsch, J., and W.H. Miller 1987 Does cone positional disorder limit near-fovea! acuity? Joumal of the Optical Society of America A 2:1481-1942. Jennings, J.NM., and W.N. Chalman 1981 Off-axis image quality in the human eye. Vision Research 21:445-455. Miller, W.H., and G.D. Bernard 1983 Averaging over the foveal receptor aperture curtails aliasing. Vision Research 23:1365-1369. Osterberg, G. 1935 Topology of the layer of rods and cones in the human retina. Acta OpAthalmologica 6(Suppl.~:1-103. Shannon, C.E. 1949 Communication in the presence of noise. Proceedings of the IRE 37:1(}21. Still, D.L., and L^N. Thibos 1987 Detection of peripheral aliasing for gratings seen in the Newtonian view. Joumal of the Optical Society America A. 4:P79-P80 (abstract). Williams, D.R. 1985 Aliasing in human foveal vision. Vision Research 24: 195-205. Williams, D.R. and N.J. Coletta 1987 Cone spacing and the visual resolution limit. Joumal of the Optical Society of America A 4:1514-1523. Yellott, J.I. 1983a Spectral consequences of photoreceptor sampling in the rhesus retina. Science 221:382-385. 1983b Nonhomogeneous Poisson disks model the photoreceptor mosaic. Investigative Ophthalmology and Visual Science 24 (Suppl.~:147 (abstract). Yen, JOLT 1956 On the nonuniform sampling of bandwidth limited signals. IRE liens. Circuit Theory CI`-3:251-257.

Next: The Invisible Cone Mosaic »
Advances in Photoreception: Proceedings of a Symposium on Frontiers of Visual Science Get This Book
×
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

One of a continuing series on Frontiers of Visual Science, this short volume contains papers presented at a recent symposium. These papers describe techniques for assessing the structure and function of photoreception, both in isolation and in aggregation as the "receptor mosaic" of the retina. Also described are dynamic and spatial sampling properties of photoreceptors, with an emphasis on techniques of measurement and associated models of retinal function.

This volume should be of interest to basic vision scientists, clinical ophthalmologists, workers in human factors, and computer scientists curious about the visual apparatus of biological systems.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!