Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 315
315
being funded to perform computations on a university com-
puter. In these cases, net cost savings and performance
gains may very well be realized by switching to a dedi-
cated minicomputer-array processor system, and the users
should be encouraged to submit a collaborative proposal
to obtain such a system.
3. The Panel applauds the support of astrophysical
computation provided by Lawrence Livermore Laboratory,
NASA/Ames, NASA/Langley, Los Alamos National Laboratory,
the National Center for Atmospheric Research, and others.
The Panel recommends that these cooperative efforts be
continued and expanded, since they work to the advantage
of both parties. Astronomers have access to larger and
faster machines than would otherwise be available, while
the laboratories receive broader recognition and stimula-
tion from more diverse viewpoints than would otherwise be
the case.
Astronomical use of these large facilities may be
expanded by providing funds to support problem definition
and software development and testing with university
computers or dedicated minicomputers (as above) and by
encouraging the large centers to make more time available
for production runs. The software that has been developed
locally may then be run at the large centers either via
remote links or travel of the investigator to the center
The Panel considered a number of other options but
rejected them as being too costly. A National Computing
Center for Astronomy with a large vector machine would
cost $5 million per year. The provision of funds to buy
time at existing large computers (at cost) could only be
justified if the problem could not be run in a reasonable
amount of time on a minicomputer-array processor system.
Some problems are large and would cost $0.5 million to
$1.0 million per problem.
V. IMAGE PROCESSING AND ANALYSI S
.
Despite the title of this section, we do not intend to
ignore the processing and analysis required for other
kinds of data. It is simply that data generated by image
detectors are so voluminous that if we have the capability
to process and analyze these data we will certainly have
the capability to handle other forms of astronomical data.
It could conceivably be argued that we are collecting
too many data--that we should cut back until the data
collected match our capabilities to analyze and interpret
OCR for page 316
316
them. Such an argument is specious. We do not generate
data simply for the sake of generating data. Each astro-
nomical observation is performed in order to obtain data
that will lead to new knowledge concerning specific
scientific questions. Even survey observations (which
may appear to some as indiscriminate collection of data)
are designed to provide new knowledge that can be obtained
only through statistical studies of large bodies of data.
We unabashedly admit that we generate new knowledge for
the sake of new knowledge.
Until a few years ago, most astronomical observing
techniques produced data at very modest rates.
A night's
observation might typically produce tens of stellar-source
brightness or radio-flux measurements. However, with the
advent of solid-state array detectors and rapid scanning
microphotometers, data are now produced at a prodigious
rate, typically in the form of digital images. In an
observing run of a few nights, optical astronomers now
generate many tens of images, which in turn consist of
over 106 picture elements (each pixel containing un to
16 bits of intensity information).
A single photographic
plate taken at the prime focus of a large reflector
contains well over 109 bits.
Optical astronomers are hardly alone in their ability
to generate large numbers of digital images. The Very
Large Array produces radio maps with spatial resolutions
from 0.1 arcsec to a few arcseconds. Each radio picture
may contain up to 107 pixels with 16 bits per pixel.
For many programs, images will be made at up to 256
frequencies simultaneously in order to map the velocity
field in selected objects.
m e Space Telescope Wide-Field/Planetary Camera will
produce typically fifty 1600 x 1600 images per day. The
high spatial resolution afforded by this space observa-
tory should, by mid-decade, provide astronomers with an
unprecedented view of the deep reaches of the Universe.
me application of these new imaging techniques, both
on the ground and in space observatories, offers enormous
. . . . . . .
promise in our quest to understand the geometry and
structure of the Universe and the evolution of galaxies
and stars. While astronomers are all beginning to take
advantage of the potential for discovery afforded by
application of these new tools, the community has just
begun to realize that these technological opportunities
present a major and immediate challenge to develop
innovative techniques for the analysis of digital images.
.
OCR for page 317
317
Application of interactive image analysis at several
large universities and national centers has
already per-
mitted astronomers to attack certain classes of astro-
nomical problems. For example, large-scale surveys of
galaxy surface brightness distributions have provided new
insights into the effects of environment on the formation
and evolution of galaxies. Statistical studies of stellar
images near the limit of detectability have suggested
that, in our Local Group of galaxies, there appear to be
radical differences in the history of star formation and
the types of stars formed in different systems. The fac-
tors that appear to influence the rate at which new stars
are formed in external galaxies and the factors that
determine how many stars are formed as a function of mass
are starting to be sorted out. Radio maps, with arcsecond
and subarcsecond resolution, have revealed structure rang-
ing from the remnants of protostellar clouds to the
detailed intensity and polarization structures of galactic
jets.
In all cases, progress on these problems depends in
large measure on our ability to study not one or a few
examples of an object or class of objects but rather a
statistically significant sample. For each problem,
astronomers must invent new techniques for extracting the
essential data from digital image arrays. The horizon
for creative analysis of these arrays is bounded solely
by the imagination of individual scientists and their
access to adequate image analysis facilities.
While we can easily predict an explosion in the number
of digital images generated on the ground and in space-
based experiments, there appear at present to be few
facilities capable of permitting individual astronomers
to interact creatively with these data. If we are to
take advantage of the promise of new imaging technology,
it is crucial that the astronomical community, in con-
junction with federal funding agencies, plan to develop
and disseminate adequate image data-analysis capability
throughout the United States.
The need for extensive image-analysis capability for
the astronomical community arises from (1) the increase
in the number of solid-state array detectors that will be
deployed at the major ground-based facilities over the
next several years; (2) the high rate of image data gener-
ation from the present-day deployment of such detectors,
which already threatens to overwhelm existing data-
analysis systems--data that will be produced by the Space
Telescope, particularly from the Wide-Field/Planetary
OCR for page 318
318
Camera and the Faint-Object Camera, will further burden
existing analysis systems; (3) our ability to recover,
with digital microphotometers, the information contained
on photographic plates (which will remain unchallenged as
recorders of moderate-photometric-accuracy large-area-
coverage data); and (4) the generation of high spatial
and spectral resolution maps from the Very Large Array
(VLA). We estimate that in the latter half of the 1980's
these sources will be generating image data at a rate of
approximately 5 X 1012 to 10 X 1012 bits per year
(including calibration frames and ancillary data).
Num-
oers 1n tnls range can ne arrived at in several ways:
(1) by multiplying the estimated number of telescopes
equipped with image detectors by a typical night's output
by the number of nights per year or (2) by scaling the
output of Space Telescope (which has been estimated at
1ol bits per year from a detailed mission simulation)
to the number of ground-based telescopes that are expected
to be equipped with image detectors (taking account of
the fact that Wide Field/Planetary Camera images will be
1600 X 1600 pixels but ground-based systems built around
the same chip will probably produce 800 X 800 pixel
images). Also, these numbers are in agreement with
estimates made for the United Kingdom Starlink system
(see below) if the difference in ~vail~hil iEv of F-1~-
scopes is taken into account.
Astronomers must be able to extract from digital images
the data relevant to the solutions of the problems they
pose. Extracting these data will involve two types of
operations--first, processing of the data, which involves
removing from the data the signatures characteristic of
an instrument. As an example, images generated from
solid-state arrays must be corrected pixel by pixel for
variation in sensitivity across the array. In the case
of intensified arrays, geometrical corrections to remove
the effects of image distortion introduced by the inten-
sifier may also have to be applied. Radio maps must be
explicitly corrected for sidelobe and other distortion
effects, to provide appropriate intensity versus position
maps at each observed frequency.
m e second function that mat he n~rfc~rm-~1 in :.n~lvĒ:i~
of the data. After an array containing accurate inten-
sity-position data is produced, the maths to problem
solution begin to diverge.
Each type of problem may
require tne Development or special techniques for ex-
tracting the relevant data from the image array and
providing such problem-specific measures as object
OCR for page 319
319
brightness, statistical "counts" of numbers of objects
brighter than a certain value, and distribution of bright-
ness with position. Operations carried out under the
rubric ~analysis" often demand the development of imagina-
tive new techniques. In some cases, analysis will reveal
that residual instrumental effects are still present in
the data, thus forcing a return to the last stares of
processing. This is particularly true in the case of VLA
Images, where often only analysis reveals the potential
and need for improvement in dynamic range.
In developing a strategy for deployment of computing
equipment required to meet the challenge of image analv-
sis, we conclude the following:
(a) Facilities capable
of processing and analyzing data should be placed at or
near the place of data origin. Each major ground-based
observatory should be provided computing capability ade-
quate to render data in the form of intensitY-position
information. Such commuters would have sufficient Dower
to permit the staff and a relatively small number of out-
side users to analyze their data in parallel with continu-
ina Processing activities. (b) Analysis functions, on
should be carried out as close to one's
~ ,
the other hand,
home institution as possible.
The best scientific results
derive from the active participation in data analysis by
research astronomers and their graduate students operating
on their own schedules with time to think about the inter-
mediate results as they are obtained. For some problems,
many man-months or -years are required in order to develop
appropriate reduction algorithms.
These two distinct environments, with their differing
constraints of user access and throughput, require differ
ent scales of investment. In the university community,
local facilities are required for data analysis and theo-
retical computation. Since these facilities are local,
comparatively small numbers of astronomers are served,
and they can make use of a ready access to the facility
over an extended period of time. By contrast, at the
National Astronomy Centers, including the Space Telescope
Science Institute (STScI), and those centers of data
acquisition where user service is contemplated, a large
number of users must be served. Each user must be able
to complete his or her work in a comparatively short time,
since most astronomers cannot easily arrange for long
periods away from their home institutions.
-
Also, at the
National Centers it is imperative that all data processing
(calibration and removal of instrumental effects) be done
so that all observers can, at minimum, be provided with
OCR for page 320
320
data as free as practical from instrumental aberrations.
This data-reduction problem should not propagate beyond
the data source. Thus, while it is usual for an analysis
problem to take an astronomer many months to complete at
his or her home institution, a few days is the typical
limit that one will likely be able to stay at a National
Center beyond the scheduled observing time for data col-
lection. National Centers should therefore be able to
provide processed data within a day or so of the end of
the observations. Routines should be available at
National Centers to allow visitors to perform standard
manipulations of data with at least the same rate as it
is gathered and provide useful output formats (maps,
pictures, tables, plots).
Home institution computer
facilities are more useful for extensive interactive
image analysis, which may include development of new
techniques requiring experimentation and iteration.
This basic difference in the required user response
time of the facility serves to set the scale and nature
of the equipment required. National Centers require
integrated computer systems that allow efficient inter-
leaving of batch and interactive processing in large
volumes. Such a system may take the form of a major
batch facility (the host) interconnected to multiple,
smaller interactive satellite computer systems. In some
cases, rather elaborate pipelined computer structures may
be needed to handle extensive, special processing prob-
lems. The multiplicity of interactive computer facilities
is essential to assure the availability and response
required by short-term visitors. These satellite pro-
cessors could serve as models (with modest upgrading of
peripherals and memory) for individual university systems.
Larger university systems could more closely follow the
structure of the host processor, using this host to serve
both batch and multiple interactive stations. For the
National Centers, at present, a combined host-interactive
satellite computer scheme will cost (capital) $1 million
for a fully integrated system capable of serving simul-
taneously 4-6 interactive picture-processing functions
and the concurrent batch stream. Single-processor
university systems will range in cost from 8100,000 to
400,000, depending on the scope of the problems to be
attacked. The costs are likely to remain relatively
constant through the 1980's; as costs drop, demand may be
expected to increase at least as much.
We recommend that the funding agencies (NASA and NSF)
provide funds for the implementation of decentralized
OCR for page 321
321
image processing and analysis systems equivalent to 20
"canonical" systems (see Appendix 5.A). A continuing
level of funding is required so that when a "steady state"
is reached, the funding provides for the replacement of
systems every 6 years--the typical life of a computer
system before it becomes obsolete. Appendix 5.A elabo-
rates on the need for replacement and also discusses the
maintenance required for these systems.
The British Starlink System (see below) provides a
similar ratio (actually somewhat higher) of computer
capacity to data volume as will be supplied by the pro-
posed systems with the data volumes estimated above.
Without array processors, the combined capacity of
these 20 systems would be sufficient to perform approxi-
mately 100 computer operations per pixel if the data
rates are as described above. (Here we use "operation"
to mean an elementary computer operation such as an
addition or multiplication of two numbers; in other
usage, the term "operation" may denote a function such as
addition of two images.)
. . . · · . . . .
This may seem like a lot of
capacity until one realizes that Lo perform a single func-
tion, many operations are required.
Lions, such as reading an image from a disk file and for-
matting it for display on a video monitor, can consume
tens of operations per pixel. Complex functions, such as
geometric corrections, correction for nonlinear transfer
functions, or applications of spatial filters can consume
hundreds of operations per pixel. The estimate that 20
systems are required is unlikely to be too large and may,
in fact, be considerably too small. However, the capa-
bility of adding array processors to these systems pro-
vides a margin of protection against this possibility.
Up to now the discussion in this section has concen-
trated on the hardware facilities required to provide
adequate image processing and analysis capabilities. Of
course, a substantial effort must be devoted to software
development. As described in Appendix 5.A, a typical
system will require, at a minimum, the support of one
full-time software person to provide for maintenance of
the system software and the development and maintenance
of basic applications software. Also as described in
Appendix 5.A, there exists considerable flexibility in
the manner in which this support is provided--part-time
student or faculty support, dedicated support, shared
support, or other.
However, with the decentralization of facilities as
proposed above, there is the potential that much of the
Even simple tunc-
OCR for page 322
322
software development will be duplicated at each facility.
This is much more likely for image processing and analy-
sis applications than for theoretical applications. Soft-
ware required for the former is based on a core of stan-
dard functions such as contrast stretching, filtering,
and object identification, while software used for theo-
retical computations tends to be much more application-
dependent. The theorist tends to put his or her knowl-
edge of the physics of the problem into the software
itself, while the image analyst uses his or her knowledge
of the problem to determine the sequence of basic func-
tions to be applied to the data. (Of course, image
analysis often requires the development of unique
software to supplement the core of standard functions.)
A possible solution to the potential problem of dup-
lication of software development efforts is to insist
that all facilities be built around machines with the
same architecture (i.e., machines from a single family
from a single vendor) and to designate one of the facil-
ities to be responsible for software development for all
facilities. Such a solution would be similar to that
adopted by the British for their Starlink System.
The Starlink System, currently being implemented, is a
distributed system dedicated to image processing and
analysis. There are to be six nodes in the system, each
built around a Digital Equipment Corporation VAX 11/780
computer. The nodes will be connected by dedicated 9600-
baud lines, and one node will be responsible for software
development and maintenance. (A baud is a measure of
information-carrying capacity--in typical applications a
baud corresponds to 0.73 bit per second, thus, a 9600 baud
line can carry about 875 eight-bit characters per second.)
The communications links will be used primarily for dis-
tribution of software updates and documentation.
While such a system has advantages and is well matched
to the needs of the British astronomical community, we
believe that the Starlink model is not entirely appropri-
ate for image processing and analysis in the United
States. The U.S. astronomical community is spread over a
much wider geographical area than is the British com-
munity, and, even with low speed lines, the communication
links would be a significant expense. Furthermore, funds
are probably not available to purchase the complete facil-
ities at the same time as the British are doing. This
means that the facilities would have to be installed grad-
ually and updated gradually. If, in addition, the key to
the successful operation of the software is the fact that
OCR for page 323
323
the same computer and peripheral architectures exist at
each node, then the facilities would be locked into the
architecture they started with until that family of
computers was discontinued by the manufacturer. At this
point a tremendous spike in funding would be required to
convert both hardware and software to a new architecture.
Finally, it appears that a distributed system such as
Starlink offers less flexibility than a decentralized
system for tailoring the capabilities of individual nodes
to the problems to be addressed at the node.
Our suggested approach to the software problem is not
a sure-fire solution, but it does represent, we believe,
the best solution available at present. The National
Centers (particularly Kitt Peak National Observatory, the
National Radio Astronomy Observatory, and STScI) should
take an active role in software development by
1. Developing software that is as machine independent
as possible;
2. Adequately documenting the software;
3. Distributing standard processing and analysis
software and/or algorithms; and
4. Assisting users in the implementation and use of
software developed at the National Centers.
Of these four functions, only the second two should impose
any additional costs on the Centers. The first two are
required in any well-managed software effort. We estimate
that the second two functions can be provided at the addi-
tional expense of one man-year (plus overhead) per center
per year.
In addition, it is necessary that there be some com-
munity-wide coordination in such areas as software devel-
opment and data format standards. For this purpose, a
permanent committee should be established, perhaps under
the auspices of the AAS and perhaps with federal support.
The committee would be responsible for continuing evalua-
tion of astronomical computing needs, developing standards
for data formats and software compatibility, and collect-
ing and disseminating software and hardware news of rele-
vance to the community. The committee should also play a
coordinating role in the archiving and data-base activ-
ities described in the next two sections. Finally, the
committee should provide a focal point for liaison with
Starlink and other non-U.S. image processing and analysis
facilities.
Representative terms from entire chapter:
software development
