Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 27
Software and Ur~ortlodox Architectures:
Where Are We Headed?
MICHAEL L. DERTOUZOS
SOFTWARE
Can you construct a 10-mile-long wooden bridge? The answer is not
very relevant. What is relevant is that if this question is posed to 100
civil engineers, they will answer either yes or no fairly consistently.
Now for a question at the cutting edge of computer science research:
Can you mimic an airline office by a computer program so that when
someone types in a query that would normally be placed by telephone
"Do they serve meals on the flight from here to Atlanta?" or "What
time does the flight leave?"—the automated system responds on that
person's terminal with the kind of answer that would be given over the
telephone? Can you construct a system that will answer 80 percent of
these questions correctly? I have asked this question of many people
working on the leading edge of computer science research, and their
typical answer is "Give us $2 million and about two years." While I
have not carried the experiment further? I know that if I were to pursue
it, in two years the answer would be either "Here it is, and it's answering
correctly 75 percent of the queries" or "Give us another $2 million and
another two years." The point here is that with the exception of com-
puter science theory and compiler design, there is very little science in
this field, which in my opinion is still largely experimental in nature.
The cutting edge of computer science is very much like alchemy in the
old days mix two liquids and if there is an explosion take notes. Leav-
ing some room for poetic license, that is the approximate situation with
27
OCR for page 28
28
COMPUTERS OF THE FUTURE
most of our advanced programs and with many of our avant-garde hard-
ware architectures, as will be seen.
As a result of this absence of "laws" in the world of software, the
programmer is the lawmaker. That, in a nutshell, is the power and
attraction of programming: to set good laws and to build complex struc-
tures based on these laws. For this reason programmers are extremely
reluctant to "buy" each other's laws; each one wants to be the lawmaker.
That is also why young people become millionaires by starting pro-
gramming companies their fresh minds are not burdened by the excess
baggage of experience, and they can accordingly thrive in creating and
adhering to new laws.
What can educators really teach to a young aspiring programmer?
Unfortunately, very little. Educators can issue generalities and plati-
tudes, such as, "Design your programs in nice modules, isolate them,
make each module neat and tidy and describe its inputs and outputs."
This is an exaggeration, of course, but there is very little about pro-
gramming that can be taught explicitly. Educators do ask young pro-
grammers to read other people's programs and to apprentice by watching
experienced programmers at work. The effectiveness of these ap-
proaches again suggests that the similarity to alchemy or even to sculp-
ture may not be too farfetched.
Traditional software generation cannot be substantially improved with
the techniques that are in hand today or that have been developed over
the last 20 years. While the hardware cost/performance improves at the
rate of roughly one decade per decade, the corresponding software-
improvement is by comparison insignificant: During the last 10 years
software productivity measured as lines of program produced per pro-
grammer per year has increased by about 50 percent.
Are there any prospects for improving software productivity? I will
discuss a few. Structured programming is a technique that is most ef-
fective if one deals with very small programs, often three or four lines
long. I am reminded of the pioneer of structured programming, the
famous Professor Dijkstra, who spent an hour explaining to a large
audience at MIT how a three-line program could be verified. One of
our students asked: "But how would you apply this to a chess-playing
program?" And Dijkstra, in his inimitable style, said: "One should never
write such complicated programs." More seriously, I believe that struc-
tured programming has been helpful, but we seem to have exhausted
its benefits.
Automatic programming was a technique attempted, without much
success, during the last 15 years. The intent was to describe something
like an inventory control process at a very high level and then to use a
OCR for page 29
SOFTWARE AND UNORTHODOX ARCHITECTURES
29
gigantic compiler to generate the high-level programs (e.g., in COBOL)
that one would normally write for inventory control. This approach did
not work well for reasons of complexity and variance among different
users' needs.
Functional programming is a recent approach that tries to build pro-
grams in the form of mathematical functions that have inputs and outputs
and, significantly, no side effects. Different functional programs can be
combined in the same way that complex mathematical functions can be
composed out of simpler ones. It is too early to predict how well this
approach will work, and it is difficult at this time to bend into the
functional mold traditional programs used, for example, in data base
management, where side effects are the governing mechanisms.
There is currently great hope that expert systems will be useful in
programming; this would mean that an "expert assistant"—a pro-
gram- would help us program more effectively. Unfortunately it is too
early for a meaningful assessment of this prospect.
My own overall assessment about the future success of programming
productivity improvements is rather bleak. But let me ask a provocative
question: Why do we need programming?
To go back 20 to 25 years—in the days of batch computing people
programmed because there was a very expensive resource called the
computer that cost several million dollars to acquire. They waited in
line for hours if not days to get their cards processed into results, only
to modify their programs against that "last bug" so that they could stand
again in the waiting line.
Later, in the late 1960s, time-sharing came along. At the beginning
at least, the reasons for developing and using time-shared systems were
the same. The same very expensive resource could now be shared by
many people in round-robin fashion so that each user did not, it was
hoped, notice the delay. Soon thereafter, time-sharing pioneers changed
their position and said that cost-sharing was no longer a valid reason
for these systems. Instead, attention was focused on the utility aspects
and the services offered by those systems. And that, indeed, turned out
to be one of the great truths about the ultimate usefulness of time-shared
systems.
Now, with computer costs dropping dramatically, the personal ma-
chine that I have next to my desk costs $5,000 and does more than the
$2.5-million time-shared machine that we had in our lab 20 years ago.
To the extent that this is now the case, why is programming needed
anymore? Certainly not for cost-sharing. It is needed for application
flexibility, i.e., to carry out different applications on the same piece of
equipment.
OCR for page 30
30
COMPUTERS OF THE FUTURE
Let me now ask a second provocative question. We have said that
while the hardware is improving at the rate of 30 percent per year,
software productivity is essentially standing still—hence we are con-
fronted with a "great software problem." The question is simply this:
Is there really a software problem? In my opinion the answer is no for
this reason: Because software involves paper and pencil and not the
traditionally difficult tasks of putting hardware together into complex
physical systems, there is a tendency to assume that software ought to
be easier. This is even reflected in the legal differentiation made between
software and hardware: one is patentable while the other is "copyright-
able."
Compare, however, the design of the airline office program mentioned
earlier with the design of a complex physical system like a jumbo jet
and not the routine twentieth design of a jumbo jet, but the original
design of such an airplane. This comparison deals with two objects of
roughly the same complexity in terms of the elements that they contain.
And while people might laugh about the airline office program taking
several years to design, they would readily agree that such a long design
time would be reasonable and proper in the case of the jet plane.
In my experience there is neither precedent nor prospect for achieving
dramatic economies in the design of complex systems, especially in the
case of first designs, which are typical of novel software systems. In
these cases creativity plays an important role since one is trying to break
new ground. There is no reason to believe that in the field of program-
ming this endeavor ought to be easier than in any other discipline that
depends heavily on creative design.
Let me close this section on software prospects with some good news.
There is ample precedent for economizing through the mass production
process, e.g., by spreading the very expensive cost of designing a com-
plex product to the people who buy it.
Accordingly, I would suggest that one of the biggest forces before
future software developments is the transition of the software devel-
opment process from the artisan to the mass production era. In partic-
ular, two significant categories of products can be expected to emerge
in the next decade.
The first category can be called hidden computers, because it deals
with "appliances" performing a useful function. These products will not
be thought of as computers any more than cars are thought of as col-
lections of carburetors and other components, or refrigerators'as electric
motors. Each hidden computer is especially programmed for one ap-
plication, not changing its program, and providing immediate utility to
its user. Examples include the drugstore machine, the personal memo
OCR for page 31
SOF7~VLARE AND UNORTHODOX ARCHITECTURES
31
pad, the automobile safety package, the automobile convenience pack-
age, the fuel control package, the automobile maintenance package,
and so on.
The second emerging category of software products is mass-manu-
factured applications in the form of diskettes, which in time will become
more powerful and more useful. In both categories the large costs in-
curred in the design and production of these items become very small
when spread to hundreds of thousands or even millions of users.
Finally, I do expect that more intelligent programs will make life easier
for all of us. When I am told that a certain program or machine has a
friendly user interface I get a bit worried, because I find that I need 40
or 50 hours to become truly acquainted with a new machine, and I am
a computer professional. Such a relationship may be friendly compared
with a life-threatening situation, but it is not a relationship we should
have with something next to our desk. I believe that the only way to
achieve true friendliness with computers is by increasing the intelligence
of the programs that they contain. Only then will machines begin to
understand what we are trying to do in the same sense that friends
understand what we mean when we communicate with them.
FORTHCOMING SYSTEM ARCHITECTURES
The dominant theme in forthcoming machine architectures will be the
myria-processor system. In Greek, myria means 10,000. That number
may be close to what will be used in practice, at least in the next 15 to
20 years. Look, then, for future computer systems that use hundreds or
thousands of processors.
Myria-processors will be used in two areas. The first involves the so-
called geographically distributed systems. This architecture is evolving
because people who generate and collect data are geographically dis-
tributed. Moreover, organizations have geographically distributed of-
fices and personnel who need to communicate with one another. And,
of course, there is a great techno-economic opportunity along with these
needs in that people can now afford to buy computers and the com-
munications products and services that interconnect them.
The second area of myria-processor revolution involves tightly coupled
multiprocessor systems. Here I envision again hundreds or thousands
of processors in one very big box, devoted to one task, for example,
recognizing the face of one human being among a few thousand or trying
to comprehend human speech. As in the case of distributed systems, a
sizable techno-economic opportunity is driving us in this direction, with
OCR for page 32
32
COMPUTERS OF THE FUTURE
the very low cost of individual processors making such large systems
affordable.
Geographically Distributed Systems
Consider 1,000 autonomous machines that are geographically distrib-
uted autonomous because each machine can support the desires of its
owner in editing text, handling messages, and running application pro-
grams, all independently of what other people are doing on their com-
puters. In addition to this autonomy, however, each computer must
cooperate with the other computers in the system, whether they are
1,000 miles away or upstairs, in order to make possible common appli-
cations, e.g., planning, putting together a manual, or trying to figure
out why there was a power failure in New York City.
The electronic interconnections among computers, which are a com-
bination of local-area networks and long-distance terrestrial or satellite
networks, are fairly straightforward at the hardware level and present
no substantive long-range problems. Large companies that have begun
to establish networks of distributed systems are approaching these sys-
tems from an era in which dumb terminals became progressively more
intelligent in the presence of a central powerful program that controls
everything in the system. It is now becoming clear that these intercon-
nected machines need to be autonomous and that there can be no central
"executive." The reason for this decentralization lies in the desire to
establish arbitrarily large systems: A centrally directed system has an
upper size limit of perhaps 50 to 100 nodes for the same reason that an
individual can only converse simultaneously with a small, limited number
of people. Achieving truly decentralized systems will require the de-
velopment of some form of common currency or, if you wish, simple
English—for computers so that these machines can "understand" each
other for the purpose of common applications. This requirement is at
least philosophically analogous to the communication means in humans
tribes: Each individual is autonomous, yet the aggregate can embark
on common activities because they can understand one another. Ulti-
mately these computer tribes will form an "information marketplace,"
i.e., a free market dedicated to the purchase and sale of information
and informational labor.
Now let me pose a challenge to computer professionals: Can you
construct a 1,000-computer distributed system that at minimum has the
behavior and performance of a traditional, centralized, time-shared sys-
tem that services perhaps 50 users in round-robin fashion? The choice
of the number 1,000 is of course arbitrary. If this goal can be achieved
OCR for page 33
SOFTWARE AND UNORTHODOX ARCHITECTURES
33
for 1,000 machines, I would immediately ask if it can be done for 10,000
or even a million computers. In short, is there a scalable system archi-
tecture that permits the kinds of services and the sharing of data that
transpire in time-shared communities? I have posed this challenge in
terms of time-shared systems because there is experience in that area,
and it is known that if this behavior and performance can be extended
to arbitrarily many users the result will have proven utility.
This question has not yet been answered. It seems, however, that it
must be answered one way or another before distributed computer sys-
tems become truly useful.
Let us look, for comparison purposes, at an existing network called
the ARPANET, which spans several hundred computers from Hawaii
to West Germany. In principle and in practice it is possible to log into
any computer on this network and to use that computer from a remote
location. Yet, hardly anyone does this. Instead, people use this network
as a very sophisticated message-exchange medium, which has proven
its utility for that purpose. The reason that remote use of computers
does not take place is that dialing into another computer generally means
being confronted with an entirely different system that speaks its own
language and has its own conventions, which are either unknown to the
person dialing in or are costly to learn. Hence, except for the great
young hackers who enjoy exploring remote computers, most people have
very little use for such an unknown and remote resource.
Regardless of the success of devising effective decentralized and dis-
tributed computer systems, individual computer "nodes" are likely to
continue their rapid evolution. The dominant trend that I see is toward
more powerful personal computation. In this regard people often ask if
they really need in their personal computers more memory than 128,000
characters, half a million characters, or at most 1 million characters (1
megabyte). The answer is an easy yes. If we focus on greater program
intelligence, which is the key to making these machines truly useful to
us, then we shall discover that a forefront-research intelligent program
occupies today 3 or 4 megabytes and, incidentally, takes about 30 to 70
man-years to develop. So having a personal computer that is truly friendly
and that tries to comprehend some free-format English as well as to
provide certain intelligent services involves the equivalent of 4 or 5 of
today's forefront intelligent programs, i.e., some 15 to 20 megabytes of
primary memory.
To conclude, I believe that success in distributed systems will depend
on our ability to address effectively the following three challenges. First,
such systems must achieve an acceptable level of overall reliability in
spite of local failures. Currently computer systems are like Swiss watches.
OCR for page 34
34
COMPUTERS OF THE FUTURE
Reach into them with a spoon and the entire edifice fails, catastrophi-
cally. It is essential to make distributed systems work more like human
tribes- if one or more nodes collapse, the tribe goes on functioning
properly.
The second and perhaps biggest challenge before us is to discover a
minimal semantic base, a minimal form of "computer English" that is
understood by all machines and that simultaneously maximizes local
autonomy and application cohesiveness.
The third challenge involves the assurance that the information carried
on distributed systems is protected and that the signatories of messages
are indeed the purported agents and not impostors.
Multiprocessor Systems
The second area of myria-processor systems might be called com-
putation by the yard, i.e., if you like 1 yard of computing you buy a 100-
processor system, and if you like 2 yards you buy a 200-processor ma-
chine. An example of a multiprocessor system is a 1,000-processor ma-
chine that converts human speech to typed text. Other applications
include the solution of large partial differential equations, signal pro-
cessing, and weather forecasting. But the most interesting applications
are in sensory computing, meaning automated eyes and ears, and in
more intelligent computing.
The force driving us toward multiprocessor systems is their cost. If
1,000 small processors can be harnessed into one computing machine,
then that machine will have 10 times the processing power of today's
fastest computer, at 1 percent of the latter's cost.
This harnessing of numerous processors in one machine is being at-
tempted in several ways. One way is the data-flow approach, which
consists of (1) a communication network that has perhaps 1,000 inputs
and 1,000 outputs, and (2) 1,000 processing elements. In this architecture
the results issued by each computer enter this network carrying a little
flag that establishes their destination. Thus, if the flag says "39," these
results will exit the network into the thirty-ninth computer where they
will be processed into new results with new flags.
The programming of such a machine is entirely different from the
programming of today's Von Neumann-type machines, because many
operations are carried out simultaneously rather than sequentially.
The challenges in trying to realize these multiprocessor systems are
as follows. First, such systems must be linearly or at least quasi-linearly
scalable this is the computation-by-the-yard notion. Second, as in the
case of distributive systems, these multiprocessor machines must work
OCR for page 35
SOFTWARE AND UNORTHODOX ARCHITECTURES
35
correctly in spite of local failures, because in a system with 1,000 pro-
cessors there will always be several processors that do not work correctly.
Finally, there must be an associated programming environment so that
these machines can be effectively programmed. Moreover, such a pro-
gramming environment should be "tuned" to the hardware architecture
to ensure good performance under the target applications.
The following question is often raised in discussions of multiprocessor
architectures: It is conceivable that we can put together 1,000 or 10,000
processors, but how are such systems going to extract the parallelism
inherent in today's large FORTRAN programs? My answer is that this
is not possible, any more than it is possible to automatically extend
procedures for managing one individual to procedures for managing a
factory or a company of 1,000 people. In both cases the techniques are
entirely different. Hence, we must forget about automatically unfolding
the parallelism in today's programs. Instead, it is necessary to learn how
to harness these "warehouses" full of computers by explicitly program-
ming and organizing them in the same sense that a complex shoe-man-
ufacturing company is planned and tuned today.
Finally, I think that following our experience with single-processor
machines there will be special-purpose multiprocessor systems dedicated
to specific applications that will always carry out these applications with-
out the need or ability to be reprogrammed for other applications.
SUMMARY AND CONCLUSION
In the software domain, programs will move from the artisan to the
mass-manufacturing era, and they will manifest themselves progressively
more as hidden computers or mass-manufactured applications on dis-
kettes. Users will adapt to these programs rather than the other way
around as has been the case in the past. Programs will become more
intelligent and will move into personal computers, distributed systems,
and multiprocessor systems where they will perform a variety of service-
oriented functions, including the sensory area. Programming productiv-
ity will not increase dramatically. Instead, the continuing large program
design costs will be absorbed in the individual cost of mass-produced
programs.
Novel system architectures will abound under the myria-processor
theme in two principal categories: (1) geographically distributed, loosely
coupled systems that behave, like tribes, with autonomy and application
cohesiveness and that will form an information marketplace; and (2)
multiprocessor tightly coupled systems, each dedicated to one applica-
tion, which are expected to open new doors in sensory and intelligent
computing.
OCR for page 36
Representative terms from entire chapter:
unorthodox architectures
