|
Items in cart [0] |
Questions? Call 888-624-8373 |
| ||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 9
3
BASIC TOOLS FOR APPLIED DECISION THEORY
Decision analysis, or applied decision theory, was developed about 35 years ago to bring
together two technical fields that had developed separately. One field was the theoretical
development of how to help a person make simple decisions in the face of uncertainty. This field
was begun in the 1 8th century with the work of Bernoulli, then Bayes, and finally Laplace. It was
improved and refined to a high state of development in the years following World War II.
At that time the control and systems engineering development during World War II was
available to make practical the application of fundamental ideas about decision making under
uncertainty to the actual problems faced by decision makers. The new field of decision analysis
provided both a formal, systematic way to analyze decisions and an important communication
medium for achieving mutual understanding between decision makers and those who advise
them.
Many important creative activities from engineering design through medical treatment to
business strategy—have unique features in their technological basis and their possible
consequences, among many other elements; however, they all share the characteristic of being
fundamental decisions. They differ in their alternatives, forms of uncertainty, and preferences for
consequences, but they share the features of all decisions: the need to distinguish the quality of
the decision from the desirability of the consequence, the need to incorporate uncertainty and to
value experiments, tests, surveys, and other forms of information gathering that might reduce
uncertainty at a cost, and the need to establish preferences for outcomes, including outcomes
achieved with different probabilities. This is true whether we are designing a planetary probe or
managing a portfolio of chemical entities for a pharmaceutical company. Recognizing the
similarities of all decision processes allows us to use important general insights in applying them;
this is particularly true for engineering design.
The purpose of decision analysis is to provide decision makers with clarity of action in an
uncertain decision situation. The metaphor for decision analysis can be conceptualized as a high-
quality conversation about a decision. Sometimes the conversation can be very brief and carried
on by oneself. More difficult and puzzling decision problems may require the assistance of
several analysts and extensive computer modeling. This spectrum is the domain of decision
analysis.
5
Perhaps the single most important distinction of decision analysis is that between making a
good decision and achieving a good outcome. The quality of the decision can be evaluated only in
light of the situation when the decision was made and not with any reference to its results.
A good decision is one that is best for me given the alternatives I have, the information I
possess, and the preferences I assess. For example, if someone offers to sell me for $10 one of
100 lottery tickets for a prize of $10,000, I would readily buy the ticket. Of course, if I am sure of
the validity of the offer, as I am assuming, I would like to buy all 100 tickets, but he offers me
only one. I know the good outcome is winning the $10,000 and the bad outcome would be losing
the $10. However, I am making a good decision to buy the ticket, even though there is a 99
9
.~
OCR for page 10
10
APPROACHES TO IMPROVE ENGINEERING DESIGN
percent chance of the bad outcome. I always want to choose the alternative that gives me the best
combination of probabilities and outcomes for my given risk preferences. The fact that I am not
likely to enjoy a good outcome, as in this case, is no indication I have made an improper decision.
There is a common but fallacious belief that experiencing a bad outcome implies a bad
decision was made. Such opinions are commonly voiced in sports. The football team tried to
make a two-point conversion and failed: The coach made a bad decision. That the operation was a
success but the patient died may simply be an example of a bad outcome following a good
decision. One necessary test of a good decision is whether you would make the same decision
again in the same situation if you had not yet learned its consequences.
THE DECISION BASIS
What is a good decision? A good decision is one that is systematically correct given a
decision situation properly framed by a committed decision maker. The specific description of
this situation is the decision basis. The three elements of the decision basis can be thought of as
the legs of a three-legged stool, as shown in Figure 3-1. The quality of a decision rests on having
framed the decision correctly, that is, answering the right question, understanding the issues
(knowledge), what can be done (options), and what you want (desired outcomes). Tools for
applying logic help the decision-making group or individual to reach a conclusion and direct
action.
One element is what the decision maker can do in the face of the alternatives to be
considered, which may call for an immediate decision or allow for postponing it to the future
after some of the uncertainties are resolved. These sequential decisions allow us to represent
options and to calculate their value.
The second element of the decision basis is information (i.e., what links the alternatives to
what will ultimately happen), which can be in the form of models describing the field of concern
in the decision. Some decisions, like the launching of a satellite, have the advantage of extensive
physical models to guide the decision. Other decisions (say, those related to the spread of
wildfires or the progress of a disease) will involve considerably more uncertainty. Still more
uncertain will be the behavior
of a jury or of consumers
responding to advertising.
Regardless of the extent of
modeling available, the
decision maker will ultimately
face some uncertainty in any
significant decision problem.
The decision maker represents
these uncertainties in the form
of probabilities or probability
distributions. These
distributions will be informed
by any available experimental
data, but in many cases,
particularly in applications of
new devices or systems, the
experienced judgment of
5
(fitted Dedsion beaker
fiat
You
I.
Can Do ~
Frame
Figure 3-1 The quality of a decision.
Vent
You
V\bnt
~ ~ Was You Know
OCR for page 11
IMPLICATIONS FOR EDUCATION AND RESEARCH
experts may be the only
available resource.
The third element of
the decision basis is
preference, or what the
decision maker wants.
Preference has three
identifiable dimensions in
most cases. The first is
valuation, or the trade-off
among different attributes
of the consequences of the
decisions. This is
summarized by a value
function specifying how
much more valuable one set
of attributes is than another.
The second dimension of
preference is time: how
much values in the future
will be worth relative to
values today. This
dimension is essential in purely financial decisions, but it also has application in, for example,
medical decisions, where the patient's quality of life must be balanced against the duration of life,
and in any other decisions where the attributes of the future must be balanced against the
attributes of the near term. The third dimension of preference is risk preference. Risk preference
is the dimension of preference stating which probability distributions over attributes of each
outcome are preferable to others. This can be as simple as whether the decision maker prefers to
receive a sure million dollars or to toss a coin for a payoff that will be either $3 million or
nothing.
1
$~,~
>
E ~
Cal Time Dynamic
of; Factor/,
~ D Statics ~ /
~3 / ~ /
wr~ ~
Ale Degree of Uncertainty
Deterministic ~ Probabilistic
Figure 3-2 The problem space for characterization
and decision making.
1
- 1
When all three elements of the decision basis are formally specified, the best decision can be
determined by employing rules (axioms) to extend the exercise of choice to the case where the
uncertainties facing a decision maker are explicitly recognized. In fact, all alternatives will be
ranked and evaluated by this decision process.
The importance of models in engineering design leads us to discuss them in more detail by
means of the problem space in Figure 3-2. This diagram illustrates the three dimensions of
difficulty a problem may have: uncertainty, complexity, and dynamic or time effects. The corners
with fewer degrees of difficulty tend to be those explored early in human history and are studied
early in the engineering curriculum. As we move from problems with few of these factors to more
of them, our ability to model decreases, until at the corner numbered 8 we have very few
mathematical representations, if any, where all three dimensions can be treated in a general way.
This means there will be a need for judgment in deciding which factors must be carefully
considered and which can be treated approximately. As stated in the Executive Summary, design
is not an endeavor that can be totally automated.
Returning to the stool, the seat is the logic acting on the decision basis to produce a course of
action. The person seated on the stool is the decision maker who has stated the decision to be
made. The ground on which the stool is placed is the frame developed for the decision situation.
5
i
f
OCR for page 12
12
A PPR OF CHES TO IMPR O VE ENGINEERING DESI ON
FRAMING
Framing concerns how we choose the decision problem to which we will apply this structure.
For example, suppose a manufacturer of buggy whips in the early 1 900s noted profits were falling
and decided to call in a consultant to see how manufacturing could be done more efficiently. The
decision would be framed as one of improving the buggy whip manufacturing process. Observing
the growing automobile revolution, however, the framing would more appropriately address how
to change the product line to serve the new
demands of automobile owners while phasing
out buggy whip production. Framing is the
most critical part of the process of decision
making, because making the right decision in
the wrong decision frame can be a serious
error.
Proper framing is the key to working on
the right problem. Improving the efficiency of Specified
buggy whip production would be the correct by Frame
solution to the wrong problem. In principle
the methods of decision analysis could be
used in selecting a frame; this use would be a
meta-application. The choice of a frame can
usually be accomplished by a guided
conversation. An aid in framing is the
decision hierarchy shown in Figure 3-3. The
frame selected for the decision separates it
from other decisions made previously or
taken as given, shown above, and those to be
made later, shown below.
SENSITIVITY ANALYSIS
/
/\
Taken as
/ Given
/ \
/ To Be Decided Now. \
, \
To Be Decided Later
Figure 3-3 The decision hierarchy.
i:
;
Because all aspects of the problem are explicitly considered, a sensitivity analysis can be
performed on every input. Sensitivity to changes in alternatives, changes in information, and
changes in preference can be determined. Because the formal decision model links every input to
the ultimate value measure, sensitivity analysis is a simple computational task. Judgment is
required to choose the extent of multi-variable sensitivity analysis.
Of particular importance, and unique to decision analysis, is the ability to place a value on the
resolution of any uncertainty, provided a value measure is one of the attributes of the decision
problem. The decision maker can thus determine what it would be worth to resolve any or all of
the uncertainty. If opportunities for information-gathering exist such as surveys, experiments,
pilot plants, prototypes, or market trials, the decision maker must determine whether their value
exceeds their cost. The same logic guides design of the most beneficial experiments and the
application of their results to present and future decisions.
Most important is the notion of framing to ensure that the problem being addressed is the
fundamental problem facing the decision maker. Often the successful practice of decision
analysis requires a complete reframing of the originally contemplated decision. Professional
decision analysts also ensure that the decision maker is actively involved in the decision-making
process, to gain their commitment both to the process and to any resulting decisions.
s
OCR for page 13
IMPLICATIONS FOR EDUCATION AND RESEARCH
The practice of decision
analysis is based not only on the
pioneering contributions from
probability, decision theory, and
systems engineering, but also on
the insights provided in the last few
decades by cognitive psychologists.
These professionals have sensitized
us to the types of mistakes people
make in thinking about decisions,
which can be as subtle as making
decisions based on percentages
rather than absolute amounts. The
cognitive biases to which we are all
subjected must be recognized both
in providing the inputs to a decision
analysis and in appreciating the
necessity for formal analysis in complex decision situations.
. .
What
You
Know
What
You
Can Do
neci~ir)n
What ~:1
· You ~
,
(Sub(con)sequence )
Figure 3-4 The decision process.
The analysis of the decision can be carried out using any of several tools, such as those
shown in Figure 3-4. Examples are shown later in the report. Only a few are illustrated here. The
decision analysis cycle depicts the overall iterative nature of the formulating, analyzing, and
learning process.
Figure 3-5 shows a simple influence or decision diagram for a simple decision. A rectangular
box represents a decision node, an action under the control of the decision maker. Ovals represent
uncertainties or, in some cases, calculations. Finally, hexagons represent the value node, with
Structure
, .
Deterministic
Analysis
Prol~abilistic
Analysis
Appraise
_
Iteration
Strategy
Table
=101~0~1
nit
Influence
Diagram
s
Deterministic
l\/bdel
Decision
Tree
~~ 5~—~ . _
Deterministic
Sensitivity
Probability
Distributions
, ~ ...., ,i I.., ~ .... ~ ...., ~~
.~ A, 1/
rev . . ., v
_
Figure 3-5 Decision diagram.
Value of
Inforrr~tion
r -_
Decision
Quality
me —
j~;
?
OCR for page 14
14
APPROACHES TO IMPROVE ENGINEERING DESIGN
We ~
by:--.,,= ~ ,:
~ ; \
\ ~
\ ail-
| Electric Start |
\ ~ Profit
Market /
Share
1~ I
| ~ Capacity
1~ By\
4'rice/Performance J
Figure 3-6 Decision diagram for design of a dual-sport motorcycle.
inputs showing what the decision maker values. Arrows into any node show what its value
depends on, either functionally or in the sense of conditioned probability. Arrows into decision
nodes show what is known when the decision is made.
In the sample decision diagram in Figure 3-6, note that the manufacturing cost will be known
only when the price decision has been made. Such diagrams have important properties. First, at
the level of relation, they reveal the logical structure of the problem in a way that allows
nontechnical people to understand and contribute. Every relevant thought should have a
representation in the diagram. However, decision diagrams are more than useful graphics. When
-900 -600
High Sport Segment Size in 1994
Market share O n
Competition
Other A
Other B
Unit cost
Development Costs
Other C
Other D
5
At'
Change in NPV "Right Stuff' Strategy
($ millions)
-300 0 300 600
H1.- ~ ~ ~ t~ ~ :~ ~ ~ ~~ ~ ~~ ~~ ~~ ~ ~
ID I'd ;:~::~ t: :~ ~:~:~ ~ a: in:: i- ~~ -: ~ :~ -:: ::: :~ ~ :: -a: a-:-:- :~: ~~:~:~:~ :~ ~ LOW
High ~ Low
Low: High
Low 0 High
Figure 3-7 Tornado diagram
OCR for page 15
5
IMPLI CA TI ONS FOR ED UCA TI ON AND RESEAR CH
15
each decision in the diagram is properly specified, the decisions can be processed directly by a
computer to yield the best overall decision. This method can calculate the value of information
regarding any uncertainty and can also illustrate many of the sensitivities for a preferred course of
action. Decision diagrams also serve as an important communication link among those involved
in the decision.
Another valuable idea is a form of sensitivity analysis known as the tornado diagram because
of its shape. Figure 3-7 illustrates for a particular strategy how the net present value of the final
decision implementation changes as each uncertainty ranges from a low to a high value. A low
value is defined as one with a very high probability of being exceeded, a high value as one with
very low probability of being exceeded. The effect on net present value for each factor is ordered
in decreasing range of impact to produce the tornado shape. This diagram quickly demonstrates
the effect of each uncertainty for a given alternative. A series of tornado diagrams would show
engineers at all levels of the design process the economic implications of their decisions.
A useful guide in assessing decision quality is the spider diagram shown in Figure 3-8. Here
the state of quality in each of the six elements can be plotted as a percentage and then connected
to form a web. In this diagram "100 percent" does not represent perfection but rather a situation in
which further improvement would not be economical. The result of such an assessment is
knowledge of where, if anywhere, to direct additional analysis effort.
Reliable Information
Including Model
Creative
Alternatives
-
-
-
-
1\ /'\
-
\
Appropriate
Frame
Clear
Preferences
/\
:\0% \ 100°/` I Correct
) / Logic
.............................
ii.~$~ .$
~1iii~i
: :':_
a_
/ Commitment
to Action
Figure 3-8 The decision quality spider. 100 percent means further
improvement is not economical.
i:
OCR for page 16
Representative terms from entire chapter:
decision analysis
