Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 23
The Use of Diagnostic Tests:
A Probabilistic Approach
Diagnostic tests and the infonnation that they convey are too often
taken- for granted by both physicians and patients. The most important
error is to assume that the test result is a true representation of what is
really going on. Most diagnostic information is imperfect; although it
changes the physician's perception of the patient, he or she remains
uncertain about the patient's true state.
As an example, consider a hypothetical test. With this test, 10 percent
of patients who have the disease win have a negative result (a false-
negative result), and 10 percent of the patients who do not have the
disease win have an abnormal result (a false-positive result). Thus, when
the result is abnormal, the clinician cannot be certain that the patient has
the disease: abnormal results occur in patients who have the disease and
in patients who do not. There is similar uncertainty if the test result is
negative. As long as tests are imperfect, this uncertainty is intrinsic to the
practice of medicine.
The physician who acknowledges the imperfections of a -diagnostic test
win ask, "In view of this test result, how uncertain should ~ be about this
patient?" Fortunately, there is a method for answering this question: the
theory of probability. This chapter is a primer for applying probability
theory to the interpretation of test results and deciding when to do a test
rather than treat or do nothing.1 It is divided into five parts: (1) first
~ This chapter is adapted from an article written by one of Me authors (Sox 1986~. The
material is covered in greater depth in standard textbooks (Sox et al. 1988, Weinstein
1980~.
23
OCR for page 24
24
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
principles; (2) interpreting test results: the posttest probability; (3) esti-
mating the pretest probability; (4) measuring test performance; (5) ex-
pected-value decisionmaking; and (6) the choice among testing, starting
treatment, or doing nothing.
FIRST PRINCIPLES
The way in which one decides to do a diagnostic test is based on two
· ·
prince es.
PRINCIPLE I: Probability is a Useful Representation of Diagnostic Uncer-
tainty.
Uncertainty is unavoidable. How can we best respond to it? A starting
point is to adopt a common language. Some express their uncertainty as
the probability that the patient has a specified disease. By using probabil-
ity rather than ambiguous terms such as "probably" or "possibly," the
clinician expresses uncertainty quantitatively. More important, probabil-
ity theory allows one to take new information and use Bayes' theorem to
calculate its effect on We probability of disease. These advantages are
compeHing, and our approach to test evaluation is based on providing the
information required to use probability theory to interpret and select
diagnostic tests.
Example: In a patient with chest pain, past history is very useful
when trying to decide whether he or she has coronary artery disease.
Patients whose pain is typical of angina pectons and is also closely
linked to overexertion are said to have "typical angina pectons."
Over 90 percent of men with this history have coronary artery dis-
ease. When anginal pain is less predictably caused by exertion, the
patient is said to have "atypical angina." About two-thirds of men
win this history have coronary artery disease.
Physicians who are uncertain about the meaning of aipatient's
chest pain often ask the padent to undergo an exercise test. The
probability of coronary artery disease after a positive exercise test
may be calculated with Bayes' theorem. If the history is typical
angina, the probability after a positive test is nearly I.0. If the history
is atypical angina, the probability after a positive test is about 0.90.
Comment: Estimating the probability of coronary artery disease
helps to identify the situations in which the probability of disease
will be altered dramatically by an abnormal test.
OCR for page 25
THE USE OF DIAGNOSTIC TESIS
25
PIUNCIPLE lo: A Diagnostic Test Should Be Obtained Only When Its
Outcome Could Alter Me Management of the Patient.
A test should be ordered only when forethought shows that it could
lead to a change in patient management. How does one decide if a test
win alter the management of a patient? There are several considerations.
The elect of a test result on the probability of disease. If the probabil-
ity of disease after the test will be very similar to the probability of disease
before the test, the test is unlikely to affect management. The posttest
probability of disease can be calculated by using Bayes' theorem, as
discussed later in this section.
Example: The probability of coronary artery disease in a person
with typical angina pectoris is 0.90. If an exercise test result is
abnormal, the probability of disease is 0.98. If the result is normal,
the probability of disease is 0.76. Many physicians would conclude
that the effect of the results is too small to make the test worthwhile
for diagnostic purposes.
The threshold mode! of decisionmaking. This approach is based on He
concept that a test is judged by its effect on the probability of disease
(Pauker and Kassirer 1975, 19801. The mode] postulates a treatment
threshold probability, below which treatment is withheld and above which
it is offered. In this situation, a test is only useful if, after it is performed,
the probability of disease has changed so much that it has crossed from
one side of the treatment threshold probability to the over. If the posttest
probability were on the same side of the threshold as He pretest probabil-
ity, the decision of whether or not to treat would be unaffected by the test
results, and the test should not be ordered. One must estimate He benefits
and the harmful effects of treatment in order to set the treatment threshold
probability.
Example: Some patients with suspected pulmonary embolism are
allergic to the contrast agents that are used to perform a pulmonary
arteriogram, the definitive test for a pulmonary embolism. Many
physicians say that if faced with this situation, they would start
anticoagulation if they thought that the patient had as little as a 5 to
10 percent chance of having a pulmonary embolism. Thus, their
treatment threshold probability is 0.05 to 0.10.
Elect of test results on clinical outcomes. Even if a test result leads to
a change in management, if the patient will not benefit, the test should not
OCR for page 26
26
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
have been done. Thus, one is concerned not only with the test itself but
also the efficacy of the actions that are taken when its result is abnormal.
Example: Investigators have calculated Me average improvement in
life expectancy that results from We management changes foBow~ng
coronary arter~ography in patients wad stable angina pectons. The
analysis shows that mid~e-aged men win gain, on average, approxi-
mately one year from undergoing coronary artenography and coro-
nary bypass surgery if severe disease is present (Stason and Fin-
eberg 1982~. This test does have an effect on clinical outcomes.
Marginal cost-effectiveness of the test. This measure of test perform-
ance is a way to characterize me efficiency with which additional re-
sources (dollars) are translated into outcomes Longevity). It takes into
account the increased costs from doing a test and me incremental benefit
to the patient. A test result may lead to a good outcome, such as improved
longevity, but the increase in cost for each unit of increase in longevity
may be so high that there is a consensus that the test should not be done.
INTERPRETING TEST RESULTS: THE POSTTEST
PROBABILITY
The inte~pretation.of a test result is an important part of technology
assessment. A test with many false-negative and false-positive results
will be interpreted with far more caution than a test with few such
misleading results. Therefore, measuring the performance characteristics
of a test is important, because the clinician must know them in order to
interpret the result.
Important Definitions
The probability of disease after leaping the results of a test is called
the posttest probability of disease. It is the answer to the question, '~What
does this test result mean?" One calculates the posttest probability with
B ayes ' theorem, which is denved from the first principles of probability
and requires both the pretest probability of disease and two measures of
the accuracy of the test. One measure is called the sensitivity of the test
2 See also He Glossary of Terms at the end of dais chapter.
OCR for page 27
THE USE OF DIAGNOSTIC TESTS
27
(true-positive rate, or TPR). It represents the likelihood of a positive test
in a diseased person, as is shown in the following equation:
Sensitivity =
number of diseased patients with positive test
number of diseased patients
.
Example: There have been many studies of the exercise electrocar-
diogram. In these studies, a padent with-chest pain undergoes both
the exercise electrocardiogram and a definitive test for coronary
artery disease, the coronary arter~ogram. About 70 percent of pa-
tients who had a positive artenogram also had a positive exercise
electrocardiogram (as defined by the presence of at least ~ mm of
horizontal or downsioping ST segment depression). Thus, accord-
ing to this result, the sensitivity of an exercise electrocardiogram for
coronary artery disease is 0.70.
The second measure of Test accuracy is its false-positive rate, Me
likelihood of a positive result in a patient without disease. Specificity, the
true-negative rate (TNR), is 1 minus the false-positive rate.
False- number of nondiseased patients with positive test
positive =
rate
number of nondiseased patients
.
Example: The studies of the exercise electrocardiogram have shown
mat about 15 percent of patients who did not have coronary artery
disease nonetheless did have an abnormal exercise electrocardio-
gram. Thus, the false-positive rate of the exercise electrocardiogram
for coronary artery disease is 0.15.
Likelihood ratio. The likelihood ratio is a measure of how much the
result alters the probability of disease.
Likelihood ratio
probability of result in diseased patients
probability of result in nondiseased patients
.
We can use this definition to define a positive test result and a negative
test result. A positive test result raises the probability of disease, and its
OCR for page 28
28
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
likelihood ratio is TO. The likelihood ratio for a positive test result is
abbreviated as ER(+3. A negative test result lowers Me probability of
disease, and its likelihood ratio is between 0.0 and I.0. The likelihood
ratio for a negative test result is abbreviated ER(-3.
Example: If an exercise test is positive, me likelihood ratio is 0.70
divided by O.IS, or 4.666. The odds of having coronary artery
disease increase by a factor of 4.666 if an exercise test is abnormal.
(Odds are defined in me Glossary of Terms.) If an exercise test is
negative, the likelihood ratio is 0.30 divided by 0.85, or 0.35. When
an exercise test is negative, the odds of having coronary artery
disease are 0.35 times the pretest odds.
Bayes' theorem uses data on test performance in me following way. In
these formulas, TPR (true-positive rated is used in place of sensitivity, and
FPR is used to denote false-positive rate. TNR denotes the true-negative
rate, and FAR denotes the false-negative rate These terms are defined in
me Glossary). The pretest probability of disease is represented by p(D).
Probability of
disease if test =
· · —
IS positive
Probability of
disease if test =
is negative
p(D) x TPR
.
p(D) x TPR + [ 1 - p(D)] x FPR
p(D) x FAR
p(D) x E;NR + [1 - p(D3] x TNR
The probability of a positive test result equals the probability of a true-
positive result plus He probability of a false-positive result.
Probability
of positive = p(D)xTPR + ([l-p(D)]xFPR.
test result
B ayes ' theorem can be written in a simplified way that facilitates cal-
culation. This form is called the odds-ratio form of Bayes' theorem.
Posttest odds = pretest odds x likelihood ratio.
OCR for page 29
THE USE OF DIAGNOSTIC TESTS
29
Example: A clinician is planning to use an exercise test with a
sensitivity (TPR) of 0.7 and a false-positive rate (FPR) of 0.15.
Suppose the pretest probability of disease, p(D), is 0.30:
Probability p(D) x TPR
of disease if = - -
test positive p(D) x TPR + t} - P(D)] x FPR
.30 x .70 .21
.30 x .70 + .70 x .15 .21 + .105
= .667.
The pretest odds are .30/.70 = 0.43 to I.0. The likelihood ratio for the
test is .70/.15 = 4.667.
Posttest odds = pretest odds x likelihood ratio
= 0.43 x 4.667 = 2.0 to 1.0.
Odds of 2.0 to I.0 are equivalent to a probability of 0.66.
The importance of Bayes' theorem in interpreting a test is that it defines
the relationship between pretest probability and posttest probability, which
is shown in Figure 2.~. The relationship between these two entities has
several implications.
The interpretation of a test result depends on the pretest probability of
disease. If a result is positive, the posttest probability increases as the
pretest probability increases (Figure 2.la). If the result is negative, the
posttest probability decreases as the pretest probability decreases (Figure
2.Ib). The consequence of this relationship is that one cannot properly
interpret the meaning of a test result without taking into account what was
known about the patient before doing the test. This statement is inescapa-
bly true, because it is based on first pnnciples of probability theory.
The effect of a test result depends on the pretest probability. The
vertical distance between the 45-degree line in Figure 2. ~ and the cuIve is
the difference between the pretest and Me posttest probability.
When the clinician is already quite certain of the patient's true state,
the probability of a disease is either very high or very low. When the
pretest probability is very low, a negative test has little effect, and a
positive test has a large effect. When the probability is very high, a
OCR for page 30
30
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
1 .0- ,
LL
0.8
_ ~
6 - 0.6
{D ~
o o
~ CL
CL C,0
— 0.4
en ~
~ CO
In
o
0.2
/
/
-
/
0~0' 1 1 1 1 1 1 1 1 1 1
0.0 0.2 0.4 0.6 0.8 1.0
PRETEST PROBABILITY
FIGURE 2.1 Relationship between pretest probability and pastiest probability of disease.
Figure 2.1a
The pastiest probability of disease corresponding to a positive test result was calculated
with B. ayes ' theorem for all values of the pretest probability. The sensitivity and specificity
of the hypothetical test were both assumed to be 0.90.
negative test has a considerable effect, and a positive test has lithe effect.
This example shows Hat a test result that confirms one's prior judgment
has little effect on the probability of disease. Tests have large effects
when the probability of disease is intermediate, which corresponds to
clinical situations in which the physician is quite uncertain. Tests can also
be useful when Weir result does not confirm the prior clinical impres-
sion—for example, a negative result in a patient who is thought vein
likely to have a disease.
The pretest probability affects the probability that a positive or nega-
live test result wid occur. The higher the pretest probability, the more
likely one is to experience a positive test. Conversely, a negative test is
less likely as the pretest probability increases.
OCR for page 31
THE USE OF DIAGNOSTIC TESTS
1.0
-
-
-
J
_ _
m ~
0.6
m c,
O ~
C: Z
- ~ 0.4
co ~
In
o
31
0.8
0.2
/
/
-
/
/
/
/
l
0.0- ~ I- . , . . . 1 .
0.0 0.2 0.4 0.6 0.8 1.0
PRETEST PROBABILITY
Figure 2.lb
The pastiest probability of disease corresponding to a negative test result was calcu-
lated win B ayes' theorem for ~1 values of Me pretest Probability. The sensitivity and
specificity of the test were both assumed to be 0.90.
The posttest probability depends on the sensitivity and the false-posi-
tive rate of the diagnostic test. This relationship is one reason to be
concerned about measuring test performance accurately.
The Assumptions of Bayes' Theorem
B ayes ' theorem is derived from first principles of probability theory.
Therefore, when it is used correctly, the result is reliable. Errors in using
Bayes' theorem can occur when people ignore several assumptions.
One assumption of Bayes' theorem is mat sensitivity and specificity
are constant, regardless of the pretest probability of disease. This assump-
tion can be false. A test may be less sensitive in detecting a disease in an
early stage, when the pretest probability is low, Han it would be in an
OCR for page 32
32
ASSESSMEN7 OF DIAGNOSTIC TECHNOLOGY
advanced stage, when mere are many signs and symptoms and the pretest
probability is high. This error may be avoided by dividing the study
population into subgroups that differ in the extent of clinical evidence for
disease (Weiner et al. 1979~.
A second assumption is that the sensitivity and false-positive rate of a
test are independent of the results of other tests. This conditional in~e-
pen~ence assumption! is important when Bayes' theorem is used to calcu-
late the probability of disease after a sequence of tests. The posttest
probability after the first test in a sequence is used as the pretest probabil-
ity for the second test. In an ideal study of two tests, bow tests in the
sequence and a definitive diagnostic procedure have been performed on
many patients. The sensitivity and specificity of the second test in the
sequence are calculated twice: in patients with a positive result on the first
test and in patients with a negative result on the first test. If the sensitivity
and specificity of the second test are the same, they are said to be
conditionally independent of the results of He first test, and the condi-
tional independence assumption is valid. In practice, the conditional inde-
pendence assumption is seldom tested, and the clinician should be cau-
tious about using recommendations for sequences of tests.
THE PRETEST PROBABILITY OF DISEASE
Why is the pretest probability of disease an important concept in
understanding He assessment of diagnostic technology? The pretest proba-
bility is required to calculate the posttest probability of disease, and thus
to interpret a diagnostic test; it is also the cornerstone of the decision
whether to treat, to test, or to do nothing. A patient's pretest probability
of disease encodes He individual's own clinical findings and is one of the
ways in which a decision can be tailored to the patient. Knowing how to
estimate the pretest ~ probability is an essential clinical skill and is de-
scr~bed in the section that follows.
When is testing particularly useful? Padents are particularly unlikely
to benefit from testing when the pretest probability is very high or very
low. If the pretest probability is very high, the physician is likely to treat
the patient unless a negative result raises doubts about the diagnosis. The
posttest probability of disease after a negative test may be so high that
treatment is still indicated.
Example: In a patient with typical angina pectoris, the posttest
probability aher a negative exercise test is 0.76. Most physicians
OCR for page 33
THE USE OF DIAGNOSTIC TESTS
33
would begin medical treatment for coronary artery disease even if
the probability of disease were considerably less than 0.76. For
these physicians, the decision to treat would not be affected by the
nonnal exercise test result.
If the pretest probability is very low, as occurs in screening asymptomatic
individuals, the clinician is likely to do nothing unless a positive test
result raises concern. If, for example, the pretest probability is less than
0.001, the posttest probability may be less than 0.01. In this situation, a
change in management is not indicated.
Figure 2. ~ shows that the greatest benefit from testing is likely to occur
when the pretest probability of disease is intermediate. This corresponds
to a clinical situation in which Mere is uncertainty about the patient's true
state. Patients are also likely to benefit from testing when the pretest
probability is close to a treatment threshold probability. At this point, it
requires only a small change in the probability of disease to cross the
threshold and alter management.
Physicians customarily use their intuition to estimate me probability of
disease. The two principal influences on probability estimates are per-
sonal experience and the published literature.
Using personal experience to estimate probability. To estimate proba-
bility, the physician should recall patients with characteristics similar to
the patient in question, and then try to recall what proportion of these
patients had disease. This cognitive task is forbiddingly difficult. In
practice, the assignment of a probability to a clinical situation is largely
guesswork.
There are several cognitive principles for estimating probability (Tver-
sky and Kahneman 1974~. These principles are caned heuristics.
A clinician is using the representat~veness heuristic when he or she
operates on the principle Cat "If the patient looks like a typical case, he
probably has the disease." Thus, if a patient has all the findings of
Cushing's disease, he is thought very likely to have the disease itself.
The representativeness heuristic can be misleading, because it leads the
physician into ignoring the overalD prevalence of a disease. It can also
lead to error if the patient's findings are poor predictors of disease or if the
physician overestimates probability when there are many redundant pre-
dictors. Additionally, the clinician's internal representation of the disease
may be incorrect because it is based on a small, atypical personal experi-
ence.
Clinicians are using the availability heuristic when they judge the
OCR for page 44
44
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
su rgery
' 0=.05
medical treatment
survive
pain unchanged
—|post-op death |
40 [pain unchanged!
lPain resolved I
-
fIGURE 2~ A decision tree for deciding between surgery ar~d medical management of
chronic pancreaiitis. The square represents a decision node. The circles represent chance
nodes. The square represents a decision node. The rectangles represent terminal nodes for
He venous outcome states, and the numbers enclosed within the rectangles are the products
of Be length of life and the quality of life in the outcome state.
Setting Up the Decision Tree
Surgery: The first node on the surgery branch is a chance node
(represented by a circle), which represents uncertainty about whether
the patient would survive the operation. The patient may survive or
may die, but the true outcome of the operation is unknown and can
only be represented by a probability. On average, the mortality rate
of the operation is 5 percent, which seemed a reasonable representa-
tion for this padent, who was otherwise well. The next uncertainty
was the outcome of treatment. Only about 60 percent of patients
obtain relief of pain after surgery. The possible outcomes are
represented by terminal nodes (shown as a rectangle). Each out-
come is assigned a quantitative measure, such as the life expectancy
in that outcome state. This patient's life expectancy was 20 years.
Medical treatment: Because management associated with the medical
option does not change, there are no chance nodes, and the patient's life
expectancy is 20 years.
OCR for page 45
THE USE OF DIAGNOSTIC TESIS
45
Weighing the Outcomes for Quality of Life
The patient's life expectancy was 20 years if he survived the operation,
and it was thought to be the same regardless of whether he experienced
chronic pain or was pan-free. The patient pointed out that 20 years of life
with chronic pain was equivalent to 12 years of being pain-free. In other
words, to be free of pain he was wining to give up eight years of life win
chronic pain. This memos for weighing the length of life in a certain state
of health by a factor that represents the quality of life in that state is called
the "time trade-off' method. It is described in standard textbooks (Wein-
stein et al. 1980, Sox et al. 19881.
Calculating the Expected Value of the Treatment Options
The average (or expected) outcome is calculated by taking the product
of aU the probabilities along a path to a terminal node and multiplying it
by the value assigned to Me terminal node. The management alternative
with the highest expected value is usually the preferred choice. In this
case, the expected length of life, measured in healthy years, was 16.8
years for surgery and 12 years for medical management. The surgeons
were convinced by this analysis and scheduled the patient for surgery.
Note that expected-value decisionmaking allows one to balance the
risks and benefits of treatment. These factors are usually considered
intuitively. By assigning a value to each outcome and weighing it by the
chance that it win occur, expected-value decisionmaking alBows one to
integrate risks and benefits.
THE CHOICE AMONG DOING NOTHING, TESTING, OR
STARTING TREATMENT
The art of medicine is making good decisions with inadequate data.
Physicians often must start treatment when stilU uncertain about whether
the patient has the disease for which He treatment is intended. If treat-
ment is started, there is a risk of causing harm to a person who does not
have the disease, as well as the prospect of benefiting the person who
does. If treatment is withheld, a person who is diseased win be denied a
chance at a rapid, effective cure. This situation is often unavoidable, and
the physician has three choices.
OCR for page 46
46
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
· Do nothing: the chance of disease is low, treatment is either hammed
or ineffective (or both), and a false-positive result might occur and lead to
handful treatment for someone who does not have disease.
· Get more irg~ormatzon: do a test or observe the patient's course in the
hope that Me correct choice win become apparent.
· Start treatment now: the chance of disease is relatively high, treat-
ment is safe and effective, and a false-negative result might lead to
withholding useful treatment from someone with disease.
The method for solving this problem analytically is caned me thresh-
old mode! of medical decisionmaking (Pauker and Kassirer 1975' 1980;
Doubilet 19X3~. The threshold mode] is an example of expected-value
decisionmaking that is applied to a particular type of decision.
The key idea is We treatment threshold probability, which is the
probability of disease at which one is indifferent between treating and not
treating. The basic principle of the threshold mode] is the following
dictum: Do a test only if the probability of disease could change enough
to cross the treatment threshold probability. Three steps are required to
translate this idea into action.
Step ·: Estimate the pretest probability of disease.
Step 2: Set the treatment threshold probability. This step is difficult
because it requires the clinician to express the balance of the risks
and benefits of treatment in a single number. One can use clerical
intuition to set the treatment threshold probability. This task is made
easier by the following relationship:
C
Treatment threshold =
C + B
where C is the cost of treating nondiseased patients, and B is the
benefit of treating diseased patients (Pauker and Kassirer 1975~.
The cost and the benefit must be expressed in the same units, which
can be dollars, life expectancy, or a measure of the patient's attitudes
toward treatment and the disease.
Note that when the costs of treating nondiseased patients equal the
benefits of treating diseased patients, the treatment threshold is 0.5. Thus,
for many treatments, the treatment threshold probability will be less than
OCR for page 47
THE USE OF DIAGNOSTIC TESTS
47
0.50. For a safe, beneficial treatment, such as antibiotics for commuruty-
acquired pneumonia, the treatment threshold probability may be less than
0.10. If there is good reason to suspect disease, the pretest probability win
be above the treatment threshold. In deciding whether to perform a test,
Me clinician must ask whether the posttest probability after a negative test
result would be below the treatment threshold probability. This question
is answered by taking Step 3, descnbed below.
One can also use analytic methods to set the treatment threshold proba-
biiity (Sox et al. 1988~. Consider me decision tree in Figure 2.6, which
shows a hypothetical problem in which treatment must be chosen despite
uncertainty about whether the patient has the disease for which the treat-
ment is intended.
To use We decision tree to estimate the treatment threshold, recall that
this threshold is the probability of disease at which one is indifferent
between treating and not treating. First, one assigns values to each of the
probabilities and outcome states except for the probability of disease.
Second, one calculates the expected value of the two options, leaving the
probability of disease as an unmown. Third, one sets the expression for
the expected value of the treatment option equal to that of the nontreat-
ment option. Fourth, one solves for the probability of disease.
To use a decision tree, one must assign a probability to each chance
node and a numerical value to each outcome state. The latter value could
be life expectancy. Altematively, as shown in Figure 2.6, one could
assign each outcome state a Utility which is a quantative measure of
relative preference. A utility of I.0 is assigned to the best outcome, and a
utility of 0.0 to the worst. The utility of each intermediate outcome state
is then assessed on this scale of 0.0 to I.0. When utility is used as the
measure of outcome, the altemative with the highest expected utility
should be the preferred alternative.
Step 3: Use Bayes' theorem to calculate the posttest probability of
disease. If the pretest probability is above the treatment threshold,
one must calculate the probability of disease if the test is negative. If
the pretest probability is below the treatment threshold, one must
calculate the probability of disease if the test is positive.
If the pretest probability is far enough above or below the treatment
threshold, a test result will not affect management because the posttest
probability will be on the same side of the treatment threshold as the
pretest probability. There is a pretest probability for which the posttest
probability is exactly the point at which one is indifferent between not
OCR for page 48
48
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
operative
death
p=03 U O
no tumor
SURGERY
I,
NO SURGERY
1
L
tumor
survive
surgery
U=.98
operative
death
p_.O3 U O
I P[tumor] ~ Tsurvive ~
r
tumor U=.25
,
|p[tumorj|
U=.98
p=.48
no cure U 23
|no tumor U 1 0
FIGURE 2.6 A decision Bee for choosing between treatment and no treatment when the
clinician does not know whether He patient has the disease for which treatment is indi-
cated.
OCR for page 49
THE USE OF DIAGNOSTIC TESTS
if}
49
............................................................................................................................
. ~ ...........................................................................
... . . ... at re at
,,, ., ,., ., , ~
......... .....
DON'T TEST
[ no treat|
~ '~ ~
DON'T TEST | | no treat| |
· ;.;.;;; ; ; ;;;; ; ;;; ;; ;;;; ;; ;;; ;;;;; ;;; ;;; ; ;;; ; ;
.............................................................................................
................................. ...........................................................
............................................................ ~ ~
.,, ,., ,,,.,, treat
.........................................................................................................................................................................................
TOSS-UP Llnotreatl
.............................................................................................
treat
...................................................... ...........................................................................
................................................................ ...........................................................................
- 3~
. ...............................
................................................................... ...........................................................................
TCOT ~ ~ ~ ~
. _-, no treat area ~
................................................................... ..... -.- - - - .......................................................................
...........................
FIGURE 2.7 Illustration of how to set die no treat-test threshold. As pO, the pretest
probability, is gradually increased, the pastiest probability is first below the treatment
threshold, then equal to it, and finally above it. At the point where pO equals the treatment
threshold, one should be indifferent between not treating and testing. This probability is
the no treat-test threshold.
treating and testing (the treatment threshold probability). Below this
pretest probability, caned the no treat-test threshold (Pauker and Kassirer
1980), a positive test result could not increase the probability of disease
enough to cross the treatment threshold, and both testing and treatment
should be withheld. Above this threshold, the posttest probability will
exceed the treatment threshold, and testing is indicated. These concepts
are illustrated in Figure 2.7.
OCR for page 50
so
ASSESSMENT OF DIAGNOSTIC TECHNOLOGY
|do nothing | ~
, -
; l
~ l
l ,
|treatment |
threshold pretest
p[D]
~ ~ '
t.~.~.~.~ ~ ~ ~ ~ ~ ~ ~ ~ ~ "~".'."'"""""''""""~"~"""""'"""''t""'"''"""""~'"."'""''"'."'~'e'e''"
T E ~ T
.... .. , :., . . ;
.... .. ...... ........ .........
~ _
.50 1.0
probability of disease
FIGURE 2~8 Using the treatment threshold probability to help decide whether to do a test.
One can use the same approach to calculate the point at which one
should be indifferent between testing and treating (the test-treatment
threshold). Both of these thresholds are a function of the true-positive rate
of the test, the false-positive rate of the test, the treatment threshold, and a
measure of what experiencing the test means to the patient (the cost of the
test). Figure 2.8 shows the three zones of the probability scale. Using
Figure 2.8, one needs only to estimate the pretest probability to know
whether testing is the preferred action or whether one should treat or do
nothing.
SUMMARY
The purpose of this chapter is to provide a working knowledge of how
probability theory and expected-value decisionmaking are used to help
make decisions about diagnostic testing. Past studies of diagnostic tests
have measured only test performance. A complete evaluation should
provide information about the treatment threshold and how to estimate the
pretest probability of disease. With this information, the clinician can
decide when a test will alter management and can use the results to choose
the action that will most benefit the patient.
OCR for page 51
THE USE OF DMGNOSTIC ~:STS
GLOSSARY OF TERMS
51
Bayes' theorem: an algebraic expression for calculating the posttest
probability of disease if the pretest probability of disease [p(D)] and the
sensitivity and specificity of a test are known.
Clinically relevant population: the patients on whom a test is normally
used.
Cost-effectiveness analysis: comparison of clinical policies in teens of
their cost for a unit of outcome. Marginal cost-~ectiveness: the increase
in cost of a policy for a unit increase in outcome.
False-negative rate: the likelihood of a negative test result In a diseased
patient (abbreviated F~.
False-negative result: a negative result in a patient with a disease.
False-positive rate: the likelihood of a positive test result in a patient
without a disease (abbreviated FPR).
False-positive result: a positive result in a person who does not have the
disease.
Gold-standard test: the test or procedure that is used to define the true
state of the patient.
Index test: He test for which performance is being measured.
Likelihood! ratio: a measure of discrimination by a test result. A test
result with a likelihood ratio >~.0 raises the probability of disease and is
often referred to as a "positive" test result. A test result with a likelihood
ratio <~.0 lowers the probability of disease and is often called a "nega-
tive" test result.
Likelihood ratio =
probability of result in diseased persons
probability of result in nondiseased persons
.
OCR for page 52
52
ASSESSMENT OF DIAGNO=IC TECHNOLOGY
Negative test result: a test result mat occurs more frequency in patients
who do not have a disease than in patients who do have the disease.
Odds: the probability.
Odds =
probability of event
- probability of event
. —
Positive test result: a test result that occurs more frequently in patients
with a disease than in patients who do not have the disease.
Posttest probability: the probability of disease after the results of a test
have been reamed (synonyms: posterior probability, posttest risk).
Predictive value negative: probability of the absence of the disease if a
test is negative.
Predictive value positive: probability of a disease if a test is positive.
Pretest probability: the probability of disease before doing a test (syno-
nyms: prior probability, pretest risk).
Probability: an expression of opinion, on a scale of 0.0 to I.0, about the
likelihood that an event will occur.
Sensitivity: the likelihood of a positive test result in a diseased person
(synonym: true-positive rate, abbreviated TPR).
Sensitivity =
number of diseased patients with positive test
number of diseased patients
.
Specificity: the likelihood of a negative test result in a patient without
disease (synonym: true-negative rate; abbreviated TAR).
Specificity =
number of nondiseased patients win negative test
number of nondiseased patients
-
OCR for page 53
THE USE OF DIAGNOSTIC TESTS
53
Study population: the patients for whom test performance is measured
(usually a subject of the clinically relevant population).
Treatment threshold probability: the probability of disease at which the
clinician is indifferent between withholding treatment and giving treat-
ment. Below the threshold probability, treatment is withheld; above the
threshold, treatment is given.
True-negative result: a negative test result in a person with a disease.
True-positive result: a positive test result in a person with a disease.
REFERENCES
Doubilet, P. A mathematical approach to interpretation and selection of
diagnostic tests. Medical Decision Making 3:177-195, 1983. -
Goldman, L., Caldera, D.L., Nussbaum, S., et al. Multifactorial index of
cardiac risk in non-cardiac surgical procedures. New England Jour-
nal of Medicine 297:845-850, 1977.
Greener, P.F., Mayewski, Ret., Mushlin, A.I., and Greenland, P. Selection
and interpretation of diagnostic tests and procedures: Principles and
applications. Annals of Internal Medicine 94(part 2~:553-560, 1981.
ouye, S.K., and Sox, H.C. Standard and computed tomography in the
evaluation of neoplasms of the chest. Annals of Internal Medicine
105:906-924, 1986.
Metz, C.E. Basic principles of ROC analysis. Seminars in Nuclear
Medicine 8:283-298, 1978.
Pauker, S.G., and Kassirer, I.P. The threshold approach to clinical deci-
sion making. New England Journal of Medicine 302:1109-1117,
1980.
Pauker, S.G., and Kassirer, J.P. Therapeutic decision making: A cost-
benefit analysis. New England Joumal of Medicine 293:229-234,
1975.
Philbrick, I.T., Horwitz, At., Feinstein, A.R., Langou, R.A., and Chan-
dler, I.P. The limited spectrum of patients studied in exercise test
research: Analyzing the tip of the iceberg. Journal of the American
Medical Association 248:2467-2470, 1982.
Philbrick, I.T., Horwitz, R.~., and Feinstein, A.R. Methodology problems
of exercise testing for coronary artery disease: Groups, analysis, and
bias. American Joumal of Cardiology 46:807-812, 1980.
OCR for page 54
54
ASSESSMENT OF DIAGNOSTIC TEClINOLOGY
Ransohoff, D.F., and Feinstein, A.R. Problems of spectrum and bias in
evaluating the efficacy of diagnostic tests. New England Joumal of
Medicine 299:926-930, 1978.
Sox, H.C., Blatt, M.A., Higgins, M.C., and Marton, K.~. Medical Deci-
sion Making. Boston, Buttenvor~, 1988.
Sox, H.C. Probability theory in the use of diagnostic tests: An inDoduc-
tion to critical study of He literature. Annals of Intema] Medicine
104:60-66, 1986.
Stason, W.B., and Fineberg, H.V. Implications of alternative strategies to
diagnose coronary artery disease. Circulation 66(Suppl. III):80-86,
1982.
Tversky, A., and Kahneman D. Judgment under uncertainty: Heuristics
and biases. Science 185:1124-1131, 1974.
Wasson, I.H., Sox, H.C., Neff, R.K., and Goldman, Lo. Clinical prediction
rules: Applications and methodologic standards. New En~lan`1 Imlr
nal of Medicine 313:793-799. 1985.
~ ~ _ _ _
Weiner, D.A., Ryan, T.~., McCabe, C.H., et al. Exercise stress testing:
Correlation among history of angina, ST-segment response and preva-
lence of coronary-artery disease in the Coronary AItery Surgery
Study (CASS). New England Joumal of Medicine 301:230-235,
1979.
Weinstein, M.C., Fineberg, H.V., Elstein, A.S., Frazier, H.S., Neuhauser,
D., Neutra, R.R., and McNeil, B.~. Clinical Decision Analysis. Phila-
delphia, W.B. Saunders, 1980.
Weinstein, M.C., and Stason, W.B. Foundations of cost-effectiveness
analysis for health and medical practices. New England Joumal of
Medicine 296:716-721, 1977.
Representative terms from entire chapter:
test result
