Click for next page ( 24


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 23
The Use of Diagnostic Tests: A Probabilistic Approach Diagnostic tests and the infonnation that they convey are too often taken- for granted by both physicians and patients. The most important error is to assume that the test result is a true representation of what is really going on. Most diagnostic information is imperfect; although it changes the physician's perception of the patient, he or she remains uncertain about the patient's true state. As an example, consider a hypothetical test. With this test, 10 percent of patients who have the disease win have a negative result (a false- negative result), and 10 percent of the patients who do not have the disease win have an abnormal result (a false-positive result). Thus, when the result is abnormal, the clinician cannot be certain that the patient has the disease: abnormal results occur in patients who have the disease and in patients who do not. There is similar uncertainty if the test result is negative. As long as tests are imperfect, this uncertainty is intrinsic to the practice of medicine. The physician who acknowledges the imperfections of a -diagnostic test win ask, "In view of this test result, how uncertain should ~ be about this patient?" Fortunately, there is a method for answering this question: the theory of probability. This chapter is a primer for applying probability theory to the interpretation of test results and deciding when to do a test rather than treat or do nothing.1 It is divided into five parts: (1) first ~ This chapter is adapted from an article written by one of Me authors (Sox 1986~. The material is covered in greater depth in standard textbooks (Sox et al. 1988, Weinstein 1980~. 23

OCR for page 23
24 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY principles; (2) interpreting test results: the posttest probability; (3) esti- mating the pretest probability; (4) measuring test performance; (5) ex- pected-value decisionmaking; and (6) the choice among testing, starting treatment, or doing nothing. FIRST PRINCIPLES The way in which one decides to do a diagnostic test is based on two prince es. PRINCIPLE I: Probability is a Useful Representation of Diagnostic Uncer- tainty. Uncertainty is unavoidable. How can we best respond to it? A starting point is to adopt a common language. Some express their uncertainty as the probability that the patient has a specified disease. By using probabil- ity rather than ambiguous terms such as "probably" or "possibly," the clinician expresses uncertainty quantitatively. More important, probabil- ity theory allows one to take new information and use Bayes' theorem to calculate its effect on We probability of disease. These advantages are compeHing, and our approach to test evaluation is based on providing the information required to use probability theory to interpret and select diagnostic tests. Example: In a patient with chest pain, past history is very useful when trying to decide whether he or she has coronary artery disease. Patients whose pain is typical of angina pectons and is also closely linked to overexertion are said to have "typical angina pectons." Over 90 percent of men with this history have coronary artery dis- ease. When anginal pain is less predictably caused by exertion, the patient is said to have "atypical angina." About two-thirds of men win this history have coronary artery disease. Physicians who are uncertain about the meaning of aipatient's chest pain often ask the padent to undergo an exercise test. The probability of coronary artery disease after a positive exercise test may be calculated with Bayes' theorem. If the history is typical angina, the probability after a positive test is nearly I.0. If the history is atypical angina, the probability after a positive test is about 0.90. Comment: Estimating the probability of coronary artery disease helps to identify the situations in which the probability of disease will be altered dramatically by an abnormal test.

OCR for page 23
THE USE OF DIAGNOSTIC TESIS 25 PIUNCIPLE lo: A Diagnostic Test Should Be Obtained Only When Its Outcome Could Alter Me Management of the Patient. A test should be ordered only when forethought shows that it could lead to a change in patient management. How does one decide if a test win alter the management of a patient? There are several considerations. The elect of a test result on the probability of disease. If the probabil- ity of disease after the test will be very similar to the probability of disease before the test, the test is unlikely to affect management. The posttest probability of disease can be calculated by using Bayes' theorem, as discussed later in this section. Example: The probability of coronary artery disease in a person with typical angina pectoris is 0.90. If an exercise test result is abnormal, the probability of disease is 0.98. If the result is normal, the probability of disease is 0.76. Many physicians would conclude that the effect of the results is too small to make the test worthwhile for diagnostic purposes. The threshold mode! of decisionmaking. This approach is based on He concept that a test is judged by its effect on the probability of disease (Pauker and Kassirer 1975, 19801. The mode] postulates a treatment threshold probability, below which treatment is withheld and above which it is offered. In this situation, a test is only useful if, after it is performed, the probability of disease has changed so much that it has crossed from one side of the treatment threshold probability to the over. If the posttest probability were on the same side of the threshold as He pretest probabil- ity, the decision of whether or not to treat would be unaffected by the test results, and the test should not be ordered. One must estimate He benefits and the harmful effects of treatment in order to set the treatment threshold probability. Example: Some patients with suspected pulmonary embolism are allergic to the contrast agents that are used to perform a pulmonary arteriogram, the definitive test for a pulmonary embolism. Many physicians say that if faced with this situation, they would start anticoagulation if they thought that the patient had as little as a 5 to 10 percent chance of having a pulmonary embolism. Thus, their treatment threshold probability is 0.05 to 0.10. Elect of test results on clinical outcomes. Even if a test result leads to a change in management, if the patient will not benefit, the test should not

OCR for page 23
26 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY have been done. Thus, one is concerned not only with the test itself but also the efficacy of the actions that are taken when its result is abnormal. Example: Investigators have calculated Me average improvement in life expectancy that results from We management changes foBow~ng coronary arter~ography in patients wad stable angina pectons. The analysis shows that mid~e-aged men win gain, on average, approxi- mately one year from undergoing coronary artenography and coro- nary bypass surgery if severe disease is present (Stason and Fin- eberg 1982~. This test does have an effect on clinical outcomes. Marginal cost-effectiveness of the test. This measure of test perform- ance is a way to characterize me efficiency with which additional re- sources (dollars) are translated into outcomes Longevity). It takes into account the increased costs from doing a test and me incremental benefit to the patient. A test result may lead to a good outcome, such as improved longevity, but the increase in cost for each unit of increase in longevity may be so high that there is a consensus that the test should not be done. INTERPRETING TEST RESULTS: THE POSTTEST PROBABILITY The inte~pretation.of a test result is an important part of technology assessment. A test with many false-negative and false-positive results will be interpreted with far more caution than a test with few such misleading results. Therefore, measuring the performance characteristics of a test is important, because the clinician must know them in order to interpret the result. Important Definitions The probability of disease after leaping the results of a test is called the posttest probability of disease. It is the answer to the question, '~What does this test result mean?" One calculates the posttest probability with B ayes ' theorem, which is denved from the first principles of probability and requires both the pretest probability of disease and two measures of the accuracy of the test. One measure is called the sensitivity of the test 2 See also He Glossary of Terms at the end of dais chapter.

OCR for page 23
THE USE OF DIAGNOSTIC TESTS 27 (true-positive rate, or TPR). It represents the likelihood of a positive test in a diseased person, as is shown in the following equation: Sensitivity = number of diseased patients with positive test number of diseased patients . Example: There have been many studies of the exercise electrocar- diogram. In these studies, a padent with-chest pain undergoes both the exercise electrocardiogram and a definitive test for coronary artery disease, the coronary arter~ogram. About 70 percent of pa- tients who had a positive artenogram also had a positive exercise electrocardiogram (as defined by the presence of at least ~ mm of horizontal or downsioping ST segment depression). Thus, accord- ing to this result, the sensitivity of an exercise electrocardiogram for coronary artery disease is 0.70. The second measure of Test accuracy is its false-positive rate, Me likelihood of a positive result in a patient without disease. Specificity, the true-negative rate (TNR), is 1 minus the false-positive rate. False- number of nondiseased patients with positive test positive = rate number of nondiseased patients . Example: The studies of the exercise electrocardiogram have shown mat about 15 percent of patients who did not have coronary artery disease nonetheless did have an abnormal exercise electrocardio- gram. Thus, the false-positive rate of the exercise electrocardiogram for coronary artery disease is 0.15. Likelihood ratio. The likelihood ratio is a measure of how much the result alters the probability of disease. Likelihood ratio probability of result in diseased patients probability of result in nondiseased patients . We can use this definition to define a positive test result and a negative test result. A positive test result raises the probability of disease, and its

OCR for page 23
28 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY likelihood ratio is TO. The likelihood ratio for a positive test result is abbreviated as ER(+3. A negative test result lowers Me probability of disease, and its likelihood ratio is between 0.0 and I.0. The likelihood ratio for a negative test result is abbreviated ER(-3. Example: If an exercise test is positive, me likelihood ratio is 0.70 divided by O.IS, or 4.666. The odds of having coronary artery disease increase by a factor of 4.666 if an exercise test is abnormal. (Odds are defined in me Glossary of Terms.) If an exercise test is negative, the likelihood ratio is 0.30 divided by 0.85, or 0.35. When an exercise test is negative, the odds of having coronary artery disease are 0.35 times the pretest odds. Bayes' theorem uses data on test performance in me following way. In these formulas, TPR (true-positive rated is used in place of sensitivity, and FPR is used to denote false-positive rate. TNR denotes the true-negative rate, and FAR denotes the false-negative rate These terms are defined in me Glossary). The pretest probability of disease is represented by p(D). Probability of disease if test = IS positive Probability of disease if test = is negative p(D) x TPR . p(D) x TPR + [ 1 - p(D)] x FPR p(D) x FAR p(D) x E;NR + [1 - p(D3] x TNR The probability of a positive test result equals the probability of a true- positive result plus He probability of a false-positive result. Probability of positive = p(D)xTPR + ([l-p(D)]xFPR. test result B ayes ' theorem can be written in a simplified way that facilitates cal- culation. This form is called the odds-ratio form of Bayes' theorem. Posttest odds = pretest odds x likelihood ratio.

OCR for page 23
THE USE OF DIAGNOSTIC TESTS 29 Example: A clinician is planning to use an exercise test with a sensitivity (TPR) of 0.7 and a false-positive rate (FPR) of 0.15. Suppose the pretest probability of disease, p(D), is 0.30: Probability p(D) x TPR of disease if = - - test positive p(D) x TPR + t} - P(D)] x FPR .30 x .70 .21 .30 x .70 + .70 x .15 .21 + .105 = .667. The pretest odds are .30/.70 = 0.43 to I.0. The likelihood ratio for the test is .70/.15 = 4.667. Posttest odds = pretest odds x likelihood ratio = 0.43 x 4.667 = 2.0 to 1.0. Odds of 2.0 to I.0 are equivalent to a probability of 0.66. The importance of Bayes' theorem in interpreting a test is that it defines the relationship between pretest probability and posttest probability, which is shown in Figure 2.~. The relationship between these two entities has several implications. The interpretation of a test result depends on the pretest probability of disease. If a result is positive, the posttest probability increases as the pretest probability increases (Figure 2.la). If the result is negative, the posttest probability decreases as the pretest probability decreases (Figure 2.Ib). The consequence of this relationship is that one cannot properly interpret the meaning of a test result without taking into account what was known about the patient before doing the test. This statement is inescapa- bly true, because it is based on first pnnciples of probability theory. The effect of a test result depends on the pretest probability. The vertical distance between the 45-degree line in Figure 2. ~ and the cuIve is the difference between the pretest and Me posttest probability. When the clinician is already quite certain of the patient's true state, the probability of a disease is either very high or very low. When the pretest probability is very low, a negative test has little effect, and a positive test has a large effect. When the probability is very high, a

OCR for page 23
30 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY 1 .0- , LL 0.8 _ ~ 6 - 0.6 {D ~ o o ~ CL CL C,0 0.4 en ~ ~ CO In o 0.2 / / - / 0~0' 1 1 1 1 1 1 1 1 1 1 0.0 0.2 0.4 0.6 0.8 1.0 PRETEST PROBABILITY FIGURE 2.1 Relationship between pretest probability and pastiest probability of disease. Figure 2.1a The pastiest probability of disease corresponding to a positive test result was calculated with B. ayes ' theorem for all values of the pretest probability. The sensitivity and specificity of the hypothetical test were both assumed to be 0.90. negative test has a considerable effect, and a positive test has lithe effect. This example shows Hat a test result that confirms one's prior judgment has little effect on the probability of disease. Tests have large effects when the probability of disease is intermediate, which corresponds to clinical situations in which the physician is quite uncertain. Tests can also be useful when Weir result does not confirm the prior clinical impres- sionfor example, a negative result in a patient who is thought vein likely to have a disease. The pretest probability affects the probability that a positive or nega- live test result wid occur. The higher the pretest probability, the more likely one is to experience a positive test. Conversely, a negative test is less likely as the pretest probability increases.

OCR for page 23
THE USE OF DIAGNOSTIC TESTS 1.0 - - - J _ _ m ~ 0.6 m c, O ~ C: Z - ~ 0.4 co ~ In o 31 0.8 0.2 / / - / / / / l 0.0- ~ I- . , . . . 1 . 0.0 0.2 0.4 0.6 0.8 1.0 PRETEST PROBABILITY Figure 2.lb The pastiest probability of disease corresponding to a negative test result was calcu- lated win B ayes' theorem for ~1 values of Me pretest Probability. The sensitivity and specificity of the test were both assumed to be 0.90. The posttest probability depends on the sensitivity and the false-posi- tive rate of the diagnostic test. This relationship is one reason to be concerned about measuring test performance accurately. The Assumptions of Bayes' Theorem B ayes ' theorem is derived from first principles of probability theory. Therefore, when it is used correctly, the result is reliable. Errors in using Bayes' theorem can occur when people ignore several assumptions. One assumption of Bayes' theorem is mat sensitivity and specificity are constant, regardless of the pretest probability of disease. This assump- tion can be false. A test may be less sensitive in detecting a disease in an early stage, when the pretest probability is low, Han it would be in an

OCR for page 23
32 ASSESSMEN7 OF DIAGNOSTIC TECHNOLOGY advanced stage, when mere are many signs and symptoms and the pretest probability is high. This error may be avoided by dividing the study population into subgroups that differ in the extent of clinical evidence for disease (Weiner et al. 1979~. A second assumption is that the sensitivity and false-positive rate of a test are independent of the results of other tests. This conditional in~e- pen~ence assumption! is important when Bayes' theorem is used to calcu- late the probability of disease after a sequence of tests. The posttest probability after the first test in a sequence is used as the pretest probabil- ity for the second test. In an ideal study of two tests, bow tests in the sequence and a definitive diagnostic procedure have been performed on many patients. The sensitivity and specificity of the second test in the sequence are calculated twice: in patients with a positive result on the first test and in patients with a negative result on the first test. If the sensitivity and specificity of the second test are the same, they are said to be conditionally independent of the results of He first test, and the condi- tional independence assumption is valid. In practice, the conditional inde- pendence assumption is seldom tested, and the clinician should be cau- tious about using recommendations for sequences of tests. THE PRETEST PROBABILITY OF DISEASE Why is the pretest probability of disease an important concept in understanding He assessment of diagnostic technology? The pretest proba- bility is required to calculate the posttest probability of disease, and thus to interpret a diagnostic test; it is also the cornerstone of the decision whether to treat, to test, or to do nothing. A patient's pretest probability of disease encodes He individual's own clinical findings and is one of the ways in which a decision can be tailored to the patient. Knowing how to estimate the pretest ~ probability is an essential clinical skill and is de- scr~bed in the section that follows. When is testing particularly useful? Padents are particularly unlikely to benefit from testing when the pretest probability is very high or very low. If the pretest probability is very high, the physician is likely to treat the patient unless a negative result raises doubts about the diagnosis. The posttest probability of disease after a negative test may be so high that treatment is still indicated. Example: In a patient with typical angina pectoris, the posttest probability aher a negative exercise test is 0.76. Most physicians

OCR for page 23
THE USE OF DIAGNOSTIC TESTS 33 would begin medical treatment for coronary artery disease even if the probability of disease were considerably less than 0.76. For these physicians, the decision to treat would not be affected by the nonnal exercise test result. If the pretest probability is very low, as occurs in screening asymptomatic individuals, the clinician is likely to do nothing unless a positive test result raises concern. If, for example, the pretest probability is less than 0.001, the posttest probability may be less than 0.01. In this situation, a change in management is not indicated. Figure 2. ~ shows that the greatest benefit from testing is likely to occur when the pretest probability of disease is intermediate. This corresponds to a clinical situation in which Mere is uncertainty about the patient's true state. Patients are also likely to benefit from testing when the pretest probability is close to a treatment threshold probability. At this point, it requires only a small change in the probability of disease to cross the threshold and alter management. Physicians customarily use their intuition to estimate me probability of disease. The two principal influences on probability estimates are per- sonal experience and the published literature. Using personal experience to estimate probability. To estimate proba- bility, the physician should recall patients with characteristics similar to the patient in question, and then try to recall what proportion of these patients had disease. This cognitive task is forbiddingly difficult. In practice, the assignment of a probability to a clinical situation is largely guesswork. There are several cognitive principles for estimating probability (Tver- sky and Kahneman 1974~. These principles are caned heuristics. A clinician is using the representat~veness heuristic when he or she operates on the principle Cat "If the patient looks like a typical case, he probably has the disease." Thus, if a patient has all the findings of Cushing's disease, he is thought very likely to have the disease itself. The representativeness heuristic can be misleading, because it leads the physician into ignoring the overalD prevalence of a disease. It can also lead to error if the patient's findings are poor predictors of disease or if the physician overestimates probability when there are many redundant pre- dictors. Additionally, the clinician's internal representation of the disease may be incorrect because it is based on a small, atypical personal experi- ence. Clinicians are using the availability heuristic when they judge the

OCR for page 23
44 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY su rgery ' 0=.05 medical treatment survive pain unchanged |post-op death | 40 [pain unchanged! lPain resolved I - fIGURE 2~ A decision tree for deciding between surgery ar~d medical management of chronic pancreaiitis. The square represents a decision node. The circles represent chance nodes. The square represents a decision node. The rectangles represent terminal nodes for He venous outcome states, and the numbers enclosed within the rectangles are the products of Be length of life and the quality of life in the outcome state. Setting Up the Decision Tree Surgery: The first node on the surgery branch is a chance node (represented by a circle), which represents uncertainty about whether the patient would survive the operation. The patient may survive or may die, but the true outcome of the operation is unknown and can only be represented by a probability. On average, the mortality rate of the operation is 5 percent, which seemed a reasonable representa- tion for this padent, who was otherwise well. The next uncertainty was the outcome of treatment. Only about 60 percent of patients obtain relief of pain after surgery. The possible outcomes are represented by terminal nodes (shown as a rectangle). Each out- come is assigned a quantitative measure, such as the life expectancy in that outcome state. This patient's life expectancy was 20 years. Medical treatment: Because management associated with the medical option does not change, there are no chance nodes, and the patient's life expectancy is 20 years.

OCR for page 23
THE USE OF DIAGNOSTIC TESIS 45 Weighing the Outcomes for Quality of Life The patient's life expectancy was 20 years if he survived the operation, and it was thought to be the same regardless of whether he experienced chronic pain or was pan-free. The patient pointed out that 20 years of life with chronic pain was equivalent to 12 years of being pain-free. In other words, to be free of pain he was wining to give up eight years of life win chronic pain. This memos for weighing the length of life in a certain state of health by a factor that represents the quality of life in that state is called the "time trade-off' method. It is described in standard textbooks (Wein- stein et al. 1980, Sox et al. 19881. Calculating the Expected Value of the Treatment Options The average (or expected) outcome is calculated by taking the product of aU the probabilities along a path to a terminal node and multiplying it by the value assigned to Me terminal node. The management alternative with the highest expected value is usually the preferred choice. In this case, the expected length of life, measured in healthy years, was 16.8 years for surgery and 12 years for medical management. The surgeons were convinced by this analysis and scheduled the patient for surgery. Note that expected-value decisionmaking allows one to balance the risks and benefits of treatment. These factors are usually considered intuitively. By assigning a value to each outcome and weighing it by the chance that it win occur, expected-value decisionmaking alBows one to integrate risks and benefits. THE CHOICE AMONG DOING NOTHING, TESTING, OR STARTING TREATMENT The art of medicine is making good decisions with inadequate data. Physicians often must start treatment when stilU uncertain about whether the patient has the disease for which He treatment is intended. If treat- ment is started, there is a risk of causing harm to a person who does not have the disease, as well as the prospect of benefiting the person who does. If treatment is withheld, a person who is diseased win be denied a chance at a rapid, effective cure. This situation is often unavoidable, and the physician has three choices.

OCR for page 23
46 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY Do nothing: the chance of disease is low, treatment is either hammed or ineffective (or both), and a false-positive result might occur and lead to handful treatment for someone who does not have disease. Get more irg~ormatzon: do a test or observe the patient's course in the hope that Me correct choice win become apparent. Start treatment now: the chance of disease is relatively high, treat- ment is safe and effective, and a false-negative result might lead to withholding useful treatment from someone with disease. The method for solving this problem analytically is caned me thresh- old mode! of medical decisionmaking (Pauker and Kassirer 1975' 1980; Doubilet 19X3~. The threshold mode] is an example of expected-value decisionmaking that is applied to a particular type of decision. The key idea is We treatment threshold probability, which is the probability of disease at which one is indifferent between treating and not treating. The basic principle of the threshold mode] is the following dictum: Do a test only if the probability of disease could change enough to cross the treatment threshold probability. Three steps are required to translate this idea into action. Step : Estimate the pretest probability of disease. Step 2: Set the treatment threshold probability. This step is difficult because it requires the clinician to express the balance of the risks and benefits of treatment in a single number. One can use clerical intuition to set the treatment threshold probability. This task is made easier by the following relationship: C Treatment threshold = C + B where C is the cost of treating nondiseased patients, and B is the benefit of treating diseased patients (Pauker and Kassirer 1975~. The cost and the benefit must be expressed in the same units, which can be dollars, life expectancy, or a measure of the patient's attitudes toward treatment and the disease. Note that when the costs of treating nondiseased patients equal the benefits of treating diseased patients, the treatment threshold is 0.5. Thus, for many treatments, the treatment threshold probability will be less than

OCR for page 23
THE USE OF DIAGNOSTIC TESTS 47 0.50. For a safe, beneficial treatment, such as antibiotics for commuruty- acquired pneumonia, the treatment threshold probability may be less than 0.10. If there is good reason to suspect disease, the pretest probability win be above the treatment threshold. In deciding whether to perform a test, Me clinician must ask whether the posttest probability after a negative test result would be below the treatment threshold probability. This question is answered by taking Step 3, descnbed below. One can also use analytic methods to set the treatment threshold proba- biiity (Sox et al. 1988~. Consider me decision tree in Figure 2.6, which shows a hypothetical problem in which treatment must be chosen despite uncertainty about whether the patient has the disease for which the treat- ment is intended. To use We decision tree to estimate the treatment threshold, recall that this threshold is the probability of disease at which one is indifferent between treating and not treating. First, one assigns values to each of the probabilities and outcome states except for the probability of disease. Second, one calculates the expected value of the two options, leaving the probability of disease as an unmown. Third, one sets the expression for the expected value of the treatment option equal to that of the nontreat- ment option. Fourth, one solves for the probability of disease. To use a decision tree, one must assign a probability to each chance node and a numerical value to each outcome state. The latter value could be life expectancy. Altematively, as shown in Figure 2.6, one could assign each outcome state a Utility which is a quantative measure of relative preference. A utility of I.0 is assigned to the best outcome, and a utility of 0.0 to the worst. The utility of each intermediate outcome state is then assessed on this scale of 0.0 to I.0. When utility is used as the measure of outcome, the altemative with the highest expected utility should be the preferred alternative. Step 3: Use Bayes' theorem to calculate the posttest probability of disease. If the pretest probability is above the treatment threshold, one must calculate the probability of disease if the test is negative. If the pretest probability is below the treatment threshold, one must calculate the probability of disease if the test is positive. If the pretest probability is far enough above or below the treatment threshold, a test result will not affect management because the posttest probability will be on the same side of the treatment threshold as the pretest probability. There is a pretest probability for which the posttest probability is exactly the point at which one is indifferent between not

OCR for page 23
48 ASSESSMENT OF DIAGNOSTIC TECHNOLOGY operative death p=03 U O no tumor SURGERY I, NO SURGERY 1 L tumor survive surgery U=.98 operative death p_.O3 U O I P[tumor] ~ Tsurvive ~ r tumor U=.25 , |p[tumorj| U=.98 p=.48 no cure U 23 |no tumor U 1 0 FIGURE 2.6 A decision Bee for choosing between treatment and no treatment when the clinician does not know whether He patient has the disease for which treatment is indi- cated.

OCR for page 23
THE USE OF DIAGNOSTIC TESTS if} 49 ............................................................................................................................ . ~ ........................................................................... ... . . ... at re at ,,, ., ,., ., , ~ ......... ..... DON'T TEST [ no treat| ~ '~ ~ DON'T TEST | | no treat| | ;.;.;;; ; ; ;;;; ; ;;; ;; ;;;; ;; ;;; ;;;;; ;;; ;;; ; ;;; ; ; ............................................................................................. ................................. ........................................................... ............................................................ ~ ~ .,, ,., ,,,.,, treat ......................................................................................................................................................................................... TOSS-UP Llnotreatl ............................................................................................. treat ...................................................... ........................................................................... ................................................................ ........................................................................... - 3~ . ............................... ................................................................... ........................................................................... TCOT ~ ~ ~ ~ . _-, no treat area ~ ................................................................... ..... -.- - - - ....................................................................... ........................... FIGURE 2.7 Illustration of how to set die no treat-test threshold. As pO, the pretest probability, is gradually increased, the pastiest probability is first below the treatment threshold, then equal to it, and finally above it. At the point where pO equals the treatment threshold, one should be indifferent between not treating and testing. This probability is the no treat-test threshold. treating and testing (the treatment threshold probability). Below this pretest probability, caned the no treat-test threshold (Pauker and Kassirer 1980), a positive test result could not increase the probability of disease enough to cross the treatment threshold, and both testing and treatment should be withheld. Above this threshold, the posttest probability will exceed the treatment threshold, and testing is indicated. These concepts are illustrated in Figure 2.7.

OCR for page 23
so ASSESSMENT OF DIAGNOSTIC TECHNOLOGY |do nothing | ~ , - ; l ~ l l , |treatment | threshold pretest p[D] ~ ~ ' t.~.~.~.~ ~ ~ ~ ~ ~ ~ ~ ~ ~ "~".'."'"""""''""""~"~"""""'"""''t""'"''"""""~'"."'""''"'."'~'e'e''" T E ~ T .... .. , :., . . ; .... .. ...... ........ ......... ~ _ .50 1.0 probability of disease FIGURE 2~8 Using the treatment threshold probability to help decide whether to do a test. One can use the same approach to calculate the point at which one should be indifferent between testing and treating (the test-treatment threshold). Both of these thresholds are a function of the true-positive rate of the test, the false-positive rate of the test, the treatment threshold, and a measure of what experiencing the test means to the patient (the cost of the test). Figure 2.8 shows the three zones of the probability scale. Using Figure 2.8, one needs only to estimate the pretest probability to know whether testing is the preferred action or whether one should treat or do nothing. SUMMARY The purpose of this chapter is to provide a working knowledge of how probability theory and expected-value decisionmaking are used to help make decisions about diagnostic testing. Past studies of diagnostic tests have measured only test performance. A complete evaluation should provide information about the treatment threshold and how to estimate the pretest probability of disease. With this information, the clinician can decide when a test will alter management and can use the results to choose the action that will most benefit the patient.

OCR for page 23
THE USE OF DMGNOSTIC ~:STS GLOSSARY OF TERMS 51 Bayes' theorem: an algebraic expression for calculating the posttest probability of disease if the pretest probability of disease [p(D)] and the sensitivity and specificity of a test are known. Clinically relevant population: the patients on whom a test is normally used. Cost-effectiveness analysis: comparison of clinical policies in teens of their cost for a unit of outcome. Marginal cost-~ectiveness: the increase in cost of a policy for a unit increase in outcome. False-negative rate: the likelihood of a negative test result In a diseased patient (abbreviated F~. False-negative result: a negative result in a patient with a disease. False-positive rate: the likelihood of a positive test result in a patient without a disease (abbreviated FPR). False-positive result: a positive result in a person who does not have the disease. Gold-standard test: the test or procedure that is used to define the true state of the patient. Index test: He test for which performance is being measured. Likelihood! ratio: a measure of discrimination by a test result. A test result with a likelihood ratio >~.0 raises the probability of disease and is often referred to as a "positive" test result. A test result with a likelihood ratio <~.0 lowers the probability of disease and is often called a "nega- tive" test result. Likelihood ratio = probability of result in diseased persons probability of result in nondiseased persons .

OCR for page 23
52 ASSESSMENT OF DIAGNO=IC TECHNOLOGY Negative test result: a test result mat occurs more frequency in patients who do not have a disease than in patients who do have the disease. Odds: the probability. Odds = probability of event - probability of event . Positive test result: a test result that occurs more frequently in patients with a disease than in patients who do not have the disease. Posttest probability: the probability of disease after the results of a test have been reamed (synonyms: posterior probability, posttest risk). Predictive value negative: probability of the absence of the disease if a test is negative. Predictive value positive: probability of a disease if a test is positive. Pretest probability: the probability of disease before doing a test (syno- nyms: prior probability, pretest risk). Probability: an expression of opinion, on a scale of 0.0 to I.0, about the likelihood that an event will occur. Sensitivity: the likelihood of a positive test result in a diseased person (synonym: true-positive rate, abbreviated TPR). Sensitivity = number of diseased patients with positive test number of diseased patients . Specificity: the likelihood of a negative test result in a patient without disease (synonym: true-negative rate; abbreviated TAR). Specificity = number of nondiseased patients win negative test number of nondiseased patients -

OCR for page 23
THE USE OF DIAGNOSTIC TESTS 53 Study population: the patients for whom test performance is measured (usually a subject of the clinically relevant population). Treatment threshold probability: the probability of disease at which the clinician is indifferent between withholding treatment and giving treat- ment. Below the threshold probability, treatment is withheld; above the threshold, treatment is given. True-negative result: a negative test result in a person with a disease. True-positive result: a positive test result in a person with a disease. REFERENCES Doubilet, P. A mathematical approach to interpretation and selection of diagnostic tests. Medical Decision Making 3:177-195, 1983. - Goldman, L., Caldera, D.L., Nussbaum, S., et al. Multifactorial index of cardiac risk in non-cardiac surgical procedures. New England Jour- nal of Medicine 297:845-850, 1977. Greener, P.F., Mayewski, Ret., Mushlin, A.I., and Greenland, P. Selection and interpretation of diagnostic tests and procedures: Principles and applications. Annals of Internal Medicine 94(part 2~:553-560, 1981. ouye, S.K., and Sox, H.C. Standard and computed tomography in the evaluation of neoplasms of the chest. Annals of Internal Medicine 105:906-924, 1986. Metz, C.E. Basic principles of ROC analysis. Seminars in Nuclear Medicine 8:283-298, 1978. Pauker, S.G., and Kassirer, I.P. The threshold approach to clinical deci- sion making. New England Journal of Medicine 302:1109-1117, 1980. Pauker, S.G., and Kassirer, J.P. Therapeutic decision making: A cost- benefit analysis. New England Joumal of Medicine 293:229-234, 1975. Philbrick, I.T., Horwitz, At., Feinstein, A.R., Langou, R.A., and Chan- dler, I.P. The limited spectrum of patients studied in exercise test research: Analyzing the tip of the iceberg. Journal of the American Medical Association 248:2467-2470, 1982. Philbrick, I.T., Horwitz, R.~., and Feinstein, A.R. Methodology problems of exercise testing for coronary artery disease: Groups, analysis, and bias. American Joumal of Cardiology 46:807-812, 1980.

OCR for page 23
54 ASSESSMENT OF DIAGNOSTIC TEClINOLOGY Ransohoff, D.F., and Feinstein, A.R. Problems of spectrum and bias in evaluating the efficacy of diagnostic tests. New England Joumal of Medicine 299:926-930, 1978. Sox, H.C., Blatt, M.A., Higgins, M.C., and Marton, K.~. Medical Deci- sion Making. Boston, Buttenvor~, 1988. Sox, H.C. Probability theory in the use of diagnostic tests: An inDoduc- tion to critical study of He literature. Annals of Intema] Medicine 104:60-66, 1986. Stason, W.B., and Fineberg, H.V. Implications of alternative strategies to diagnose coronary artery disease. Circulation 66(Suppl. III):80-86, 1982. Tversky, A., and Kahneman D. Judgment under uncertainty: Heuristics and biases. Science 185:1124-1131, 1974. Wasson, I.H., Sox, H.C., Neff, R.K., and Goldman, Lo. Clinical prediction rules: Applications and methodologic standards. New En~lan`1 Imlr nal of Medicine 313:793-799. 1985. ~ ~ _ _ _ Weiner, D.A., Ryan, T.~., McCabe, C.H., et al. Exercise stress testing: Correlation among history of angina, ST-segment response and preva- lence of coronary-artery disease in the Coronary AItery Surgery Study (CASS). New England Joumal of Medicine 301:230-235, 1979. Weinstein, M.C., Fineberg, H.V., Elstein, A.S., Frazier, H.S., Neuhauser, D., Neutra, R.R., and McNeil, B.~. Clinical Decision Analysis. Phila- delphia, W.B. Saunders, 1980. Weinstein, M.C., and Stason, W.B. Foundations of cost-effectiveness analysis for health and medical practices. New England Joumal of Medicine 296:716-721, 1977.