Appendix C
Supporting Materials for Chapter 2

This appendix provides the supporting technical material for Chapter 2. This information includes the relationship between biological efficacy and effectiveness, approximations for the effects on sample size of incorrect assumptions about HIV incidence and the relative risk of an intervention, and the justification for the approximate formula for determining the relative size of those two factors.

RELATIONSHIP OF EFFECTIVENESS TO BIOLOGICAL EFFICACY AND ADHERENCE

The overall effectiveness of an intervention depends on its biological efficacy and the degree to which individuals adhere to the product’s intended use. Nonadherence can be due to several factors. For example, individuals may never initiate the intervention, or they may be required to discontinue use of a product at some point because of a serious side effect, pregnancy, or other reason.

This section develops a simple approximation to illustrate how imperfect adherence dilutes the effect of an intervention. In this simple case, only a proportion, say f, of the individuals assigned to the intervention actually initiate it and use it as intended, while the remaining subjects never initiate the intervention.

Suppose that in the absence of an intervention, the incidence rate of HIV infection is I0. Suppose also that the biological effect of the intervention, if it was initiated, reduced the incidence rate from I0 to I1. Biologi-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 236
Appendix C Supporting Materials for Chapter 2 T his appendix provides the supporting technical material for Chapter 2. This information includes the relationship between biological efficacy and effectiveness, approximations for the effects on sample size of incorrect assumptions about HIV incidence and the relative risk of an intervention, and the justification for the approximate formula for determining the relative size of those two factors. RELATIONSHIP OF EFFECTIVENESS TO BIOLOGICAL EFFICACY AND ADHERENCE The overall effectiveness of an intervention depends on its biologi- cal efficacy and the degree to which individuals adhere to the product’s intended use. Nonadherence can be due to several factors. For example, individuals may never initiate the intervention, or they may be required to discontinue use of a product at some point because of a serious side effect, pregnancy, or other reason. This section develops a simple approximation to illustrate how imper- fect adherence dilutes the effect of an intervention. In this simple case, only a proportion, say f, of the individuals assigned to the intervention actually initiate it and use it as intended, while the remaining subjects never initiate the intervention. Suppose that in the absence of an intervention, the incidence rate of HIV infection is I0. Suppose also that the biological effect of the interven- tion, if it was initiated, reduced the incidence rate from I0 to I1. Biologi- 

OCR for page 236
 APPENDIX C cal efficacy is defined as the proportionate reduction in the incidence rate associated with use of the intervention. That is, I1 Biological efficacy = 1 − = 1 – r, I0 where r = I1 / I0. The quantity r is sometimes referred to as the relative risk, denoted RR. Now consider the effectiveness of the intervention in a population of individuals where a proportion f use the intervention as intended, and as a result have incidence rate I1. The remaining (in proportion 1 – f) do not initiate the intervention, and as a result have incidence rate I0. We refer to f in this setting as the adherence rate. The incidence rate I in this population is a weighted average of the incidence rates I0 and I1: I = frI 0 + (1 − f ) I 0 . We define the effectiveness of the intervention to be the proportion- ate reduction in the incidence rate, when accounting for the possibility of imperfect adherence. That is, the effectiveness of the intervention is defined as I 1− R = 1− I0 = 1 − ( fr + 1 − f ) = (1 − r ) f . That is, (1) Effectiveness = Biological Efficacy × Adherence Rate. Thus the effectiveness of an intervention depends on both its biological efficacy and the adherence rate. IMPACT OF ERRORS IN ASSUMPTIONS ON REQUIRED SAMPLE SIZE This section discusses the impact on sample size requirements of errors in estimating two critical factors: the incidence rate of HIV infection in the control group, and the relative risk of the intervention.

OCR for page 236
 METHODOLOGICAL CHALLENGES IN HIV PREVENTION TRIALS Impact of the HIV Incidence Rate on Required Sample Size Consider a two-arm randomized intervention trial with equal numbers of subjects on each arm. Suppose that the number of events (infections), X1, in the control group has a Poisson distribution with expectation I T, where I is the HIV incidence rate and T is the total person-years of follow- up in the control group. Similarly, assume that the number of events in the intervention group has a Poisson distribution with mean (I R T), where R is the relative risk of the intervention. We assume that the trial is designed with equal sample sizes and follow- up in the two groups, so that person-years are the same in each group. We are interested in testing the null hypothesis of no effect of the intervention: that is, Ho:R = . Investigators typically calculate sample size proceed by assuming a plausible a priori value for R, and then size the trial to have adequate power to detect that relative risk. The value of R is usually of sufficient public health significance to warrant adoption of the intervention in the community if the trial demonstrates efficacy. As discussed by Cox and Hinkley (1974), it is appropriate to condition on the total number of events (D = X + X). Conditional on D, we have X~binomial(D, p), where p = 1/(1 + R). Thus, the problem of determining the sample size for the two-arm trial is essentially equivalent to finding the sample size in a one-sample binomial situation. Accordingly, the number of required events D depends only on the Type 1 error rate α, and on the power β to detect a specified relative risk R (see Breslow and Day, 1987, p. 282). We denote D, the required number of events, by the function g (R, α, β). Suppose we initially assume that the incidence in the control group is I1. Then, because the expected number of events is equal to the person-time per group (T) multiplied by I1 ( + R), the person-time of follow-up that is needed is g (α , β , R) . I1 (1 + R) However, suppose that the true HIV incidence rate in the control group is I2 and not I1. Then the person-time per group that would be required to have the power β to detect the relative risk R is actually g (α , β , R) . I 2 (1 + R) Taking the ratio of these last two equations, we see that the factor g (α,

OCR for page 236
 APPENDIX C β, R)/(1 + R) cancels. Thus the ratio of person-time that is actually needed to that which was originally planned is I/I. That is, the person-time that is actually required to achieve the desired power is (1/I2) times larger than originally planned. If the duration of follow-up is unchanged, then the required sample size is also (I1/I2) times larger than originally planned. We illustrate this result with an example. Suppose the incidence rate is initially assumed to be 5 percent, but that it is actually only 4 percent. Then the study would require 25 percent more person-years (1/I2 = 1.25) than originally planned to achieve the desired power. The increase in person- years can be achieved by increasing the sample size by 25 percent, increas- ing follow-up time by 25 percent, or through a combination of increases in follow-up duration and sample size. For example, a 25 percent increase in person-years can be achieved by a 10 percent increase in sample size together with a 14 percent increase in duration of follow-up for each participant (that follows because 1.10 × 1.14 = 1.25). As shown in Chapter 2, the consequence of not modifying the sample size or planned duration of a trial in this setting is a reduction in the power of the trial to detect an intervention effect. Impact of the Relative Risk on Required Sample Size In this section we develop some simple approximations for the impact on the sample size of changes in the assumed relative risk. The required total number of events (D) for a two-sided test at level α, with power β to detect a relative risk of R is to a first approximation (see Breslow and Day, 1987, p. 282), is (z ) (1 + R) 2 2 + z1− β (2) α /2 D≈ , (1 − R) 2 where zα is the α critical value of a standard normal. Suppose a study is designed with an assumed HIV incidence rate of I and power β to detect a relative risk R1. Then the number of person-years T1 (per group) needed to yield the expected number of events is obtained by dividing equation (2) by I1 (1+R). However, suppose in fact that the true incidence rate and relative risk are I and R, respectively. Then the ratio of the required person-time T2 (per group) based on the correct specifications (I2, R2) to the person-time T1 (per group) originally planned based on the incorrect assumptions (I1, R1) is T2  I1   (1 − R1 )   (1 + R2 )  2 ≈   . T1  I 2   (1 − R )2   (1 + R )  (3)    2 1

OCR for page 236
0 METHODOLOGICAL CHALLENGES IN HIV PREVENTION TRIALS This equation can be further simplified if we ignore the last factor, yielding the following approximation: T2  I1   (1 − R1 )  2 (4) ≈  . T1  I 2   (1 − R )2     2 Thus, the ratio of the person-years that are actually required to those originally planned approximately varies inversely with the product of the ratio of incidence rates and the ratio of the square of the effectiveness parameters. The approximations leading to equations (3) and (4) are most accurate when the relative risks are near 1. We illustrate these results with an example. Suppose the study was orig- inally designed to detect an effectiveness of 0.3 (corresponding to a relative risk of 0.70), but that in fact the effectiveness is actually 0.2 (corresponding to a relative risk of 0.80). How much larger would the study need to be to detect this more modest effect with the same power as originally planned? If all other factors remained unchanged, the person-years required would need to be at least about 2.25 times larger than originally planned (because (0.3/0.2)2 = 2.25). (This calculation is based on equation 4. If a more precise calculation is desired, equation 3 could be used instead, which gives a factor of 2.38.) If no change is made to the duration of follow-up, then the sample size would need to be inflated by the factor 2.25 to achieve the needed increase in person-years. Impact of Imperfect Adherence on Required Sample Size The results in the preceding section can be used to illustrate the impact of imperfect adherence on the size of trials. As shown by equation 1, non- adherence dilutes the effectiveness of an intervention. Suppose we design a trial to evaluate an intervention that is believed to have a biological efficacy (1 – r). Suppose we also believe that the adher- ence rate can be expected to be about f1. We design the trial to detect an effectiveness of f1 (1 – r) with a specified power. We then ask how sensi- tive the required size of the trial is to the assumed value of the adherence rate, all other things being equal (such as HIV incidence, study power, and biological efficacy). That is, suppose we assume an adherence rate of f instead of f1. It follows from equation 4 that the ratio of person-years, if we assume an adherence rate of f instead of f1, is 2 T2  f1  ≈ . T1  f2  

OCR for page 236
 APPENDIX C Thus, the ratio of required person-years varies inversely with the square of the ratio of the adherence rates. For example, suppose a trial is designed with the expectation that the adherence rate for an intervention (such as a microbicide) is f1 = 0.90. We ask how much larger the trial would have to be if the adherence rate is actually only f = 0.5. We find that the person-years required would need to be 3.24 times larger (because (0.9/0.5)2 = 3.24). IMPACT OF MULTIPLE FACTORS The required sample size for a trial can be sensitive to a small differ- ence in a single key design parameter, such as the HIV incidence rate or the relative risk, as the preceding sections show. If there are small differences in both the actual incidence rate and the actual relative risk compared with the assumed rates, then these differences compound, and can result in large differences in the required sample sizes. For example, suppose a trial is designed to detect an effectiveness of 0.30 based on an annual incidence rate of 0.05. If instead we assume an effective- ness of 0.25 based on an incidence rate of 0.04, we find that the trial would need to be approximately 1.8 times larger (because (0.05/0.04)(3/0.2)2 = 1.8). Such sensitivity analyses of the trial size to multiple design parameters are important in assessing the feasibility of a trial and the likelihood of success. JUSTIFICATION FOR THE APPROXIMATE FORMULA FOR DETERMINING THE RELATIVE SIZE OF EFFECTIVENESS AND EFFICACY TRIALS As noted in Chapter 2, because of the lack of a validated surrogate endpoint for HIV infection for non-vaccine HIV prevention trials, a tradi- tional efficacy trial and and an effectiveness trial differ in duration, the HIV incidence rate in the control group, and the relative risk of the intervention. For example, a trial that focuses on evaluating the biological efficacy of an intervention may follow a highly compliant population for a short period of time, whereas an effectiveness trial may follow a more heterogeneous population for a longer period of time. The degree to which the risk-taking behavior of subjects changes dur- ing either trial may determine the HIV incidence rate in the control group. And the underlying incidence rate in the more compliant population could well be different from that in the more general population—either lower or higher. The third factor, relative risk, may largely reflect adherence. Investiga- tors might expect a population used in an efficacy trial to be more adherent

OCR for page 236
 METHODOLOGICAL CHALLENGES IN HIV PREVENTION TRIALS than a population used in a longer effectiveness trial, though this may not always be the case, as Chapter 2 notes. Table C-1 gives the approximate ratios in sample size of efficacy and effectiveness trials for different HIV incidence rates (I1 and I2) in the control group, relative risks (RR1 and RR2), and duration of the effectiveness trial (D2). For all settings, the efficacy trial is of six months duration. In almost all settings, the sample size needed for the efficacy trial exceeds that of the effectiveness trial, in many cases by a factor of two or more. The efficacy trial is smaller only when its control-group incidence rate is substantially higher and the intervention effect is substantially stronger in the efficacy trial than in the effectiveness trial. Let I1 and I2 be the HIV incidence rates in the control group for the two designs, and let R1 and R2 denote the corresponding relative risk of intervention: control. The ratio of the required person-years in the second trial compared with the first trial is approximately 2 T2 I1  1 − R1  = . T1 I 2  1 − R2    Suppose the duration of follow-up of each participant in these trials is denoted d1 for the first design and d2 for the second design. Then the ratio of required sample sizes (n) for the two trial designs is approximately 2 n2  I1   1 − R1   d1  = . n1  I 2   1 − R2   d2      Thus, the relative sample sizes approximately depend multiplicatively on the ratio of incidence rates, the ratio of the square of the effectiveness, and the relative duration of follow-up. To illustrate, suppose investigators are considering two designs to eval- uate a particular microbicide. The adherence rates in the first and second designs are 0.90 and 0.50, respectively. Assume that the biological efficacy of the microbicide is the same in the two designs. If follow-up (per person) were twice as long in the second design as in the first, the first design would require a larger sample size if I1/I2 <0.62 (because (0.5/0.9)2 × 2 = 0.62). Thus, in this example, if the HIV incidence rate in the control arm of the first design is smaller than that in the second design—in particular, less than 62 percent that in the second design—then the first design would require a larger sample size.

OCR for page 236
 APPENDIX C TABLE C-1 Relative Sample Sizes of Efficacy Trials to Effectiveness Trials for Varying Control Group HIV Incidence Rates, Relative Risks (RR), and Participant Follow-Up (in years) Efficacy trial size Effectiveness trial Sample Pt FU Incidence RR Pt FU Incidence RR Ratio 0.5 2% 0.4 4 4% 0.6 7.11 0.5 3 5.33 0.5 2 3.56 0.5 0.3 4 0.6 5.22 0.5 3 3.92 0.5 2 3.56 0.5 3% 0.4 4 4% 0.6 4.74 0.5 3 3.56 0.5 2 2.37 0.5 0.3 4 0.6 3.48 0.5 3 2.61 0.5 2 1.74 0.5 4% 0.3 4 4% 0.6 2.61 0.5 3 1.96 0.5 2 1.30 0.5 0.2 4 0.6 2.00 0.5 3 1.50 0.5 2 1.00 0.5 4% 0.4 4 4% 0.6 3.56 0.5 3 2.67 0.5 2 1.78 0.5 0.3 4 0.6 2.61 0.5 3 1.96 0.5 2 1.31 0.5 4% 0.4 4 3% 0.6 2.67 0.5 3 2.00 0.5 2 1.33 0.5 0.3 4 0.6 1.96 0.5 3 1.47 0.5 2 0.98 0.5 4% 0.3 4 2% 0.6 1.31 0.5 3 0.98 0.5 2 0.65 0.5 0.2 4 0.6 1.00 0.5 3 0.75 0.5 2 0.50 NOTE: Pt FU = participant follow-up.

OCR for page 236
 METHODOLOGICAL CHALLENGES IN HIV PREVENTION TRIALS REFERENCES Breslow, N. E., and N. E. Day. 1987. Statistical methods in cancer research. International Agency for Research on Cancer workshop, May 25–27, 1983. IARC Scientific Publica- tions (82):1-406. Cox, D. R., and D. V. Hinkley. 1974. Theoretical statistics. Toronto: Chapman & Hall.