Annex D. Correlation Between TD50and MTD
In this annex, we derive an analytical expression for the correlation between the TD50 and the MTD. To this end, suppose that the probability P(d) of a tumor occurring in an animal exposed to dose D = MTD satisfies the Weibull model

in (2.5), where the background parameter α > 0 and the shape parameter k > 0 are known. This is a generalization of the one-stage model used by Bernstein et al. (1985) in which k = 1.
Suppose that x of the n animals exposed to dose D develop tumors. Since and k are assumed known, β may be estimated by

where p0 = P(0) describes the spontaneous response rate. This leads to an estimate

of the TD50.
The estimate of β is appropriate for r ≤ x ≤ n-1. The lower limit of x = r is the minimum value of x that would lead to a statistically significant result at a nominal significance level of 0 < γ < 1; the value of r is determined from the fact that in the absence of a treatment effect at dose D, x follows the binomial distribution Bin (50, P(D)). The upper limit of x = n-1 is included since β, and hence TD50, is undefined for x = n.
The constraint x ≤ n-1 implies that

whereas x ≤ r implies

We wish to find the correlation between Y = logeTD50 and X = log eD. (Although the correlation will be identical using logarithms to the base 10, the derivation of the correlation given here is simpler using natural logarithms.) Suppose now that W = TD50 follows a uniform distribution on the interval [a,b], reflecting the fact that given the value D of the MTD, the estimated value of the TD50 is unrelated to the MTD. Suppose further that X follows some distribution with mean µ and variance σ2. Although Bernstein et al. (1985) observed that the empirical distribution of X is approximately normal, the correlation between Y and X does not depend on the distribution of X other than through its variance σ2.
To calculate corr (Y, X), note that

and

where

and

Thus we have

with V(X) = σ2. Noting that

where µ = E(X), we have

This leads to the desired result:

It can be shown that h2 - h12 + 1 ≥ 0, so that 0 < ρ ≤ 1. It can also be shown that ρ ↓ [σ2/(σ2 + 1)]1/2 as k ↓ 0, and that r 1 as k → ∞. Thus [σ2/(σ2 + 1)]1/2 ≤ ρ ≤ 1. In the limiting case as n → ∞, (D. 13) reduces to

The values of the correlation coefficient ρ in (D.13) as a function of