Inclusion of Women in Clinical Trials: Policies for Population Subgroups (1993)

The Board on Health Sciences Policy of the Institute of Medicine is concerned that policies designed to ensure the appropriate inclusion of women and minorities in clinical trials of treatment efficacy might be applied so noncritically that they could hamper rather than enhance the advancement of scientific information about members of various population subgroups. This paper* explores the consequences of guidelines that mandate subgroup analysis in clinical trials, particularly as they pertain to women. However, similar consequences also arise for other subgroups (e.g., minorities).

Recent political and scientific arguments contend that although inclusion of women and minorities in medical research is necessary for valid inference about their health and disease, many medical studies have excluded or underrepresented both women and minorities. Therefore, Congress has proposed that the National Institutes of Health (NIH) ensure that a valid analysis be done on

variables being tested in clinical research that affect women or members of minority groups differently than other subgroups in the research (see Table 1).

This new policy would extend the previously stated policies of several agencies and organizations by requiring subgroup analysis of variables being tested for women or members of minority groups.^2–4

Of particular concern is Part (c) of Section 492B, which implies that clinical trials must analyze whether men, women, and minority populations respond differently to treatment. Detection of significant differences among relevant population subgroups generally requires clinical trials that are prohibitively large, time-consuming, and expensive. Medical researchers, caught between limited resources and many pressing medical questions, must consider sacrificing either statistical power or accurate representation of the population of interest. Clinical science now faces the challenge of providing information specific to members of distinct population groups while economically collecting accurate information.

The entire issue of inclusion of women in clinical trials would be moot if men and women responded identically to therapy. Current knowledge about differential response is incomplete. However, exclusion of women from clinical research that establishes the safety, efficacy, and mode of administration of drugs or other interventions prevents physicians from having sufficient information to make informed judgments about treatment of women. As pointed out by the American Medical Association’s (AMA’s) Council on Ethical and Judicial Affairs, the very factors that lead to the exclusion or underrepresentation of women are evidence of the importance of including women in clinical studies.⁵ We should learn about: the effect of an intervention on the fetus and on the potential for giving birth to a healthy child; whether there is variation in the response of women at different stages of the menstrual cycle; whether pre-and postmenopausal women respond differently to therapy; whether oral contraceptives or estrogen replacement therapy affects the response to other therapies; and whether women and men respond differently to an intervention.

Some facts about biological differences between men and women are known.^6–8 Reasons for the disparities may include genetic, physiologic, “lifestyle,” cultural, and social differences—the mechanisms of which are to a great extent unknown. In addition, sex differences in response to drugs and other interventions are being reported more frequently.^9–12 Questions are also being raised about whether women respond differently to therapy during different phases of their menstrual cycle and pre- and postmenopause. A major

concern expressed by the AMA’s Council on Ethical and Judicial Affairs is that medical treatments of women are based on a male model despite the fact that women may react differently from men or that some diseases manifest themselves differently in women. The results of medical research are generalized to women without sufficient evidence of their applicability to women.

The literature is inconclusive as to whether women have been excluded or importantly underrepresented in clinical trials. Certainly, women of childbearing potential have been systematically excluded from most clinical research in early phases of drug development. The FDA recommends exclusion of women of childbearing potential from the early phases of study of a new drug except when the drug is intended to treat a life-threatening disease or a disease with severe morbidity. This category includes women using contraception and those not currently sexually active, but not those who have had a hysterectomy or tubal ligation.** Moreover, many of the largest clinical trials of cardiovascular disease have explicitly excluded women.^13–17 On the other hand, in trials of the efficacy of a new drug, women have often been included roughly in proportion to the relevant prevalence of disease in men and women.¹⁸

Women face several types of barriers to full participation in clinical studies. Some barriers are ethical, some legal, some scientific, and some are the consequences of efforts to conserve scarce resources. These barriers result from: (1) the responsibility to protect the reproductive systems of women of childbearing potential and fetuses; (2) the fear of legal liability if a woman (or, subsequently, a child) suspects damage to a fetus or gamete due to a study; (3) the ease of recruitment, the compliance, and the condition of being at high risk for the endpoints being examined in a study cohort; (4) the availability of identifiable, convenient cohorts (e.g., veterans, army recruits); and (5) known variations in hormonal status affecting laboratory test results and inferences about treatments.¹⁹

A clinical trial must balance conflicting desires for homogeneity and heterogeneity. Ideally, the study cohort is homogeneous enough to yield a high probability of learning whether a therapy is safe and effective while heterogeneous enough to provide assurance that the observed results are not limited to a narrowly defined subgroup. No rules provide reliable guidance for planning the composition of the study cohort for a single study or for structuring a series of studies to investigate a therapy in different populations. Rational choice of cohorts in a clinical trial depends on the disease under investigation and the questions being asked, as well as on the conviction that the internal validity of a study translates into an ability to generalize to a more diverse population.

Members of the study group in a clinical trial generally differ from the population to whom the inferences are to be made in several important ways. Unlike a randomly selected sample of the population, they are volunteers who have consented to participate in an experiment. Second, the inclusion and exclusion criteria of the study select a more narrowly defined set of people than the group that may be eligible for treatment. To minimize the number of “drop outs,” clinical trials often exclude people who plan to change residence during the follow-up period, people likely to die soon from a disease other than the one being investigated, and people the investigators believe will not adhere well to the requirements of the protocol. A crucial assumption in making inferences from the observed results of the study is that these and other characteristics that differentiate the study cohort from the target population are not likely to translate into important physiologic differences that lead to distinct responses to therapy. Most clinical studies cannot rely on classical statistical inference from a random sample to the population from which the sample is drawn. Instead, the formal basis of inference rests on the principle of random assignment (for randomized clinical trials) and on the belief that different subgroups of a population are likely to respond similarly to interventions.²⁰ Thus, asking whether there is a biologically plausible reason to expect different responses to an intervention among subgroups is a crucial step in the design of clinical studies if results are to be generalized to people outside the study cohort. When homogeneous response cannot be assumed for specific subgroups of the population, it is essential that enough members of the relevant groups be included so that a differential response can be detected and measured.

Exclusion of a given subgroup from a study precludes formal inference about that subgroup. Therefore, a commonly recommended strategy is to design studies in which the composition by subgroup of the study cohort mirrors the affected group in the population that would eventually receive the treatment. To make inferences from such a trial to women, one might ask a number of questions:

• “Main effect”—Is the therapy effective overall?

• “Subgroup effect”—Is the therapy effective for women?

• “Quantitative interaction”—Is the therapy effective in both men and women but the magnitude of the effect different?

• “Qualitative interaction”—Is the therapy effective in one sex but ineffective or harmful in the other?

From the point of view of treating the population at large, valid answers to the “main effect” and “qualitative interaction” questions are crucial, for they help in the formulation of general policy regarding treatment. From the points of view of understanding the biologic mechanism of the disease and its treatment, of selecting the dose or otherwise modifying the administration of the therapy, of designing new therapies, and of determining sex-specific costs and benefits, the “subgroup effect” and “quantitative interaction” questions are also important.

When a study is designed to detect the main effect of treatment, if no reasonable hypothesis of important differences between subgroups exists, investigators must be cautious in interpreting surprising findings of differences. Although clinical trials are rarely large enough to test reliably a treatment effect within a subgroup of the population, most clinical trials report their results not only for the entire study cohort, but also for subgroups “of interest.” For example, investigators typically report data from clinical studies separately for men and women even when the sample size is inadequate for that purpose. If the sample size is chosen to show an overall treatment effect, an equivalent response in men and women can masquerade as a differential response, and vice versa.²¹

Some post hoc analyses of men and women are difficult to interpret. For example, Fisher et al.²² observed that for postoperative treatment of rectal cancer, a combined chemotherapeutic regimen consisting of 5-fluorouracil, semustine, and vincristine was preferable to radiation in men, but there was a nonsignificant opposite result for women.

Unfortunately, extremely large samples may be needed to detect differences among subgroups, for example, subgroups of different hormonal status. A recent overview of 133 randomized clinical trials of treatment of early breast cancer included a total of 75,000 women.²³ This very large sample size demonstrated that different subgroups of women have different responses to therapy. Their high cost and complexity prohibit performing megatrials except in common diseases. Moreover, only in the case of very few diseases will there be a

sufficient number of people in the collection of relevant randomized clinical trials to allow pooling very large numbers of people across trials.

But, if the study groups are small, how should one interpret an observed difference in therapeutic response between men and women? Some recommended approaches for identifying likely true subgroup effects include empirical Bayes methodology,²⁴ incorporation of methods to correct for multiplicity, and consideration of structured hypotheses.²⁵ As in all clinical research, biologic plausibility should be examined.

This analysis reveals a series of conundrums. In the absence of plausible scientific hypotheses concerning the existence of qualitative interactions, we waste time and money if clinical trials are large enough to detect differences between men and women. The situation becomes even more difficult if we consider women and men in various minority populations. But, if the sample size is too small, we may miss true effects or overinterpret apparent effects. Excluding women entirely prevents us from gaining even hints about differential response. In fact, a hypothesis that men and women respond similarly logically implies that sex should not be a criterion for participating in a clinical trial.

Merely including some women in a study is not sufficient to learn how to treat women. Similarly, including women in clinical trials in a ratio consistent with the prevalence of disease in both sexes may not provide reliable information about treatment. Neither is oversampling of women necessarily sufficient unless the realized sample sizes for men and women are large. Failure to see an effect does not necessarily mean that no effect is present; it may instead reflect inadequate statistical power. Introducing men and women from the various minority groups intensifies the problem, since examining a multiplicity of subgroups may lead to an observed effect that is not truly present in the population.

We suggest some approaches to deal with these issues. First, to help researchers (and study sections reviewing clinical research) develop appropriate samples, a set of guidelines based on the best available scientific evidence is needed (as specified in Section 492B (d)(1) of the Law).¹ Second, even if the investigators in a clinical trial do not intend to analyze data separately for men and women, it would be useful to collect data on certain variables (e.g., hormonal status of women, weight, adiposity) to allow eventual analyses should suggestive trends be found or should hypotheses arise from other studies.

Third, it is important not to shy away from extrapolating from the experience in one sex when the conclusions have biological plausibility in the other. It is also important to analyze by sex if there is scientific reason to

hypothesize that the responses of men and women differ substantially. Only by formulating scientifically meaningful questions and then testing them in rigorous studies can we expect to learn how women and men respond to therapy and how best to prevent and treat diseases.

Finally, avenues other than the clinical trial are available to help formulate hypotheses about the differential response of men and women to therapy. The case history has historically provided insight into subgroups. An unusual cluster of events may spark suspicion that a therapy is harmful to a subgroup of people. For example, the surprising occurrence of vaginal cancer in a group of young women led to the implication of DES in the etiology of vaginal abnormalities in daughters of women who had taken the drug during pregnancy.²⁶

More formal approaches to observational studies include many epidemiologic methods, the “Phase IV” postmarketing study, and outcomes research.²⁷ The pharmaceutical industry is currently studying some new approaches designed to expedite and improve the results of drug development activities. Such approaches include pharmacokinetic screens and the use of surrogate endpoints for pharmacodynamic measures. Another approach is meta-analysis, a set of techniques for combining data from various studies.²⁸ Data from various demographic subgroups can be pooled across many studies to provide more information on treatment effect than is available from a single trial.

An armamentarium of methods to learn about subgroup responses to therapy is available and must be used. Research designs and analytic techniques must be appropriate to the specific questions being asked and data must be collected that will ultimately be useful for sorting out the relationships between sex and hormonal status and response to therapy. A global solution, such as the one proposed in Section 492B, cannot provide the answers to complex and varied questions about the effects of therapy on women.

In summary, neither adherence to quotas in the composition of a clinical trial’s study group nor the irrational exclusion of a subgroup of people can be supported scientifically. The determination of the number of women to be included in a trial should reflect reasonable hypotheses about the relationship of treatment efficacy to sex, not to global rules about the composition of study cohorts.