An Assessment of the NIH Women's Health Initiative (1993)

Has research into women's health come into its own? The growing recognition of its importance by the Executive Branch of the government, the first female director of the National Institutes of Health (NIH), the consumer lobbies for treatment and prevention of breast cancer, the Congressional Caucus on Women's Issues, and hard-working researchers has culminated in the Women's Health Initiative. This 14-year, three-component project, budgeted at $625 million, is the largest single research study to be funded by NIH. As planned, it would involve 160,000 postmenopausal women at 45 clinical centers across the country and many other women residing in the communities that would participate in the Community Prevention Study.

The NIH Women's Health Initiative (WHI) is designed to learn about strategies to prevent breast cancer, cardiovascular disease, and osteoporotic fractures —leading causes of death and decreased quality of life for older women. Were the $625 million public investment guaranteed to find useful prevention or treatment strategies, it is hard to imagine many objections to the expenditure. Research is always a gamble, however. Simply put, if one knew the answer, one would not need to do the research. When NIH undertakes a research project that is as costly and complex as the WHI, it is necessary to maximize the likelihood that the research will yield definitive answers to important health questions. Clear goals, wise reasoning and planning, and funding adequate to the task are all essential to increasing the chances of success.

Scientists at NIH, the Vanguard Clinical Centers of the WHI, and advisory committees have contributed wisdom, time, and experience to building the WHI. Nagging doubt exists, however, that political concerns influenced the design, timetable, and budget of the study. The task of this committee is to respond to the concerns of the Committee on Appropriations of the U.S. House of Representatives over the study 's design, its concomitant cost, and the probability that the WHI will produce results that will be meaningful to the health of women in the United States. The committee's goal was to evaluate whether the investment in the WHI can be expected to produce useful enough results to justify the cost.

The WHI is an NIH-developed and funded study focusing on the health of postmenopausal women in the United States. Its three components are the Clinical Trial, the Observational Study, and the Community Prevention Study. NIH plans to administer the WHI, budgeted at $625 million over 14 years of activities, as two separate parts. A contract mechanism is in place for the Clinical Trial and Observational Study: a Clinical Coordinating Center and 16 Vanguard Clinical Centers began recruitment on September 1, 1993; in July 1993 NIH issued a Request for Proposals for an additional 29 Clinical Centers. The Community Prevention Study, which the NIH anticipates funding through Requests for Applications under a cooperative agreement, will be administered separately.

The Clinical Trial (CT), designed to include approximately 63,000 women, has three major branches, each based on a hypothesized intervention to prevent morbidity and mortality. Appendix A describes exclusion criteria. The diet modification branch, the primary endpoints of which are breast and colorectal cancer, includes randomization either to a control group or to a group receiving instruction, guidance, and support of a low fat dietary pattern. The hormone replacement therapy branch will focus primarily on potential protection against cardiovascular disease as well as potential adverse effects on breast and endometrial cancer risks; participants will be randomized to estrogen alone, estrogen plus progestin, or placebo. In the calcium and vitamin D branch, the primary endpoint is osteoporotic hip fracture, and women will be randomized to supplementation or placebo. The CT is designed as a partial factorial study (Figure 2-2) in order to reduce participant recruitment and follow-up costs.

The Observational Study (OS) will use, and go beyond, questionnaire and examination data gathered in the recruitment and enrollment of women into the CT. Approximately 100,000 women, ineligible or unwilling to participate in the CT, are expected to participate in the OS. Information from baseline examinations and interviews, annual questionnaire follow-up, three-year clinic visit follow-up, and selected clinic follow-ups at three-year intervals thereafter, are expected to yield valuable data for generating and testing hypotheses and examining associations.

Details of the Community Prevention Study (CPS) have not yet been established by NIH. Its overall goal would be to develop and test various strategies to influence women to change health behaviors and practices in order to decrease the risk of morbidity and mortality.

Dr. Bernadine Healy, as NIH director-designate, announced at her 1991 Senate confirmation hearing that NIH would launch a massive research initiative in women's health—afield that had been inadequately funded and championed. In that public statement,

Dr. Healy estimated a package that would cost approximately half a billion dollars. Soon afterwards, the estimate was adjusted to about $600 million.

The U.S. House of Representatives' Appropriations Committee—one of the purposes of which includes attention to the uses of the public's money—was concerned that the rapid escalation in cost estimates might be a signal of poor planning. In its July 23, 1992, report (Appendix B), the Appropriations Committee directed NIH to contract with the Institute of Medicine (IOM) to complete a review of the design and estimated costs of the WHI by February 1, 1993, in time to make adjustments, if necessary, before major study activities began. The House Appropriations Committee report (102-809:90) stated:

the study design and cost estimates should be thoroughly reviewed by an external group with expertise in this area. . . . This study should focus on the issue of cost, as well as the issue of whether the study will produce sufficiently reliable results to justify such a massive investment.

NIH first formally approached IOM in February 1993 to discuss the Appropriations Committee-requested project. NIH signed a contract allowing IOM to hire staff and begin work on April 23, 1993, well after the due date of the study requested by the Congress. Whether the delay was due to a slow-moving bureaucracy or to other causes is unknown to the IOM committee. As a result, however, the committee was faced with the task of critiquing a study in progress. In the interim, NIH had requested, received, and evaluated proposals, and entered into contracts with 17 research institutions to serve as the Clinical Coordinating Center and Vanguard Clinical Centers. Those funded centers have drafted an operating manual, hired staff, publicized the WHI in their communities, and begun to recruit participants.

IOM formed a committee with a neutral core, supplemented by additional members that included some members perceived to be in favor of and others opposed to basic elements in the WHI design. The chair was chosen to be both neutral and out of the arena in which WHI funds might be sought. Committee members represent expertise in the areas of epidemiology, clinical trial design and implementation, community trials, biostatistics, health education, ethics, nutrition, dietary assessment and dietary modification, hormone replacement therapy, cardiovascular disease, breast cancer, osteoporosis, and geriatrics.

The Statement of Task (Appendix C), approved by the IOM Governing Board and reflected in the contract with NIH, requested an examination of four issues:

1. Rationale for topics. What was the rationale behind the choice of these particular studies? Are appropriate topics considered? Should other topics be included?

2. Methodology. Are the methodologies and the study designs appropriate to address the stated research objectives? Is the size of the study population appropriate?

3. Costs. Are the costs accurately estimated and sufficient to reach the desired objectives?

4. Justification. Will the study produce sufficiently reliable results to justify the costs?

Questions 3 and 4 paraphrase the language in the House Appropriations report: “This [IOM] study should focus on the issue of cost, as well as the issue of whether the study will produce sufficiently reliable results to justify such a massive investment.”

In order to answer question 4, however, it is necessary to first answer question 3. Whatever the total cost, inadequate funding based on inaccurate estimates of staff, equipment, space, and time would severely threaten the completion of the project. At the opposite extreme, overfunding would not be an appropriate use of scarce public funds.

Cost estimates could be exquisitely accurate, yet not be justified by the results if the study design were flawed to the extent that it did not provide a structure by means of which the study questions could be answered. Hence, the IOM committee considered it within its charge to address question 2—to review the study design and statistical methods proposed by NIH.

Finally, of what use is a well-designed, well-funded study, if the questions it asks are not well conceived? In order to answer the bottom line question 4, the IOM committee needed also to discuss the rationale of the WHI and its component parts—that is, question 1.

It was not the committee's task to design a women's health study as if the WHI did not exist, nor was its task to decide whether other designs were preferable. In fact, the committee expressed concern that a precedent not be set whereby Congress exercised oversight with respect to research designs. The committee was charged to begin with the existing WHI design, consider threats to its successful completion—whetherdesign, financial, or ethical—and to consider whether it would yield reliable results. The committee chose in this context to interpret “reliable” to denote “meaningful” rather than, in the statistical sense, “repeatable.” To be meaningful in terms of public health significance, findings must meet some standard of internal validity, precision, and external validity.

Politics—the expression of community standards and desires—has been present throughout the development of the WHI and its precursor designs. Questions of feasibility and worth require judgments that are informed by both scientific and community standards. The IOM review has focused on the science base in dealing with the selection of committee members, topics to review, and the content of the report.

In writing the report, the committee agreed that it was not appropriate to include an extensive literature review of the interventions and diseases on which the WHI focuses. Although the committee reviewed the relevant literature and applied it when assessing the Initiative, this report is not intended to be a resource document for such literature.

An 11-member committee, as described above, held three meetings in Washington, D.C., in June, July, and August 1993. To assess whether the costs estimated are sufficient to carry out the planned design, the committee requested numerous documents from NIH, some under the Freedom of Information Act mechanism. The committee reviewed some available documents ( Appendix D ), and held extensive discussions with the NIH scientists ( Appendix E ) at its June meeting and in subsequent telephone and written contacts. At its July meeting, the committee met with representatives of the NIH-funded WHI Clinical Coordinating Center and 15 of the 16 Vanguard Clinical Centers ( Appendix E ). These contacts allowed the committee to garner not only the facts but a feel for the organization and philosophy of the WHI.

The following chapters present the committee's findings, questions, and recommendations with regard to the WHI Clinical Trial ( Chapter 2 ), Observational Study ( Chapter 3 ), and Community Prevention Study ( Chapter 4 ). A final chapter presents the committee's response to the questions it was charged to answer, along with concluding remarks on the societal context of the WHI and of biomedical research in general.