Executive Summary

A growing recognition of the importance of research on women's health, sparked by concern for the treatment and prevention of breast cancer, has led the National Institutes of Health (NIH) to launch the Women's Health Initiative (WHI). This unprecedented study will involve approximately 160,000 women at 45 clinical centers and thousands of others in community studies over a period of 14 years. The WHI will investigate strategies for preventing three of the leading causes of death and decreased quality of life for older women—cardiovascular disease, breast cancer, and osteoporotic fractures. But while recruitment of study participants has already begun, serious questions remain about the design, timetable, and likely results of this $625 million public investment.

Biomedical research is always a gamble; no study can be guaranteed to find useful prevention or treatment strategies. Nor can one study answer all pertinent questions. But when NIH undertakes a research project as costly and complex as the WHI, one would like to increase the likelihood that the research will yield definitive answers to important health questions. Clear goals, wise planning, adequate funding, and sufficient flexibility to deal with the unknown—all will help to increase the chances of success.

As described in the report, the committee had a defined charge: to determine whether the WHI or parts of it could be justified scientifically. The committee's charge was not to design a women's health study as if the WHI did not exist, nor was its task to decide whether other designs were preferable.

The task of the committee was shaped by the fact that the committee was assessing a research project at the worst possible time. The project was already underway, but it was too early to have proof of adequate implementation or to have any findings. Public funds had already been committed to institutions and investigators, and public expectations had been raised. These considerations did have some influence on the recommendations made by the committee.

This report acknowledges the planning and preparation done by the scientists at NIH, the Vanguard Clinical Centers, and the WHI advisory committees. Following its review, the committee's overall recommendation is that the ongoing WHI study could proceed but it should be modified. In order to provide a basis for improvement, however, the report



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Executive Summary A growing recognition of the importance of research on women's health, sparked by concern for the treatment and prevention of breast cancer, has led the National Institutes of Health (NIH) to launch the Women's Health Initiative (WHI). This unprecedented study will involve approximately 160,000 women at 45 clinical centers and thousands of others in community studies over a period of 14 years. The WHI will investigate strategies for preventing three of the leading causes of death and decreased quality of life for older women—cardiovascular disease, breast cancer, and osteoporotic fractures. But while recruitment of study participants has already begun, serious questions remain about the design, timetable, and likely results of this $625 million public investment. Biomedical research is always a gamble; no study can be guaranteed to find useful prevention or treatment strategies. Nor can one study answer all pertinent questions. But when NIH undertakes a research project as costly and complex as the WHI, one would like to increase the likelihood that the research will yield definitive answers to important health questions. Clear goals, wise planning, adequate funding, and sufficient flexibility to deal with the unknown—all will help to increase the chances of success. As described in the report, the committee had a defined charge: to determine whether the WHI or parts of it could be justified scientifically. The committee's charge was not to design a women's health study as if the WHI did not exist, nor was its task to decide whether other designs were preferable. The task of the committee was shaped by the fact that the committee was assessing a research project at the worst possible time. The project was already underway, but it was too early to have proof of adequate implementation or to have any findings. Public funds had already been committed to institutions and investigators, and public expectations had been raised. These considerations did have some influence on the recommendations made by the committee. This report acknowledges the planning and preparation done by the scientists at NIH, the Vanguard Clinical Centers, and the WHI advisory committees. Following its review, the committee's overall recommendation is that the ongoing WHI study could proceed but it should be modified. In order to provide a basis for improvement, however, the report

OCR for page 1
An Assessment of the NIH Women's Health Initiative focuses on identifying potential problems and finding strategies for dealing with them, in order to increase the probability that the WHI will make a meaningful contribution to the health of women in the United States. THE WOMEN'S HEALTH INITIATIVE The WHI is the largest research study ever funded by NIH. Budgeted at $625 million over 14 years, it is designed to test strategies to prevent cardiovascular disease, breast cancer, and osteoporotic fractures—leading causes of death, disability, and decreased quality of life for older women. The WHI consists of three major components: the Clinical Trial, the Observational Study, and the Community Prevention Study. By far the costliest and most complex of these components is the Clinical Trial (CT), which will involve 63,000 postmenopausal women at 45 clinical centers across the United States. The CT has three branches that, as currently designed, will test several hypotheses: Dietary Modification (DM) Primary hypotheses: A low fat dietary pattern reduces the risk of breast cancer and colorectal cancer. Secondary hypothesis: A low fat dietary pattern reduces the risk of coronary heart disease. Hormone Replacement Therapy (HRT) Primary hypothesis: Estrogen replacement therapy (ERT) and progestin and estrogen replacement therapy (PERT) reduce the risk of cardiovascular disease. Secondary hypotheses: ERT and PERT reduce the risk of osteoporotic fractures. ERT and PERT increase the risk of breast cancer. Calcium and Vitamin D Supplementation (CaD) Primary hypothesis: Supplementation reduces the risk of hip fractures. Secondary hypotheses: Supplementation reduces the risk of other fractures. Supplementation reduces the risk of colorectal cancer. The Observational Study will interview, obtain blood specimen samples from, and follow another 100,000 women who were ineligible or unwilling to participate in the CT, generating a rich database on the CT hypotheses and related health questions. Committee calculations corroborated by estimates provided by the Clinical Coordinating Center, find the CT and the

OCR for page 1
An Assessment of the NIH Women's Health Initiative OS together are budgeted at $586 million. The Community Prevention Study will involve thousands of other women in the development of community-based strategies and infrastructure to promote healthy behaviors and practices to decrease the risk of morbidity and mortality. The CPS is under design and therefore does not have an announced budget. THE IOM STUDY Concern over the size and complexity of the WHI, as well as rapidly escalating cost estimates, led the U.S. House of Representatives ' Appropriations Committee to direct NIH to contract with the Institute of Medicine (IOM) for a review of the proposed Initiative. To respond to this request, the IOM constituted the Committee to Review the NIH Women's Health Initiative. Under the terms of this request, the IOM review of the WHI has focused on four principal questions: Topics considered. What was the rationale behind the choice of these particular studies? Are appropriate topics considered? Should other topics be included? Methodology. Are the methodologies and the study designs appropriate to address the stated research objectives? Is the size of the study population appropriate? Costs. Are the costs accurately estimated and sufficient to reach the desired objectives? Justification. Will the study produce sufficiently reliable results to justify the costs? Questions 3 and 4 paraphrase the language in the House Appropriations report: “This study should focus on the issue of cost, as well as the issue of whether the study will produce sufficiently reliable results to justify such a massive investment.” * In order to answer question 4, however, it is necessary to answer question 3 first. Whatever the total cost, inadequate funding based on inaccurate estimates of staff, equipment, space, and time would severely threaten the completion of the project. At the opposite end of a cost estimate error, overfunding would not be an appropriate use of scarce public funds. Cost estimates could be exquisitely accurate yet not be justified by the results if the structure of the study design were flawed to the extent that it could not answer the study questions. Hence, the IOM committee considered it within its charge to address question 2—to review the study design and statistical methods proposed by NIH. * U.S. House of Representatives, Departments of Labor, Health and Human Services, and Education, and Related Agencies Appropriations Bill, 1993, Report 102-809, p. 90.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Finally, of what use is a well-designed, well-funded study, if the questions it asks are ill-conceived? To answer the bottom-line question 4, the committee needed first to discuss the rationale of the WHI and its component parts, that is, question 1. It was not the committee's task to design a women's health study as if the WHI did not exist. In fact, the committee expressed concern that a precedent not be set whereby Congress exercised oversight with respect to research designs or mandated special review committees for all large trials after the normal review processes have taken place. The committee was charged to begin with the WHI design, to consider threats to its successful completion—whethermethodological, financial, or ethical—and to consider whether it would yield reliable results. After consultation with IOM and Congressional staff, the committee chose in this context to interpret “reliable” to denote “meaningful” rather than, in the statistical sense, “repeatable.” This was the intent of the Appropriations Committee in raising the question. FINDINGS AND SUGGESTIONS The committee feels that the WHI had inadequate peer review from within NIH or from outside scientists. Although various elements of the WHI were reviewed at one time or another (e.g., the dietary modification trial was reviewed multiple times in earlier proposals, none of which were allowed to proceed), the committee's impression is that the complicated interlocking combination of the clinical trial and the observational study was not reviewed as rigorously as the usual Institute-initiated project. It seems that this inter-Institute study fell outside the review process. The committee suggests that NIH reexamine and strengthen the mechanism through which it reviews future inter-Institute proposed projects. Clinical Trial The committee concentrated on two fundamental questions: Can the design answer the questions it addresses, if no operational difficulties occur? If the study design is appropriate, what threats are there to the successful completion of the study? The committee identified seven issues involving conceptual problems that are built into the design. Even if all study operations were to proceed without incident, these design issues threaten the validity of the findings. Where appropriate, the committee has also suggested strategies to overcome the difficulties.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Factorial Design NIH argued that conducting a partial factorial design would reduce the required number of women and attendant costs and allow assessment of interactions among intervention branches. The committee feels that the factorial design has major drawbacks. The overlap of 15.9 percent between the DM and HRT interventions is insufficient to provide adequate statistical power to assess interactions, and the difficulties of maintaining adherence to two or three interventions detracts from the attractiveness of a factorial design. In essence, the integrated design has become primarily a matter of economic efficiency; it is not essential to hypothesis testing. Sample Characteristics In determining sample size, the study design relies heavily on extremely uncertain assumptions regarding magnitude of effect and lag times. This concern is a factor in the recommendation described below regarding study duration. Participants will not be categorized by risk for breast cancer, colorectal cancer, or coronary heart disease. This allows a more generalizable study, but the lack of risk restrictions requires a much larger sample size. The factorial design does not allow specific branches to focus on the most efficient samples, such as women at high risk of CHD for an HRT trial or women at high risk of breast cancer for a DM trial, according to NIH assumptions. Proposed Analytic Techniques Committee concerns center on choice of endpoints for trial closeout and the planned use of methods to adjust for multiple comparisons when considering interim decisions by the Data and Safety Monitoring Board (DSMB). The committee suggested that unadjusted data be made available to the DSMB. The committee felt that the Bonferroni statistical adjustment, for which current analysis plans call, might be too conservative and therefore might deprive many participants of an appropriately timed conclusion to the study. The committee also suggested the use of a two-sided test of significance to maintain a scientifically-justified neutral stance regarding whether the interventions might yield beneficial or adverse effects.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Ethics The informed consent measures do not provide an adequate understanding of the likelihood or magnitude of major risks and benefits. The obligation to inform potential and current research participants would require much more information at the outset, as well as a commitment to provide evolving information over the course of the project. The committee believes that studywide material must inform potential participants of risks as well as benefits. The committee suggested that the counselors at the clinical centers be knowledgeable and have access to algorithms, guidelines, and printed material about known risks and benefits. These counselors would need supervision, training, and monitoring. In addition, new information from this as well as other pertinent trials (as judged by the WHI coordinators and the DSMB) must be shared with the participants to allow them to make their own decisions about ongoing risks and benefits of the interventions. The inclusion of several interventions with several endpoints in a single trial makes the stopping rules difficult to formulate. Therefore, the committee suggested that the DSMB should (a) use preexisting or external information to establish a prior probability that internal data could confirm (this might mean accepting an earlier “stopping” conclusion than would be justified by data arising solely from the CT); (b) perform pre-specified subset analyses on participants groups that are especially likely to evidence harm or benefit; (c) ask to examine uncorrected estimates of effect and do any analyses it feels are warranted; (4) review the monitoring of the consent process; and (5) evaluate pre-specified event rates for potential morbidity and mortality outcomes. Minority Analysis Plan As currently designed, the study will have insufficient power to compare individual minority groups to the majority population. The study will be able to observe differences, if they exist, but will not be able to test them with adequate power. The committee encourages NIH to make these limitations known to those who may be expecting definitive comparative findings among minority and majority groups. Specificity of Intervention and Effect The CT design does not distinguish which element of the low fat dietary pattern may be responsible for any observed outcome. Similarly, the design will not allow analyses to distinguish whether calcium or calcium plus vitamin D is responsible for any observed outcome. Because some endpoints can be affected by more than one of the study

OCR for page 1
An Assessment of the NIH Women's Health Initiative interventions, and because the factorial design is modified by participant decisions, the overlap and interactions will be difficult to analyze. Outcome Definition and Measurement Threats to accurate and unbiased endpoint detection include the obscure meaning of many mammography-detected tiny malignancies; the unstandardized method of detecting colorectal cancer; and the inadequate development of behavioral, psychological, and quality of life measures for use in the study. The committee encourages NIH to include measures of constructs such as pain, mobility, and psychological status. In addition to the conceptual problems described above, any study —nomatter how well designed—is subject to setbacks by operational problems. The WHI CT is particularly vulnerable to such problems because of its size, complexity, and duration. The committee has identified five operational issues that could jeopardize the success of the study: Recruitment, Retention, and Adherence The message of the study is not adequately developed and may be misleading. The committee suggests that NIH and the clinical centers develop an overall message for the study that pays particular attention to long-term recruitment strategies for older and minority participants, and does not emphasize the WHI as a breast cancer prevention trial. In addition, investigators should set higher standards for studywide materials than currently appear to exist, including introductory brochures, consent forms, and videotape information. This information should be available in conversational language. NIH has made overly optimistic assumptions about recruitment, retention, and adherence, especially in subgroups with which researchers have less clinical trial experience, such as older women, minority women, and the spectrum of socioeconomic status (SES), and in recruitment plans that cover many years. Nevertheless, the committee encourages NIH to seek diversity within the sample and suggests that attempts should be made to include the entire SES range in this study. The acceptability of the various branches of the CT to women is unclear at this stage, especially since the interventions are difficult and have potential side effects. To maintain adequate statistical power, the CT must have funds available to boost recruitment efforts if as the committee expects, recruitment rates are lower than anticipated.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Secular Trends If secular trends toward a decreasing fat content in the U.S. diet continue, and if there is appreciable nonadherence in the DM treatment group, the difference between the treatment and control diets is likely to be too small to show a treatment effect. Provision of Health Care Services to Participants The current protocol includes a referral to a regular source of care. This is not adequately responsible. The committee suggests that the clinical centers must develop adequate links with reliable community providers to ensure that adequate follow-up care is available. It may become essential for the project to pay for some kinds of follow-up for some poor or uninsured women. Research staff need to spend considerable time discussing side effects with participants, and dealing with associated apprehension, both in the clinic and on the telephone. To fail to do so is to risk unethical behavior and increased study dropout. The current budget may not include adequate staff time for these activities. Cost The committee believes that the total costs of the CT will be greater than the $625 million provided by NIH. NIH and Vanguard Clinical Center representatives have indicated that the additional funds necessary for successful completion of the trial will be covered by the institutions in which the Clinical Centers are based. This reliance on institutional support may be reasonable in the case of the Vanguard Clinical Centers, but the committee felt it is unlikely that an additional 29 institutions can be identified that have both the experience to carry out the tasks of high quality research and the ability to provide additional resources. Potential sources of budget shortfall include lagging participant recruitment, which could require increased staff resources; staff turnover, which could require training and travel resources and might delay recruitment, threaten adherence, and, therefore, affect study validity; and cross-over of participants between study intervention regimens and control status. The CT funding per-person per-year is less than half that for other recent NIH studies of women's health, including, specifically, those that use similar drug regimens and approaches. There does not seem to be a budget adjustment plan for unanticipated changes in either the scope of work or medical technology during the course of the trial.

OCR for page 1
An Assessment of the NIH Women's Health Initiative In addition to its concerns about initial funding levels, the committee was concerned about long-term funding and suggested that NIH clarify what the contract requires financially in terms of anticipated or unanticipated changes throughout the duration of the study. Observational Study Although NIH considers the Observational Study (OS) an important component of the WHI, the OS has not received the same amount of attention as has the CT. It is anticipated that the 45 clinical centers involved with the CT will enroll approximately 100,000 women, collecting baseline data and following them for an average of nine years. An observational study of this scope would require a very large investment if it were initiated independently, but since the OS was designed as a WHI component, the marginal cost of its addition is reasonable —as estimated at $105 per woman over the duration of the study by the Clinical Coordinating Center—in terms of the information it can acquire. Community Prevention Study NIH has not provided a clear vision for the Community Prevention Study (CPS). Because only an initial plan for the CPS has been presented thus far—with rationale, design and methods, and cost incompletely conceptualized—the committee was unsure of the scope of the CPS and the resources necessary to complete it. Based on information available about the anticipated costs of the CT and OS, the committee calculates that as little as $25 million could be available for the CPS component of the WHI. Given the tremendous need for strategies that would result in lifestyle change in women, particularly among lower socioeconomic status and minority women, the committee feels strongly that the amount will not suffice. The committee suggested that the following issues be highlighted in the development of the CPS protocol: project selection should ensure that components are suitable for incorporation into comprehensive community-based programs; projects should attempt to decrease unfavorable disparities between lower SES and racial/ethnic minority women and higher SES nonminority women and to promote the creation of culturally appropriate strategies at low cost; projects should include strong training components (that will permit personnel to train others in additional communities), possibly resulting in a set of regional training centers; projects should employ many process measures needed for cost-effectiveness analyses; and

OCR for page 1
An Assessment of the NIH Women's Health Initiative NIH should favor projects that are multifactor in both risk factors and targeted methods of intervention. RECOMMENDATIONS Clinical Trial The committee was charged to begin with the existing WHI design, consider threats to its successful completion—whether design, financial, or ethical—and to consider whether it would yield reliable results. The committee recommends that the dietary modification-breast cancer hypothesis be considered a subsidiary rather than a primary hypothesis, shifting the emphasis to the effect of dietary modification on coronary heart disease outcomes, making those the primary hypotheses. The committee recommends that the consent process be outlined more carefully, be conscientiously implemented and monitored across all centers, and be evaluated and updated as needed. The committee recommends that the CT be scheduled to end in mid-2002, rather than close out the interventions by April 2005, and that the findings of an Objective Prescheduled Reassessment (OPR) be available by April 2002 (see figure at the end of the Executive Summary). The OPR, managed through an internal or external review board, would consider whether continuation or modification of the CT could be justified. Recruitment for the CT began in September 1993, so the project would run unimpeded for more than eight years (unless the Data Safety and Monitoring Board moves to stop the trial sooner based on interim data). Data analysis would begin in October 2001 and conclude with a recommendation by April 2002. Between October 2001 and the decision to extend, modify, or terminate, the CT would continue in its active mode. Sufficient time would be provided for closeout or redesign and data analyses. This recommendation addresses the primary concerns of the committee in the following ways: Data from nearly six years mean follow-up time would be available for the OPR. According to NIH power calculations, this timeframe would allow hypotheses regarding stronger, expected associations (HRT and coronary heart disease; and HRT and combined fractures) to be tested and findings disseminated in a timely manner. If the intervention effect is strong, this timeframe also allows the hypotheses regarding the

OCR for page 1
An Assessment of the NIH Women's Health Initiative weaker, expected associations (DM and coronary heart disease; CaD and hip fractures; and HRT and hip fractures) to be tested. This timeframe does not allow for adequate follow-up for the DM and breast cancer hypothesis, the DM and colorectal cancer hypothesis, or the HRT and breast cancer hypothesis. However, the committee feels that, as currently designed, the CT does not have a high probability of yielding statistically significant results for the DM and breast cancer hypothesis or the HRT and breast cancer hypothesis, even after more prolonged follow-up. The committee would therefore prefer to see the other hypotheses analyzed in an appropriate timeframe. While the DM and colorectal cancer hypothesis is strong, it alone does not justify continuing the CT. This recommendation allows an assessment that would be informed by recruitment, retention, adherence, and incidence experience; if any of these estimates have not been or are not being met, the problem can be addressed. For example, if HRT is demonstrated to be favorable compared with control, the CT could reassign the control participants (with their permission) to ERT or PERT, thus increasing statistical power for that direct comparison, which as designed is not currently adequate. If recruitment or adherence experience is so poor that an adequate test of a hypothesis would not be possible in any reasonable time frame, the CT or a branch of it could terminate. If, on the other hand, recruitment or adherence problems are discretely identifiable, the study could be redesigned for the remaining duration to compensate for these problems. Any clinically beneficial findings of the CT can be made available to participants. Clinical knowledge resulting from other studies can also be applied to participants in both intervention and control arms of the CT. Therefore, WHI investigators would not be pressured to deny benefits to women in the CT to keep intact its overlapping studies. Observational Study The committee recommends that NIH treat the OS as a precious investment even though its marginal cost to the WHI is low. The progress of the OS should be monitored carefully to ensure that the quality of the data is as high as possible, and that losses to follow-up are minimized. The committee recommends that NIH make OS data available to qualified investigators outside the WHI network early enough to maximize the use of the data. Outside investigators should also be able to add ancillary studies to the OS. Such data availability increases the scientific and public health yield of a public investment and would increase the justification of the expenditure. Clear policies about the timing and conditions under which the data and cohort will be made available to outside investigators need to be established from the outset.

OCR for page 1
An Assessment of the NIH Women's Health Initiative The committee encourages both NIH and the WHI OS Subcommittee to consider the implications for the OS of the recommendation that the CT undergo an OPR after eight years. It is likely that contingency plans for the OS will need to be devised. Community Prevention Study The development of the Community Prevention Study is of equal importance to the CT and the OS. It urgently needs a more definitive plan and the committee recommends that NIH develop one quickly. The level of resources allocated for the CPS should be an affirmative decision, one that is based on the appropriate funding necessary to accomplish the task, not using only what funds might remain after the completion of the other two components of the WHI. The committee also feels that certain aspects of the CPS are critical to highlight. While these are not as essential to WHI success as are the primary recommendations, the committee urges the NIH to seriously consider the following suggestions: Given the importance of women's health and the vast range of circumstances influencing it, numerous projects (probably between 40 and 50) should be funded over approximately eight years. These projects should adequately encompass needs related to diversity of health topic; intended recipients of interventions; geographic regions of varied cultures; and approach or strategy. Within the eight-year project, three years of funding is recommended in order to focus on strengthening infrastructure development and dissemination techniques. Approximately $50 to $100 million should be targeted for the CPS. Internal resources should be developed in conjunction with the coordinating and disseminating functions related to the CPS. In general, NIH should strengthen its public health and disease prevention component; the coordination and dissemination activities of the CPS can aid NIH in reaching that goal. NIH should ensure that a mechanism (such as comparable data collection instruments) exists to link the projects and facilitate useful exchanges among investigators. This would also serve to transfer knowledge and technology to relevant communities during a later dissemination phase. The cooperative agreement is considered as a possible mechanism for this purpose.

OCR for page 1
An Assessment of the NIH Women's Health Initiative RESPONSES TO THE STATEMENT OF TASK After reviewing the information provided by NIH and considering additional perspectives provided by the Vanguard Clinical Center * representatives and other researchers, the committee reached agreement about its responses to four questions posed in its statement of task. The responses are summarized in this Executive Summary and discussed more fully in the report. What was the rationale behind the choice of these particular studies? Are appropriate topics considered? Should other topics be included? NIH has provided a justifiable rationale for the diseases selected for study in the WHI. These diseases are important causes of morbidity and mortality among women in the United States; they merit further research to test the efficacy of preventive measures and to develop effective programs that educate and motivate women to adopt proven prevention strategies. Heart disease and cancer are the leading causes of death among women, and the high incidence of breast cancer is appropriately a matter of concern. Osteoporotic fractures, which occur with much higher frequency among women than men, are a leading cause of disability among older women. Research leading to prevention of these conditions, or at least postponement of the age of onset, would be expected to improve substantially the quality of life of older women. NIH could have chosen other diseases or risk factors for ill health as the focus for the WHI, and those choices could also be defended. There are many unanswered questions regarding prevention of illness, enhancement of well-being, and delay of morbidity and mortality while maintaining an acceptable quality of life. Because the WHI is among the most complex studies ever undertaken, and because the WHI Clinical Trial is already under way, it would not be appropriate to suggest other topics to be included. This might overburden an already complex set of studies, could incur additional costs, and would almost certainly lead to further delays. Moreover, as noted above, it was not the committee's task to redesign the WHI. Are the methodologies and the study designs appropriate to address the stated research objectives? Is the size of the study population appropriate? The methodologies selected—a clinical trial, an observational study, and community studies—are appropriate for the study of the efficacy or effectiveness of certain interventions to improve women's health, for generating further hypotheses, and for implementing and evaluating community intervention strategies. Because details of the CPS have not been * The Vanguard Clinical Centers are the 16 clinical centers chosen by NIH to initiate the trial. An additional 29 clinical centers are scheduled to begin in 1994.

OCR for page 1
An Assessment of the NIH Women's Health Initiative decided, little can be said about the appropriateness of the specific methods and designs of the study. The majority of NIH staff time and effort has been devoted to designing the CT; therefore, it is the focus of most of the comments and recommendations in this report. Three primary hypotheses are the basis of the CT: A low fat dietary pattern will reduce the incidence of breast cancer and colorectal cancer; Hormone replacement therapy will reduce the incidence of cardiovascular disease; and Calcium and vitamin D supplements will reduce the incidence of osteoporotic hip fracture. The design of the trial, the number of women who will participate, and the methods used to determine health status all flow from these hypotheses. Some of the design decisions are based on evidence obtained from earlier studies, and some decisions are based on assumptions. Assumptions must be made regarding many aspects of any clinical trial, and examples include the strength of the protective effect, the number and timing of the endpoint events (occurrence of heart attacks, hip fractures, breast cancer), the ability to recruit and retain the study participants, and the ability to maintain adherence and behavior change in the intervention groups. To determine whether the proposed methods for the CT were appropriate for the stated research objectives, the committee analyzed the assumptions underlying the methods for each hypothesis and identified the major threats to the successful achievement of each hypothesis. The committee then weighed the risks of an unsuccessful trial against the potential benefits of additional information to be learned from a successful trial. Using this approach, the primary hypothesis that a low fat dietary pattern will reduce breast cancer and colorectal cancer incidence was judged by the committee to be inappropriate for emphasis as a primary hypothesis in the trial. The primary hypotheses of the HRT branch and the CaD branches were judged to be appropriate. Several secondary hypotheses are also included in the design of the CT. Most prominent is the hypothesis that a low fat dietary pattern will reduce the incidence of cardiovascular disease. Using the same approach weighing the threats of an unsuccessful trial against the benefits of a successful trial, the committee judged this secondary hypothesis to be appropriate for inclusion in the trial. These conclusions led to the specific recommendations about the design and conduct of the CT which have been presented.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Are the costs accurately estimated and sufficient to reach the desired objectives? Cost estimates are fairly well developed for the CT, deducible for the OS, and absent for the CPS. To a large extent, this reflects the stage of development of each component. It is therefore difficult to say with certainty that overall costs are sufficient to reach the objectives of the WHI. The CT appears to be very tightly budgeted. Although the approximately $1000 perparticipant per-year costs are within the range of costs (unadjusted for inflation) NIH incurred for other clinical trials conducted over the past 20 years, they are very low relative to the costs of more recently conducted comparable trials. As noted earlier in the Executive Summary, the committee believes that the total costs of the CT will be greater than the $625 million provided by NIH. NIH and Vanguard Clinical Center representatives have indicated that the additional funds necessary for successful completion of the trial will be covered by the institutions in which the Vanguard Clinical Centers are based. This reliance on institutional support may be reasonable in the case of the Vanguard Clinical Centers, but the committee felt it is unlikely that an additional 29 institutions can be identified that have both the experience to carry out the tasks of high quality research and the ability to provide additional resources. If all participating institutions honor their agreements to provide additional support for the CT, and if there are no unanticipated problems, the budgeted amount might be sufficient. However, it is very unlikely that there will be no problems. Will the study produce sufficiently reliable results to justify the costs? The committee concluded that valuable scientific information could be obtained in the redesigned study. However, the question of whether the investment is justified is, in part, a question about cost-effectiveness, and that involves consideration of alternative study designs. The committee thought it likely that much of the information could be obtained in better designed, smaller, more focused studies that could have a greater chance of success and probably be less costly. It recognized however, that a study in a broad-based population can have merit and proponents as well. The committee did not consider its task to be the consideration of alternative designs, and therefore did not do so. Thus, the committee admits some skepticism about the merits of this particular investment, but it cannot offer a definitive conclusion about cost-effectiveness. It is important to note that the modified study that the committee felt could be justified scientifically is quite different in its aims from the study proposed by NIH. The study proposed by NIH has as a primary focus the test of whether DM will reduce the risk of breast cancer. The modified study focuses on coronary heart disease. If NIH decides to accept the recommendations of the committee about modification of the study, it will need to consider whether the methods and study population proposed to test interventions on the risk of both coronary heart disease and breast cancer should be the same as those for a test

OCR for page 1
An Assessment of the NIH Women's Health Initiative of effects on the risk of coronary heart disease. Moreover, NIH will need to consider whether the study objectives that can be achieved in the modified study are worth the investment. Whatever the merits of the WHI, the committee has no doubt about the need for a substantial investment in research on women's health.

OCR for page 1
An Assessment of the NIH Women's Health Initiative Figure 1 Objective Prescheduled Reassessment (OPR) time frame.

OCR for page 1
An Assessment of the NIH Women's Health Initiative This page in the original is blank.