Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
1 Introduction CONTEXT OF THIS REPORT Since 1977 the National Institutes of Health (NIH) has conducted consensus development conferences to evaluate biomedical technolo- gies and practices and has disseminated me results to health profes- sionals and the public. The NIH Consensus Development Program (CDP), administered by the Office of Medical Applications of Re- search (OMAR) in NIH, is one of the more prominent health tech- nology~ assessment activities in the United States. The strengths of the NTH consensus development process are in its potential to translate a large body of research evidence into practical clinical policy, bring together apparently conflicting viewpoints, with the evi- dence as the "common denominator," draw public as well as professional attention to important clinical issues, obtain front-line practitioner input on the feasibility of evidence-generated clinical policy, and increase the exposure of all parties to the existing research evidence in an area" (Lomas, 1986~. The NIH CDP is widely known and has served as a model for the development of consensus development and group judgment pro- grams in the United States and abroad (Andreasen, 198X; CaUtorp, Throughout this report, technology includes drugs, devices, equipment, medical and surgical procedures, and other techniques and practices used in delivering health care and the systems in which such care is delivered. 3
4 CONSENSUS DEKEU7PME=AT THE NIH 1988; Goodman, 1988; Institute of Medicine [IOMI, 1985; Lomas, 19X6~. Nevertheless, and perhaps because of the initiative of the program and its broad exposure, concerns about specific conferences and the program as a whole exist. Some observers have contended that data provided to certain panels have been incomplete, overinter- preted, or misrepresented (Perry, 19871; that a number of recom- mendations have been made in the absence of supporting evidence (Ahrens, 1985~; that there is bias in the selection of panelists and speakers (Oliver, 1985~; and that topics have not always been suffi- ciently important to warrant this type of evaluation or expenditure (Perry, 1987~. The conference format, particularly the "grueling night sessions" (MuBan and Jacoby, 1985), has been described as onerous (Perry, 1987) and as perhaps leading to hurried conclusions (Oliver, 1985~. Others have noted that minority views may have been obscured or submerged, that there has been too much focus on compromise between viewpoints given the available research evi- dence, that ambiguous or overly generalized recommendations have resulted because of poorly worded questions or excessive compro- mise, and that a vocal few have been able to dominate some pro- ceedings gloom es, 1986~. Questions have been raised about the aware- ness of conclusions by clinicians (lacoby, 198S) and me program's impact on physicians' behavior (Gleicher, 1984; Kanouse et al., 1989~. Since the early 1980s OMAR has been actively involved in evaluat- ing past conferences and incorporating He results of the- analyses into its program guidelines. Although progress has been made, OMAR continues to seek improvements in the NTH COP (IOM, 1985; Kanouse et al., 1989). In response to a request from NIH to review the elements of its consensus development process and to develop recommendations to improve the process, lOM formed the Committee to Improve the National Institutes of Health Consensus Development Program un- der the aegis of the Council on Health Care Technology. The formal charge to the committee was to consider the following major compo- nents of the NTH process: · topic selection · role, size, and composition of the committee · formulation of the agenda · selection and preparation of speakers · presentation and synthesis of conference data
INTRODUCrlON · development of consensus · dynamics of the committee · preparation of initial and final consensus statements s The committee was also charged with formulating recommendations to improve the NIH CDP. The committee charge was specifically limited by the request of NIH not to address matters of He program's dissemination activity and impact, given already ongoing NTH ac- tivities in these areas. In particular, NIH contracted with Prospect Associates of Bethesda, Maryland, to organize an expert pane] in January 1989 on the role of medical opinion leaders in disseminat- ing consensus development conference recommendations. Also, the results of an evaluation of the NTH CDP's impact have been pre- sented in a recent report by the RAND Corporation, Changing Medi- cal Practice Through Technology Assessment: An Evaluation of the National Institutes of Health Consensus Development Program (Kanouse et al., 19891. The committee found its charge to be narrow. Factors beyond program function (i.e., the components of the NTH CDP noted in the committee charge) affect the success of the program. Therefore, the committee's recommendations address program purpose and scope, organizational issues, and financial support in addition to program function. Furler, dissemination of CDP findings is an essential element in the process, given the importance of using consensus findings to influence medical practice. Program impact should be examined when considering ways to improve the program. Thus, although the committee's circumscribed charge and the time avail- able for the committee's work precluded it from specifically ad- dressing the dissemination of consensus statements and impact of the process, the committee identified these as issues that require additional attention by NTH. NIH CONSENSUS DEVELOPMENT PROGRAM The NTH CDP2 borrows from Free models: (~) the judicial model, in which evidence is heard and weighed by knowledgeable impartial 2An extensive discussion of the NIH CDP is included in the background paper prepared for the committee that appears as Appendix A of dais report.
6 CONSENSUS DEVELOPMENT AT THE NIT judges or by juries of peers; (2) the scientific meeting, in which experts discuss their work with their peers; and (3) the town meet- ing, in which a forum is provided for all interested persons to ex- press their views (Mullen and Jacoby, 198S). NTH consensus devel- opment conferences include participation by speakers who present evidence, an audience that has the opportunity to comment on the evidence, and a pane! that deliberates and produces a written state- ment based on its judgment. The planning and implementation process of a particular consen- sus development conference usually takes 12 to 15 months. It be- gins with selection of a topic from among suggestions typically or~gi- nating from the bureaus, institutes, and divisions (BIDs) of NIH; other Public Health Service agencies such as the Food and Drug Administration or Centers for Disease Control; the Deparanent of Veterans Affairs; the U.S. Congress; or organizations outside of government (Goodman, 1988~. According to OMAR, four criteria are applied in evaluating potential topics: (~) the issue should have public health importance, (2) there should be controversy over sci- entific aspects of the issue, (3) there should be available evidence on which to base evaluation of the issue, and (4) the issue should be amenable to clarification on technical grounds (OMAR, NTH, 1988~. Once a conference topic is selected, a planning committee is formed consisting of OMAR staff, sponsoring institute staff, the prospective conference chair, and outside experts. The planning committee iden- tifies key conference issues and drafts questions relating to the topic being assessed. These questions normally address issues of the safety and efficacy of the technologies at hand and define the dimensions of the conference. The planning committee also recommends con- ference panelists, program fonnat, and speakers. Pane} members are selected to represent the various areas of expertise necessary to ad- dress the conference questions (e.g., technologic, clinical, and metho- dologic expertise). Conference speakers present evidence about the technology being evaluated to the panelists, who are responsible for responding to the questions posed by the planning committee. The conferences are open meetings to which members of the pub- lic and the medical community are invited. They usually last two and a half days. The first day and a half are normally spent in a plenary session in which speakers present information on the state of the science and the safety and efficacy of the technologies under consideration. These presentations are followed by open discus- sions engaging speakers, panelists, and members of the audience.
INTRODUCTION 7 Following the plenary session, the pane} convenes to consider the expert opinions of the conference speakers and other views expressed at the meeting and to draft responses to the conference questions. The resulting document, known as the consensus statement, is read to the audience on the morning of the third day for further comment and discussion among the panel and audience. The panel may choose to incorporate comments received during this session in the final consensus statement. The consensus statement may include minority opinions if a panel cannot achieve full agreement on a particular point. However, this option has been exercised only twice. The conference concludes with a press conference. After a final consensus statement is approved by the panel, the document is published by OMAR and widely disseminated to health care providers and administrators, the biomedical research and edu- cation communities, and the general public. Conference reports and summaries are published in medical and other scientific journals pertinent to the conference topic. Most consensus statements are published in the Journal of the American Medical Association, and some are presented on dedicated medical television networks. DESCRIPTION OF THIS STUDY The TOM Committee to Improve the National Institutes of Health Consensus Development Program held a one-day meeting on No- vember 2l, 1989. It was preceded by an evening session on Novem- her 20 at which Donald S. Fredrickson, who served as director of NTH from 1975 to 1981, described the origin and development of the NTH CDP. The meeting agenda was organized to correspond with the following components of the NIH CDP: topic selection planning committee formation agenda formulation: question drafting and speaker selection consensus panel selection and composition preparation of speakers, data synthesis, and data presentation development of consensus and group dynamics initial and final consensus statement preparation In addition to committee members, persons with experience in group judgment and with the NTH CDP were invited to participate in the meeting. Participants were provided with resource materials to help Hem prepare for the meeting.
8 CONSENSUS DEVELOPMI~NTAT THE NIH The meeting included two presentations about the NTH CDP. John Ferguson, director of OMAR, provided an update on the NTH CDP. Jacqueline Kosecoff, executive vice president, Value Health Sci- ences, Inc., and adjunct professor of medicine and public health, University of California at Loos Angeles, presented the relevant find- ings of the 1987 RAND Corporation study of me NTH CDP, Chang- ing Medical Practice Through Technology Assessment: An Evalu- ation of the National Institutes of Health Consensus Development Program (Kanouse et al., 1989~. A set of recommendations for improving the NTH CDP were de- veloped during the meeting and further refined in subsequent com- mittee conference cans and related discussions involving committee members and staff. These recommendations are presented in Chap- ter 2, Recommendations for Program Structure, and Chapter 3, Rec- ommendations for Program Function. The recommendations are printed in italics and are followed by a summary of the committee's underlying rationale. The committee's deliberations were constrained by the limited time and financial resources made available for this study. Nevertheless, the committee was able to draw upon OMAR's formal documentation of the NIH CDP; the background paper pre- pared by TOM staff for the committee; previous reports on the NTH CDP; published articles about the NIH CDP, over consensus devel- opment programs, group judgment and process, and data integration and synthesis methods; presentations and discussions at the meeting; participants t experiences as NIH CDP pane} chairs, members, speak- ers, consultants, and observers; and participants' experience with other consensus development and group judgment activities and with data evaluation and integration methods used in the heath field.
