Cover Image

PAPERBACK
$15.00



View/Hide Left Panel
Click for next page ( 138


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 137
Profile of the Consensus Development Program in the United States: The National Institutes of Health Office of Medical Applications of Research Sharon R. Baratz NATIONAL CONTEXT Throughout the 1970s, the acceleration of technological innova- tion in medicine, accompanied by rising costs and increased con- ce~ns for the quality of care, generated extensive interest in tech- nology assessment. Following the initiation of the Consensus De- velopment Program in 1977, the Office of Medical Applications of Research (OMAR) was formally established in the Office of the Di- rector of the National Institutes of Health (NIH) in 1978. OMAR is the focal point for activities to improve the assessment and transla- tion of results from NIH-supported biomedical research into knowI- edge that can be applied safely and effectively in the practice of medicine and public health. OMAR is part of NIH, the primary government-sponsored biomedical research facility in the United States. The principal vehicle for OMAR's efforts in me systematic as- sessment of biomedical technologies is the Consensus Development Program. Each consensus development conference is cosponsored by OMAR and one or more of the NIH bureaus, institutes, or divi- sions (BIDs). Other federal agencies with biomedical components *The author acknowledges Jerry Elliott of OMAR, NIH, for reviewing the manu- script. 137

OCR for page 137
138 CONSENSUS DEVELOPMENT may participate in the sponsorship of particular conferences, de- pending on the topic (OMAR, NIH, 19X81. The approximate cost per assessment for the OMAR conferences is $116,000 (Elliott, 19891. The purpose of the conference is to "evaluate the available scientific information on a biomedical technology and to produce a consensus statement that advances understanding of the technology or issue in question (assessment) and that will be useful to health professionals and the public at large (transfer)" (OMAR, NIH, 1988~. The three main goals of the consensus development program are to: provide a setting for the evaluation and review of the scientific soundness of health technologies for a particular clinical condition or for a particular health-related technology, with emphasis on safety and efficacy aid in the diffusion of knowledge of advances in biomedical technology, through dissemination of the findings from the consen- sus development process to physicians and consumers to facilitate the diffusion, adoption, and appropriate use of tech- nologies found to be sound. SCOPE OF THE PROGRAM A broad variety of technologies have been topics of consensus development conferences, including medical and dental drugs, de- vices, procedures, facilities, and support systems used in prevention, diagnosis, and treatment. Conferences tend to focus on a technol- ogy, for example, electroconvuisive therapy (1985) and magnetic resonance imaging (1987), or on a particular clinical problem and the alternative technologies applied for prevention, diagnosis, treat- ment, or rehabilitation of these, for example, travelers' diarrhea (1985) and adult urinary incontinence (1988~. Although the consensus development program was to have origi- naDy focused on emerging technologies, most of the conferences have addressed technologies already in clinical use, especially new or widely used technologies. This is largely because evaluative information regarding many emerging technologies is insufficient for the level of validity sought for consensus development confer- ences and because many technologies already in widespread use have

OCR for page 137
UNITED SIATESNA770NALINSI771JTES OF HEALTH 139 not been carefully scrutinized for safety and efficacy (Perry and Kalberer, 1980~. The Consensus Development Program at OMAR is primarily concerned with the safety, efficacy, and clinical applica- tion of technologies. Conferences do not usually directly address social, ethical, legal, economic, or political issues surrounding tech- nologies. For the most part, the BIDs of NIH suggest consensus topics to OMAR for consideration. Topics are considered from other sources, including other Public Health Service agencies such as the Food and Drug Administration, the U.S. Congress, or organizations outside of government (Goodman, 1988~. The following criteria are currently in use by OMAR for the selection of conference topics. The subject under consideration should have public health im- portance. The topic should affect or have broad application to a significant number of people. There should be controversy surrounding biomedical/scientific aspects of the topic that would be clarified by the consensus ap- proach or a gap between current knowledge and practice that a con- ference might help to narrow. The topic must have an adequately defined and available base of scientific information to answer the previously posed questions and to resolve the controversies insofar as possible. The topic should be amenable to clarification on technical grounds, and the outcome should not depend mainly on the impres- sions or value judgments of panelists. Additional elements desirable for positive consideration of a consen- sus topic include health care cost impact, preventive impact, and public interest (OMAR, NIH, 1988~. The topic selection process may take from two months to a year or more. FORMAT AND CONDUCT OF THE PROCESS Once a topic is chosen, a senior OMAR staff person works with a designated BID coordinator and other BID staff to organize the con- ference. OMAR's focus is on the consensus development process while the initiating BID's contribution concerns the scientific infor- mation required for the conference topic (Elliott, 1989~. A planning committee of OMAR staff, BID staff, the conference chairperson,

OCR for page 137
140 CONSENSUS DEVELOPME:NT and outside experts (who are not federal employees) begin to organ- ize the conference. The program planning committee has four major functions: I. to draft consensus questions (usually four to six questions) 2. to draft the conference program 3. to recommend conference speakers 4. to recommend consensus panel members (OMAR, NIH, 198X). The chairperson of the consensus development conference is se- lected for his or her stature as a distinguished physician or scientist and for personal skills in chairing the open symposium portion of the conference and in leading the consensus panel. The size of the panels has varied from g to 16 individuals; most have had l0 to 12 members. OMAR seeks balanced representation from various sec- tors of professional and community life. The planning committee usually decides upon the appropriate areas of expertise needed for panelists of a particular conference. OMAR holds that all panelists, including the chair, should have no vested interest in the assessment topic and should be able to weigh evidence and to collaborate. According to OMAR, panels should include individuals involved in research in the field; health professionals who are users of the tech- nology; methodologists or evaluators such as epidemiologists or biostatisticians; and public representatives such as ethicists, law- yers, theologians, economists, public interest groups or voluntary health association representatives, consumers, and patients. Panel- ists should be residents of the United States and should not be fed- eral employees, to avoid the appearance of undue federal influence (OMAR, NIH, 1988~. The planning committee also selects speakers on the basis of their expertise and their ability to present evidence on the safety, efficacy, effectiveness, and service requirements, as appropriate, of the technology in question (Goodman, 19881. OMAR recommends that the planning committee suggest speakers to pre- sent opposing data and interpretations where controversy exists (OMAR, NIH, 1988~. The planning and implementation of a consensus development conference usually involves 12-14 months of work after topic selec- tion. Observers have noted that the consensus development confer- ence borrows aspects from the scientific meeting, the judicial pro- cess, and the town meeting (Iacoby and Rose, 1986; Mullan and Jacoby, 1985~. Consensus development conferences are open meet-

OCR for page 137
UNITED SIATE:5 NATIONAL INSrl71~ES OF HEALTH 141 ings to which members of the public and the medical community are invited. The conference begins on a Sunday night with a closed session of the conference panel. During this session, panelists meet with OMAR staff to clarify questions about the consensus develop- ment conference. Several months before the conference, panelists divide into subcommittees to focus on particular consensus ques- tions so that different pane! members are responsible for writing specific portions of the consensus statement. All panelists remain responsible for the statement as a whole and are to follow all presen- tations and deliberations in the consensus development process. Panelists may meet as subcommittees on Sunday night. The follow- ing one and a half days are devoted to the plenary session for the expert presentations (15-30 minutes each, interspersed with open discussions involving speakers, panelists, and questions from the audience). In the evenings, the panel convenes to draft consensus answers to the predetermined questions, considering the expert opinions of the conference speakers and other views expressed at the meeting. The panelists and chairperson for each conference decide on the rules for consensus in the executive sessions. The consensus view of the pane} is not necessarily that of all panelists. If a pane} cannot achieve full agreement on a particular point, the consensus statement may identify opposing or alternative opinions and/or majority-minority viewpoints. Few conferences have produced minority statements. The chair reads the consensus statement to the audience on the morning of the third day for further comment and discussion among the pane} and audience. The pane} may choose to revise the state- ment based on comments received during this session. The confer- ence concludes with a press conference. z DOCUMENTATION AND USE OF EVIDENCE IN CONSENSUS DEVELOPMENT Panelists receive abstracts of the speakers' presentations at least one month in advance of the actual conference date, to prepare them for the consensus development conference. Speakers are also asked to bring photocopies of their slides to the consensus development conference for panelists. The BID coordinator is responsible for the supply of overview articles and other supplemental materials for the panelists prior to the consensus development conference. The BID

OCR for page 137
142 CONSENSUS DEVELOPMENT coordinator meets with staff of We National Library of Medicine in order to direct and detail the strategy for preconference information retrieval. Panelists receive a copy of this literature search approxi- mately three months prior to the actual conference. Speakers and members of the audience receive conference matenals, including speakers" presentation summaries, a conference agenda, and logisti- cal information, at the first public meeting of the panel. The con- sensus statement's validity rests in part upon Me scientific evidence presented to the pane} for decision making. In general, the evidence presented prior to and during the confer- ence is not formally weighted for integration in the consensus devel- opment process. A few conferences have employed decision-assist models to help the pane} explore the implications of the data pre- sented by speakers. The extent of the panelists t dependence upon such models in decision making has varied among the conferences (lacoby and Pauker, 1986~. Decision analysis may be used to pro- vide a mode} of the sequence of potential strategies and outcomes for the questions at hand. The data available are used to structure the pathways in terms of probable occurrence and to compute utili- ties for the alternative outcomes. Strategies may be tested under different assumptions of risk and utility values. The decision-assist models are intended to `'help structure complex alternatives in a rational way [and to allow] the incorporation of expertise and infor- mation from a variety of expert consultants without abdicating the decision to any one of them" (McNeil and Pauker, 1984~. There are no explicit rules for the consideration of evidence by the panel. The panels for each conference decide upon the definition of consensus. The chairperson directs and leads the discussion and consideration of evidence in a format acceptable to the panel. The final product of the consensus development process is the consensus statement. Consensus statements do not cite articles used as evidence. Each consensus statement contains a list of the mem- bers of the program planning committee, the expert speakers, and the panelists. DISSEMINATION AND IMPACT The OMAR Director of Communications and the BID Informa- tion Officer develop a plan to announce the conference arid to dis- seminate the consensus statements. Conferences usually receive

OCR for page 137
UNITED SIATESNATlONAL INSrlTl~ES OF HEALTH 143 considerable attention from the medical media and general media at the time of their occurrence. The plan may include a combination of the following. The consensus statement is printed by OMAR and distributed routinely to a variety of federal health agencies, health care organi- zations, and the directors of continuing education of American Hos- pital Association membership hospitals. Additionally, the consensus statement is sent to targeted individuals and organizations specified in the infonnation dissemination plan. The Journal of the American Medical Association routinely publishes most of the consensus statements. Consensus statements are also published by specialty journals in the topic area. OMAR places notices in numerous professional journals to announce future conferences as well as the availability of consensus statements. The publication of the consensus statement along with selected papers from a CDC [consensus development conference] as a sym- posium is also a possibility. Proceedings of several conferences have been published in this manner either as supplements to specialty journals or as a monograph. Summary videotapes and audiotapes of the conference may also be prepared and distributed. . A summary of the statement is also prepared and sent to appro- priate specialty journals (OMAN NIT 1988). Two consensus development conferences, on prostate cancer (1987) and urinary incontinence (1989~9 were televised live via satellite throughout the nation. The consensus development program at NIH has undergone for- mal evaluation. The process continues to evolve in an attempt to improve health care practices in the United States. The procedures for conference planning, formulation of questions, and report dis- semination have become more standardized, as have the formats for conducting the conferences and the final consensus statements. Winkler et al. (1986) studied the dissemination of consensus de- velopment conference information in the popular press and concluded that the reports of the NTH conferences appear to be factual and balanced. Direct mailing seems to augment the success of informa- tion transfer to targeted groups. A variety of sources is needed to inform practitioners and the public (lacoby and Clark, 19861.

OCR for page 137
144 CONSENSUS DEVELOPMENT OMAR conducted a survey to measure me effectiveness of two conferences (computed tomography scan of the brain, 1981; hip- joint replacement, 1982), as evidenced by the extent to which physi- cians were aware of the conferences and the conclusions reached at each one. The surveys determined that awareness varied greatly among different specialties. The study concluded that there was much room for improvement in information dissemination and that it would be fruitful to examine physicians' information-seeking hab- its, such as examining the role of opinion leaders, so as to better design strategies that more effectively disseminate conference re- sults (Iacoby, 1983~. In another study by the RAND Corporation to evaluate the assess- ment program, Kanouse et al. (1987) investigated the impact of the consensus development conferences on physician awareness and behavior. They found that efforts to reach the practicing community can be improved. The consensus development program is more successful at reaching specialists than generalists. In addition, phy- sicians who frequently participate in continuing medical education are more likely to have heard of consensus development conference recommendations. The physicians surveyed knew more about the content of specific consensus findings than about the program as a whole and the consensus development conference process. The physicians found information on clinical practice, in summary form accompanied by evidence, to be most useful. The group from the RAND Corporation also investigated medical records in the state of Washington to determine whether the quality of care improved with respect to 12 recommendations by four con- sensus development conference panels. The results indicated that the conferences did not affect clinical practice in the manner in- tended. In some instances, physicians had adopted the consensus recommendations on patterns of care before the conference or no change was shown in comparisons of patterns before and after the conference or other constraints determined clinicians' practices, such as a lack of available resources (Kosecoff et al., 1987~. In early 1990, the Institute of Medicine (IOM) completed an evalu- ation of the NTH Consensus Development Program, conducted at the request of NIH. Although not charged with evaluating the program's dissemination activities and impact, the IOM did address program purpose and scope, role and placement of the program within NIH, aspects of the consensus development process, and financial support for the program (Institute of Medicine, 1990~.

OCR for page 137
UNITED STATES NATIONAL INSIlTUTES OF HEALTH REFERENCES 145 Goodman, C., ed. 1988. Medical Technology Assessment Directory. Washington, D.C.: National Academy Press. Institute of Medicine. 1990. Consensus Development at the NIH: Improving the Program. Washington, D.C.: National Academy Press. Jacoby, I. 1983. Biomedical technology, information dissemination and the NIH consensus development process. Knowledge: Creation, Diffusion, Utilization 5~2~:245-261. Jacoby, I. 1985. The consensus development program of the National Institutes of Health: Current practices and historical perspectives. International Journal of Technology Assessment in Health Care 1~2~:420432. Jacoby, I., and S.M. Clark. 1986. Direct mailing as a means of disseminating NIH consensus statements. Journal of the American Medical Association 255~10~:1328- 1330. Jacoby, I., and S. Pauker~. 1986. Technology assessment in health care: Group process and decision theory. Israeli Journal of Medical Science 22:183-190. Jacoby, I., and M. Rose. 1986. Transfer of information and its impact on medical practice: The U.S. experience. International Journal of Technology Assessment in Health Care 2~1~:107-115. Kanouse, D.E., R.H. Brook, J.D. Winkler, J. Kosecoff, S.H. Berry, G.M. Carter, J.P. Kahan, L. McKloskey, W.H. Rogers, C.M. Winslow, G.M. Anderson, L. Brodsley, A. Pink, and L. Meredith. 1987. Changing medical practice through technology assessment: An evaluation of the NIH consensus development program. Santa Monica, Calif.: RAND Corporation. Kosecoff, J., D.E. Kanouse, W.H. Rogers, L. McCloskey, C.M. Winslow, and R.H. Brook. 1987. Effects of the National Institutes of Health consensus development program on physician practice. Journal of the American Medical Association 258(19):2708-2713. McNeil, B.J., and S.G. Pauker. 1984. Decision analysis for public health: Principles and illustrations. Annual Review of Public Health 5:135-161. Mullan, F., and I. Jacoby. 1985. The town meeting for technology: The maturation of consensus conferences. Journal of the American Medical Association 254(8): 1068-1072. Office of Medical Applications of Research, National Institutes of Health. 1988. Guidelines for the selection and management of consensus development confer- ences. Bethesda, Md. Pamphlet. Perry, S., and J.T. Kalberer, Jr. 1980. The NIH consensus-development program and the assessment of health-care technologies: The first two years. New En- gland Journal of Medicine 303:169-172. Winkler, J.D., D.E. Kanouse, L. Brodsley, and R.H. Brook. 1986. Popular press coverage of eight National Institutes of Health consensus development topics. Journal of the American Medical Association 255~10~:1323-1327.

OCR for page 137