Cover Image

PAPERBACK
$15.00



View/Hide Left Panel
Click for next page ( 148


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 147
Recommendations for Strengthening Consensus Development for Assessing Health Technologies The following recommendations concern strengthening the use of consensus development for assessing health technologies. These recommendations were drafted by a writing group following the Inter- national Workshop on Consensus Development for Medical Tech- nology Assessment, held in June 1989 at the King's Fund Centre in London, and coordinated by the Council on Health Care Technol- ogy, Institute of Medicine. The writing group, which included ten persons from five countries, drafted the recommendations based on the deliberations of the workshop. Members of the writing group included Gerard Breart, Clifford Goodman, Itzhak Jacoby, Egon lonsson, Arnold Kaluzny, Pedro Koch, Jacqueline Kosecoff, Tore Schersten9 Jackie Spiby, and Caroline Weill. The recommendations do not necessarily represent the views of the National Academy of Sciences or any of its constituent parts, the U.S. Depa'-~ent of Health and Human Services, or the organiza- tions with which the authors are affiliated. For the purpose of these recommendations, a consensus develop- ment program refers to an organizational entity that coordinates a series of consensus development conferences. The program may be a unit of, or sponsored by, one or more organizations. The confer- ences themselves are normally part of a broader process involving preconference planning and preparation and postconference activity such as dissemination of consensus statements. Each consensus 147

OCR for page 147
148 CONSENSUS DEVELOPMENT development conference normally addresses one or more related tech- nologies by assessing a set of technological properties, concerns, impacts, or other aspects pertinent to the use and effects of these. Health technologies refer to the drugs, devices, procedures, and or- ganizational and administrative systems used in providing or facili- tating health care. For the purpose of these recommendations, a consensus develop- ment process involves a group of experts or other representatives (i.e., a panel) that assesses a technology and formulates or prepares a set of findings that constitutes a consensus statement. The process of consensus development is based upon evidence provided to the panel in the form of literature and other documents, expert testi- mony, or other means. Although other group judgment processes may be conducted by remote panelists, consensus process panelists must have the opportunity for direct, face-to-face interaction in for- mulating and reviewing the panel's findings. These processes may be facilitated by staff, and a process could involve more than one panel, for example, a second group to provide an independent re- view of the main panel's work before a statement is adopted as policy. RECOMMENDATIONS 1. Consensus development programs should be sponsored by organizations that have the ability to implement or effectively dis- seminate consensus findings. 2. Programs should adopt the goal of bringing about changes in health and medical practice and related policies of national health authorities, industry, payers, academic institutions, and other agents. Sponsors and panelists should be cognizant of the intended audience for the consensus findings and the intended means for disseminating the findings. The consensus program should identify the ways in which the program in general, and each conference in particular, are intended to effect change. 3. The consensus development program should describe spe- cifically the scope of its concern related to technology assessment. This may include such technological aspects (i.e., properties, con- cerns, or impacts) as safety; efficacy, effectiveness, or patient out-

OCR for page 147
RECOMMENDAT70NS FOR STRENGTHENING 149 comes; cost, cost-effectiveness, and related economic concerns; pa- tient values or utilities; social, ethical, legal, and political concerns; guidelines for appropriateness, medical necessity, or quality; manner of application in research settings and in general or routine practice; and comparison of these among alternative technologies for specific clinical problems. These may vary from one conference to another. Programs are encouraged to assess all salient aspects of a technol- ogy; however, to the extent that a program's scope is necessarily more narrowly defined, consensus statements should note which important aspects are not addressed, and should note why these ought to be addressed by others. 4. Programs should seek to conduct assessments and provide recommendations in a timely fashion. On one hand, this requires consideration of the time needed to effectively plan, conduct, and report on conferences. On the other hand, consideration must be given to the pace at which technologies are developed and diffused and the need to provide timely guidance regarding their use. 5. The procedures and criteria for selecting conference topics and pane} members should be documented. 6. The questions to be addressed by the consensus pane} should be specific and manageable, that is, commensurate with the avail- able evidence, the time available for the process, and other resources. 7. Panelists should represent the relevant health and medical professionals, methodologists such as epidemiologists and biostatis- ticians, economists, administrators, patient or other consumer repre- sentatives, and others who can provide relevant perspectives. Pane! chairpersons and consensus program staff should be recognized as objective with regard to consensus topics and skilled in group pro- cesses. X. Consensus development programs should seek the best avail- able scientific evidence concerning the safety, efficacy, effective- ness, and other pertinent aspects of the technologies to be assessed. Clinical information should be made available to panelists about the use of procedures not only in research settings but in general or routine practice as well. Data from general practice are often not available in the literature and may have to be gathered by such means as review of patient records. For aspects for which evidence is not commonly available (e.g., social, ethical, or legal issues or values, preferences, or other patient, family, or community perspec- r

OCR for page 147
150 CONSENSUS DEVELOPMENT lives), programs may have to make special efforts to acquire or elicit reliable information. 9. Prior to a consensus development conference, programs should provide an ordered and categorized compilation or synthesis of re- search reports and related evidence concerning the technological aspects at issue. Sources may include reports of laboratory or bench studies, randomized clinical trials, epidemiologic or other observa- tional studies, patient record audits, patient surveys, qualitative lit- erature reviews, and individual expert opinion. Published literature as well as relevant unpublished surveys, trials, and data should be included. Source documents should be clearly cited and should be graded or otherwise rated for the strength or magnitude of their findings and for the rigor or quality of their methodologic approach. Copies of original documents should be available to panelists as needed. When resources and time permit, a meta-analysis of applicable data should be provided. Meta-analysis is a statistical method for obtaining quantitative answers to specific questions from multiple reports of primary studies on a particular subject. Using data ob- tained from each primary source, a synthesis is made that may pro- duce a stronger conclusion than that which any of the separate re- ports can provide. 10. All panelists should be involved in interpreting evidence, even though they will have varying levels of expertise for doing so. Consensus development programs should make available basic guid- ance or training concerning the use of evidence when needed to ensure that all panelists have at least a fundamental understanding of the role of evidence in the process. ~ I. The conduct of consensus development processes should be structured and documented. In particular, processes should provide for me following. a. In advance of the formal consensus conference, panelists should develop an organized compilation of points to be addressed at the conference, rather than relying on the conference alone to formulate these. b. An operational definition of consensus should be specified (e.g., full agreement, majority agreement) as well as how to present less than full agreement in the panel's findings (e.g., by citing mi- nority opinions). This should be made known to panelists and other participants before a consensus conference is undertaken. Consen-

OCR for page 147
RECOMMENDATIONS FOR SIRE7J=HENING .'. sus may be determined by, for example, particular voting require- ments or other defined rating mechanisms. c. Evidence, including meta-analyses or other ordered informa- tion syntheses, expert testimony, etc., should be presented in a form that is comprehensible to all panelists, including those who may not be quantitative experts or specialists. Evidence should be presented in a manner that is consistent with the order and nature of the issues or questions to be considered and in a timely manner that allows sufficient opportunity for panelists' thorough review. d. The consensus development conferences should be structured so that participants have the opportunity to contribute equally to the process. This is important in instances in which panelists represent multiple disciplines and may not be familiar with others' perspec- tives;on consensus topics. In particular, a structured process is called for in the voting or other delineated means used in rendering a final set of consensus findings. e. The duration and spacing of consensus conference sessions should be such that deliberations are not unduly affected by panel- ists' experiencing fatigue or lack of attention. 12. Each consensus statement should include the following: a. a general description of the consensus development process or approach used; b. notations regarding the strength of agreement or assurance of the pane} concerning its findings; c. a description of the reasoning used by the pane} and the evi- dential basis for the consensus findings; statements should include summary descriptions of the conference source documents (i.e., at least a bibliography, preferably accompanied with annotations con- cerning topic, methodology, and findings of cited studies); and d. recommendations for research needed to resolve those issues concerning which panels could not reach agreement and to other- wise advance understanding of the topics. This documentation of process, evidence, and needed research should enhance internal consistency of the statement, enable users to follow the reasoning of the panelists, convey the context and appli- cability of the findings, and provide the basis for reassessment in light of new developments. ~ 3. The consensus development;program should provide for timely review, for example, in the form of peer review or an oversight committee established by the sponsoring organization, of the con-

OCR for page 147
152 CONSENSUS DEVELOPMI~ sensus development conference findings by experts who are not panelists for the particular conference that generated the findings. The purpose of this review would be to ensure that the questions posed to the pane} were adequately addressed and that the findings were reasonably supported and not inconsistent or otherwise errone- ous with respect to the evidence. 14. Programs should consider alternative modes of information dissemination to increase the effectiveness of inducing change. These efforts may include appropriate marketing approaches such as devel- opment of information products that best convey consensus findings to specific target audiences of medical professionals, patients, pay- ers, industry, and others and effective means of calling attention to the need for more definitive research to resolve issues for which well-founded consensus could not be achieved. 15. Consensus development programs should provide for moni- toring new developments that may overturn or significantly alter the available evidence pertaining to a technology that has been assessed in previous consensus development conferences, so that the program may call for a reassessment as appropriate. Such developments may include technological advances, reports of new clinical trials, and changes in the way technologies are applied in practice that call into question their safety or effectiveness. 16. The program snouts provide for formal evaluation u~ Fit; impacts of the program and specific conferences, such as by acquir- ing data that would measure changes in practice behavior or contr~- butions to policy initiatives, or demonstrate other impacts. This should include evaluation by parties that are independent of the program. . ~ ~ ~ ~ d ~ _ _ ~~ ~ ^~ _ 17. In order to improve consensus development processes, stud- ies should be conducted that answer the following questions. a. How effective are consensus development processes in chang- ing health and medical practice behavior? b. If consensus development processes are effective, what con- ditions or factors (e.g., identity of sponsoring organization, timing of conferences with respect to technologies' diffusion, documenta- tion of panel's reasoning, dissemination strategies) contribute to their effectiveness? c. If consensus development processes are not effective in chang- ing health and medical practice behavior, do they confirm prevailing practices? d. Can decision support resources, such as decision analysis or

OCR for page 147
RECOMMENDATIONS FOR STRENGTHENING 153 particular decision-making techniques, strengthen consensus devel- opment processes? e. Do consensus findings reflect increased understanding and convergence of the opinions of panelists or agreement only on the "least common denominator" (i.e., least controversial and most commonly accepted issues)?

OCR for page 147