Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Recommendations for Program Function PLANNIN G/PREPARATION Topic Selection The topic selection process should generate and select topics that reflect the priorities of the broader health care system. Specifically, OMAR should have a standing mechanism for actively soliciting topic suggestions throughout the health care system on a regular basis. This solicitation should include, but not be limited to, profes- sional organizations, specialty societies, third-party payers, health care providers, government agencies, and the general public. OMAR should describe the process for generating potential topics to facili- tate contributions by interested indlividuals and organizations. The process of selecting topics should be formalized! and include partici- pation by the aforementioned OMAR advisory council, OMAR stay, and BID directors. The selection of a topic is a critical determinant of the value of a consensus conference. The NTH CDP has been criticized for exam- ining topics that do not merit the evaluation or expenditure associ- ated with the program. Thus, it has been suggested that "there should be a mechanism for setting priorities among the approaches to be assessed" (Perry, 1987~. The current process for selecting 18
RECOMMENDATIONS FOR PROGRAM FUNCTION 19 topics, as described in the OMAR Guidelines (OMAR, NTH, 1988), relies largely on input from within NTH, without adequate represen- tation of other segments of the health care system or consideration of important nonbiomedical issues. The active solicitation of topic suggestions on a regular basis will assist OMAR in identifying the concerns of the health care system. Examples of agencies and organizations that might provide topic suggestions include, but are not limited to, the Agency for Health Care Policy and Research, American Academy of Family Practitio- ners, American College of Physicians, American College of Sur- geons, American Hospital Association, American Medical Associa- tion, American Nursing Association, Blue Cross and Blue Shield Association, Centers for Disease Control, Council of Medical Spe- cialty Societies and its member organizations, Department of Veter- ans Affairs, Food and Drug Administration, Health Care Financing Administration, Health Industry Manufacturers Association, Health Insurance Association of America, National Institutes of Health, and Pharmaceutical Manufacturers Association. A formal process involving an OMAR advisory council, OMAR staff, and BID directors should be used to set priorities among po- tential topics. Consistent with the earlier discussion on the relation- ship of consensus development to changing behavior, the criteria for topic selection should be augmented to reflect an emphasis on the potential to improve the effectiveness of clinical practice, as well as the feasibility of achieving such results. For each potential topic, it will be useful to identify the specific groups that most need or are likely to use the consensus findings, the use that such groups might make of the information, and the approximate magnitude of impact that the findings are anticipated to have within a specified time frame. Evidlence on the state of clinical practice associated! with poten- tial topics related! to existing technologies or practices should be analyzed to help OMAR in selecting topics, the planning committee in drafting questions, and the pane! in developing its statement. In determining whether a potential CDP topic is controversial and whether a conference on that topic is likely to be useful, OMAR should consider the available information about pertinent practice patterns as well as the current state of the applicable science (Kanouse
20 CONSENSUS DEVELOPME=AT THE NIH et al., 1989~. The extent to which there is evidence about clinical practice variation, potentially inappropriate utilization, and related aspects of practice should be taken into account when choosing CDP topics. Evidence should be examined on the extent to which prac- tices are used, the circumstances under which they are used, and the use of any particular relevant techniques (e.g., medical or surgical). Clinical practice information will help planning committees to es- tablish the scope of consideration for CDP topics and to target con- ference questions. Such information may be necessary for determin- ing whether matters of delivery or access, or related economic, so- cial, or ethical issues, should be considered. Panels may use clinical practice evidence for determining, in part, how their recommenda- tions might be expected to affect clinician behavior (e.g., result in the reduction, increase, or other modification in the use of a proce- dure). OMAR could review recent data collected for other purposes (e.g., Medicare claims data, data collected by Peer Review Organizations, the National Disease and Therapeutic Index, or the National Ambu- latory Medical Care Survey) (Kanouse et al., 1989~. OMAR should coordinate this effort with AHCPR, which is to conduct research and evaluation on the extent to which rates of utilization vary among similar populations for particular diseases, disorders, and other heady conditions; uncertainties exist on the effect of utilizing a particular service or procedure; or inappropriate services and procedures are provided (U.S. Congress, 1989~. Some medical practices and technologies have been widely adopted, despite He lack of reliable information about benefits and risks (Asch and Lowe, 1984~. Even when evidence is not adequate to support firm conclusions about safety and efficacy, NTH consensus develop- ment conferences on management of important clinical problems may be useful. For example, a consensus development conference might evaluate an important, frequently used technology for which evidence of safety and efficacy is inadequate for developing recom- mendations about its appropriate use. Such a conference could iden- tify specific areas for further research and would provide a more tenable basis for health care delivery, payment, and related deci- sions by informing clinicians, patients, provider organizations, arid third-party payers that adequate evidence for wet/-grounded recom- mendations on the technology's use is lacking.
RECOMMENDATIONS FOR PROGRAM FUNCTION 2 The evidence criterion for topic selection should be broadened to include various types of scientifically sound evidence; in addition to evidence on safety and efficacy, such evidence might include prac- tice patterns and utilization, access, and the findings of other rele- vant health services research. Currendy, for a topic to be selected for consensus review, "the topic must have an adequately defined and available base of scien- tific information to answer me previously posed questions and to resolve the controversies insofar as possible" (OMAR, NTH, 19883. Given the above recommendations to broaden me scope of the pro- gram and to consider evidence on the state of clinical practice in selecting topics, me existing topic selection cnter~on regarding data needs should be broadened. Planning Committee Formation The planning committee members should include members who understand the clinical, methodologic, and societal issues related to the conference topic. The planning committee should include repre- sentatives of relevant BIDs, health care providers (physicians, nurses, other health care professionals, and delivery organizations), third- party payers, government agencies, health services researchers, epi- d~emiologists, biostatisticians, economists, ethicists, patients, and the general public, as relevant to the topic at hand. The planning committee for each conference greatly influences the outcome of a consensus development conference by drafting the questions that define the scope and substance of the conference, by nominating panelists who will respond to those questions, and by nominating the speakers who will present the information on which, in large part, the panelists win base their findings. Accordingly, the membership of the planning committee should reflect a balance of the parties interested in the selected topic in its entirety and possess a comprehensive understanding of the conference topic (as broadly defined above). However, achieving broader membership on the planning committee should not compromise the efficiency of that group, and OMAR should balance me needs for broad representation
22 CONSEI`ISUS DEVELOPMENT AT TlIE NIH and effective group size when appointing planning committee mem- bers. The planning committee chair should be an individual who is not employed by NIH. The planning committee chair should be selected in consultation with OMAR staff, relevant BID directors, and the OMAR advisory council recommended above. As noted above, the planning committee shapes the consensus development conference and can greatly influence its outcome. There is a high degree of NIH staff member participation in the CDP. It is desirable to complement NIH staff contributions to the process by assigning the role of planning committee chair to an individual who can broaden representation from other health care sectors. Drafting Conference (~'e~tion~ The planning committee should publicly solicit questions concern- ing a selected topic from a broad base of relevant organizations and individuals, including OMAR advisory council members. This could be accomplished through announcements in professional journals, the Federal Register, and other notices and by contacting profes- sional, profiler, industry, government, and patient organizations. Conference questions should be "selected by means of a more thorough search procedure" (Wortman and Vinokur, 1982~. The questions posed to a CDP pane} should reflect a broad spectrum of health care concerns about the conference topic. In order to under- stand and include the perspectives of the various health care system participants in the questions, representatives of these participants should be consulted during the question preparation process. Persons not involved in the process of drafting conference ques- tions should review the questions to ensure that they are clearly stated and will be understood by the consensus panel, conference participants, and readers of the consensus statement. PA list of the conference questions from the NIH consensus conferences held between October 1987 and September 1989 is provided in Appendix C.
RECOMMENDATIONS FOR PROGRAM FUNCTION 23 Conference questions should be stated clearly to eliminate ambi- guity or biased interpretation. Careful and precise definition of the questions to be addressed is required to investigate the topic eff~- ciently (Fink et al., 1984), to maximize the opportunity fovea valid evaluation, and to avoid a predetermined conclusion (Oliver, 1985~. Poorly worded questions can result in ambiguous or overly general- ized recommendations Comas, 19X6~. Speaker Selection Suggestions for speakers should be solicited from a wide range of knowledgeable persons in NIT, other government agencies, profes- sional organizations, and other appropriate groups. A more thorough search procedure should be used in selecting speakers (Wortman and Vinokur, 1982~. Presentations by confer- ence speakers should reflect the breadth of knowledge and diversity of opinion about a topic. Solicitation of suggestions for conference speakers will enhance the planning committee's ability to determine when differences of opinion exist about a topic and to "include the presentation of opposing data and interpretations" (OMAR, NIH, 19881. Broadening the process for selecting speakers will lessen the potential for biased selection of speakers (Perry, 1987~. Consensus Pane' Selection and Composition The panel chair plays an important role in shaping the outcome of consensus conferences (Wortman and Vinokur, 1982~. Thus, as OMAR's guidelines state, the chair should be "knowledgeable and prestigious" in the field of medical science under consideration, but should not be "identified with strong advocacy positions regarding the consensus topics or with research that might be presented" (OMAR.~ NTH, 1988~. Further, the individual selected should have strong leadership and group process skills. There is a risk that a pane} chair may become too influential, particularly in light of the pane} chair's participation in the planning committee. Panel chair participation in the planning committee provides continuity of the planning and implementation of conferences, enhances the pane} chair's understanding of the process and the specific conference objectives, and develops a stronger relationship between the chair
24 CONSENSUS DEKE:LOPMENTAT THE NIH and OMAR staff. However, the objectivity of the process may be compromised by the potential for one individual to exert excessive control over the outcome of the process. This committee did not reach agreement on whether the pane} chair should be a member of the planning committee. Although a majority of committee members indicated that the benefits of in- volving the pane} chair in the planning process outweigh me poten- tial risks, some did not. A number of committee members indicated that OMAR should explore modified roles of the pane} chair (e.g., not having the pane} chair serve on the planning committee or intro- ducing a professional facilitator to assist the pane} chair, as dis- cussed below). Suggestions for panelists should be solicitedF from a broad range of organizations and individuals interested in the consensus topic. Solicitation of suggestions for pane} members from both within and outside of NIH will enhance the planning committee's ability to achieve "balanced representation from various sectors of professional and community life" (OMAR, NTH, 1988~. Achieving this represen- tation will validate the pane} recommendations (Asch and Lowe, 1984~. As noted above, formal solicitation of suggestions for con- ference participants may help to offset the potential for bias in the process. Conference panelists, when possible, should possess overlapping expertise. The breadth of panel expertise should be commensurate with the issues being addressed by a conference. When assembling the con- ference panel, the planning committee should carefully identify and distinguish among the various disciplines required to address confer- ence questions. For example, the related fields of biostatistics and epidemiology may both need to be represented on a conference panel. Achieving the necessary breadth of expertise must be accomplished while maintaining a manageable group size for the pane} (Wortman and Vinokur, 19X2~. Selection of panelists who possess more than one area of expertise facilitates the balancing of group size and group expertise. Further, there are potential disadvantages associ- ated with representation of a discipline by a single expert. First, the overall composition of the consensus pane} can influence the degree
RECOMMENDATIONS FOR PROGRAM FUNCTION 25 of participation by individual panelists in the consensus develop- ment process; a single pane] member with a particular expertise might be isolated during pane} deliberations. Second, sound deci- sions are best achieved when the panel is not forced to defer to a single individual on the pane} for crucial expertise. The selection of panelists with multiple areas of expertise reduces the likelihood of a pane} containing a solitary expert, without increasing the size of the panel. IMPLEMENTATION Preparation of Speakers, Data Synthesis, and Data Presentation Speakers should submit, in advance of the conference, brief state- ments that outline their positions, reasoning, or findings on each specific question they have been asked to address; the full text of the key referenced articles on which they base their positions; and, pref- erably, the full text of their presentations. Panelists must deal with a large amount of evidence during the consensus conference. Prior to the conference, panelists should re- ceive the information to be presented at the conference in a well- organized format. This will provide more time for them to evaluate the information and prepare any questions for the conference speak- ers. Providing these materials to the panelists before the conference may help to diminish any undue influence of speakers' manners of presentation or personal characteristics on the panels' evaluation of the evidence being presented. Further, this information can serve as reference material to Me pane} during its deliberations. Whenever possible (i.e., when the nature of available data makes this feasible), meta-analysis2 should be performed before the pane! 2Meta-analysis is a statistical method for obtaining quantitative findings, e.g., on die effect size of a medical intervention, from multiple reports of primary studies on a particular subject. From information obtained from each primary source, a synthe- sis is made diet may produce a stronger conclusion than that which any of the separate reports can provide. It is generally most appropriate when there are no definitive studies on a topic and the nondefinitive studies are in some disagreement (Bulpitt, 1988; Hunter, 1982; Louis et al., 1985; Pillemer and Light, 1980; Sacks et al., 1987; Whacker, 1988~.
26 CONSENSUS DEVELOPME~ AT THE NIH is convened in order to obtain more rigorous interpretations of mul- tiple sources of evidence. Evidence presented at a consensus development conference may be complex as well as conflicting. Therefore, it is often difficult for panelists to analyze and synthesize the information presented in the short period of time normally provided. Use of data analytic and reduction techniques (e.g., meta-analysis) has resulted in evidence being "well integrated into both the deliberations and the consensus statement" (lacoby, 19881. A critical element of meta-analysis is that the process takes account of the quality of the information, as well as information about the history, setting, types of patients, and other factors that may explain differences across studies. Other issues to be considered in reviewing the evidence include the preci- sion of definition of the outcome being measured, the adequacy of the study methodology and the degree to which it has been de- scribed, the adequacy of sample size, the degree to which the char- acteristics of the population studied and the activity being evaluated have been described, and We degree to which results can be general- ized (Public Health Service, 1989~. Given the subject matter and types of questions posed for a par- ticular topic, more than one meta-analysis may be prepared for a particular consensus development conference. The findings of a meta-analysis conducted on any particular research issue may vary according to such factors as the thoroughness of the literature search for research reports and analysts' decision rules about accepting and grouping reports in a meta-analysis. Meta-analysis alone is unlikely to fully address the major concerns of a consensus conference. It does offer a rigorous means of analyzing and weighing evidence and rendering more explicit areas of uncertainty in available research. OMAR should employ available statistical and related methodol- ogic techniques to synthesize the evidence into an understandable format that will facilitate the assessment of evidence by the panel. This can be accomplished by contracting with consultants who have expertise with these techniques and some working knowledge of the conference topic, or by developing the necessary staff expertise to perform meta-analysis and to work with representatives of a particu- lar BID who are familiar with the topic. A combination of experts familiar with meta-analysis and with the topic area is likely to be most desirable.
RECOMMENDATIONS FOR PROGRAM FUNCTION Evidence should be presented to the pane] in a standardized for- mat and be graded in a manner consistent with its quality. 27 The connection between evidence and recommendations has been unsatisfactory for some consensus conferences (Ahrens, 1985; Perry, 1987~. As a result, some recommendations from the NTH COP may have been premature or inadequately supported. The process of evaluating evidence and deriving recommendations from it should be explicit. As noted above' panelists should consider not only the results and findings of the studies but also the strength of the study design and the validity of the findings when developing consensus. Although there are potential disadvantages as well as advantages3 associated with grading evidence, it can be an excellent means of improving the process of evaluating evidence and deriving welI- founded recommendations. For example, a system of grades of evi- dence similar to the one used by the Canadian Task Force on the Periodic Health Examination and the U.S. Preventive Services Task Force might be appropnate.4 Potential advantages include the following. · The process is explicit for speakers and panelists, reducing the extent to which confusion about the quality of evidence, bias, and vested interest can influence the recommendations. · The process is explicit for members of the health care profession and the public, so that the way evidence was handled can be appreciated and challenged. Those who have different standards can apply them to the same evidence. Those who think that important evidence was not considered can protest or incorporate it themselves. · When new or additional information becomes available, it can be evalu- ated and, if appropriate, incorporated. Potential disadvantages include the following. · Any simple method of grading is necessarily arbitrary. · Additional staff work is required, but this will be offset by the panel being able to direct its energies to the most important evidence. · Debate exists surrounding how the levels of evidence are to be translated into grades of recommendations and how much evidence (if any) should be cited along with recommendations. · The standards for determining the strength of evidence will differ some- what for different technologies (e.g., preventive, therapeutic, and diagnostic). 4The Canadian Task Force on the Periodic Health Examination and the U.S. Preventive Services Task Force have used the following system for grading evi- dence.
28 CONSENSUS DEVELOPAlE=AT THE NIH Presentation of non-peer-reviewed original evidence should be limited to those extraordinary instances in which the conference pane] and planning committee chairs determine that compelling rea- sons exist for its presentation to the panel. Consistent with the preceding recommendation, any such evidence should be clearly gradled. The time available for the convening of panelists in consensus conferences is limited. Staff and panelists therefore should be af- forded ample time before a conference to carefully review and weigh the considerable amount of evidence that may be presented for a consensus conference topic. This process will enable panelists to ascertain and compare the quality of studies and other data. Cer- tainly, peer-reviewed evidence has been subject to a measure of expert examination for quality that most panelists otherwise would be unable to perform in the context of a consensus conference ses- sion. Expert presentations made at the conferences of peer-reviewed studies that have been examined by panelists prior to me conference can focus on key issues of direct relevance to the conference, and panelists will be better prepared to make useful inquiries. The apparent drawback of limiting conference presentations to previously available peer-reviewed evidence is that sufficient time may not have been available for very recent studies to have been I. Evidence obtained from at least one properly randomized, controlled trial. II-1. Evidence obtained from well-desi~ned controlled trials without randomiza- tion. II-2. Evidence obtained from well-designed cohort or case-control analytic stud- ies, preferably from more dean one center or research group. II-3. Evidence obtained from multiple time series studies with or without inter- vention. Dramatic results in uncontrolled experiments (such as the results of die introduction of penicillin in Me 1940s) could also be regarded as this type of evi- dence. III. Opinions of respected authorities, based on clinical experience, descriptive studies, or reports of expert committees (Goldbloom et al., 1989; Harris et al., 1989; Sackett, 1989~. Group III can be further classified into: III-1. a group opinion with formal integrative technique, III-2. a group opinion with informal consensus, and III-3. an individual's opinion. e
RECOMMENDATIONS FOR PROGRAA! FUNCrlON 29 peer reviewed and published. Even so, except under extraordinary circumstances, evidence of unknown quality should not be intro- duced to panelists for the first time in the limited format of a confer- ence, where it would be difficult to ascertain how it affects the balance of evidence already at hand. If compelling reasons exist for introducing such evidence, every effort must be made prior to the conference to subject it to grading and other critical review. Fur- ther, as final consensus statements should reflect the evidentiary basis of their findings, readers win be better prepared to consider whether the findings of evidence peer reviewed after the conference might alter the consensus statements' findings and whether a reevalu- ation of the topic is in order. OMAR's guideline that "speakers should be asked to confine their presentations to the scientific topic that they have agreed to ad- dress" should be strictly enforced by the pane! chair during the conference. This guideline should be expanded to require speakers to focus on the scientific evidence related to the question being addressed, rather than relying on anecdote or opinion. As noted above, a relatively short period of time is available during the conference for presentation of evidence, and a great deal of evidence must be considered by the panel. In order to make efficient use of the available time arid to avoid burdening the pane} with irrelevant evidence, speakers should be limited to appropriate evidence in their presentations. Evidence considered as part of the NTH CDP should be scientifically sound and not based on anecdote · ~ Or oplmon. Development of Consensus and Group Dynamics A working definition of consensus should be made available to, and agreed upon by, panels prior to initiating deliberations and should be included in the consensus statement. There is no standard definition of consensus; that is, it may con- note unanimity, majority rule, or other levels or expressions of agree- ment. A panel should specify an operational definition of consensus (e.g., full agreement, majority agreement, voting, or mean score) to
30 CONSENSUS DEVELOPMENT AT THE NIH ascertain the level of agreement during the deliberations and to re- veal the existence of substantial minority opinions- (Fink et al., 19g4~. The definition of consensus and level of agreement actually achieved should be reflected in the consensus statement to provide an indica- tion of the strength of the recommendations.5 OMAR should experiment with quantitative decision modeling as a means of supporting the work of consensus panels. A panel's charge to reach consensus within the allotted time may be difficult. Although benefits are not yet fully known, OMAR is encouraged to explore the use of different means of consensus de- velopment support in order to improve the NTH CDP and provide guidance for other consensus development efforts. Decision modeling addresses well-focused questions that involve substantial uncertainty in order to "provide a decision maker with an explicit reproducible process that can help structure complex alter- natives in a rational way" (McNeil and Pauker, 1984~. The basic technique involves developing a formal, explicit model of questions under consideration. Such a mode} provides language for represent- ing conflicting data and conflicting objectives and can be explored under a variety of alternative assumptions to determine whether or not the choice of an optimal strategy is dependent on variations in the data base (Pauker, 1986~. Quantitative decision aids and their use of explicit processes to assist in evaluating evidence and deriving recommendations may be of value to panels in achieving consensus. Such tools mat facilitate achieving "mathematical consensus" might provide a more cost-ef- fective approach to consensus development and avoid the risk of inadequately considered conclusions because of the potentially oner- ous night sessions. Implementation of more formal definitions of consensus would be facilitated by the use of these methods. Panels should be provided with information and decision tools to support their efforts. The program should experiment with these to deter- SOne way to indicate the strength of recommendations would be for panels to indicate the certainty of their recommendations qualitatively or quantitatively. The "lowest common denominator" of acceptance is a potential problem for many is- sues, and might be alleviated by providing a better indication of the degree of consensus actually achieved. Readers may want to know, for example, whether there is a near-unanimous opinion, with little variance, or whether the result reflects an averaging of extremely diverse opinions.
RECOMMENDATIONS FOR PROGRAM FUNCTION 31 mine how they can be used most appropriately to serve the needs of consensus development (e.g., decision analysts could be included on the pane} or as speakers, or OMAR could contract for decision analy- sis to be perfonned in support of the conference). However, if quantitative decision aids are to be used, panels must be receptive to their use and careful consideration must be given to how these tools will be incorporated into the process. Further, use of these tools will require considerable interaction between the individuals per- forming the modeling and the pane} members. OMAR should experiment with various resources (e.g., profes- sional facilitator) and structures (e.g., varying ranges of conference duration) to aid the efforts of panels to achieve a well-founded con- sensus. A professional facilitator may assist the pane} chair in managing the consensus development process. The experience of working groups indicates that the opportunity for full participation by all members is desirable and is a function of delicate social balances among the group members. As a fellow professional, the chair of the group is part of the panel's social structure and is in competition with the other members for time and influence. The measure of control given to the chair (i.e., to determine who speaks and when) tends to influence group synergy. An expert facilitator may be able to gain more from the different viewpoints and knowledge bases or skills of the various individuals than could be gained otherwise. Specifically, the facilitator might assist the pane} chair in ensuring participation by all panelists, preventing domination by strong per- venalities on the panel, monitoring and allocating panel discussion time, and keeping the pane} focused on the questions to be answered. The limited time available to consensus panels may raise con- ce~ns about the quality of the resulting consensus statements by creating pressure for the panels to produce statements before they have an adequate opportunity to review the available evidence and reach well-founded conclusions. OMAR should continue to offer a conference duration of between two and a half and four days to the planning committee (Elliott, 1989~. The pane] should take full account of alternative interpretations or recommendations during the consensus development process. The consensus statement should include well-reasoned! minority opinions
32 CONSENSUS DEVELOPME'AT TlIE NIH when they exist, note when such minority opinions do not exist, identify items that lack adequate evidence for a judgment to be made (i.e., explicitly address what is not known about a topic), and iden- tify specific areas for further research. A repeated criticism of the program has been that minority views are submerged or obscured (Perry, 1987~. According to OMAR, there have only been two consensus statements that have included minority opinions. Full consensus on all issues cannot always be achieved, and valuable information can be produced from a confer- ence that does not reach definitive conclusions. Specific techniques should be considered to ensure that We pane} does not override alternative interpretations of evidence or recom- mendations. For example, one pane} member could be assigned to critique the group process at intermediate points or to play the role of "devil's advocate" (lanis, 1971~. Consensus should not be forced or reported if none exists, and consensus statements should indicate the degree of agreement achieved. Approaches for improving group dynamics and development of consensus are being developed and tested in many fields (Dalkey, 1969; Delbecq et al., 197S; TOM, 1985; Linstone and Turoff, 1975; Olsen, 1982; Policy Research Incorporated, 1977; Porter-et al., 1980~. OMAR can take a leadership role by looking within and beyond the health field for alternative techniques and models used to engage groups of experts in rendering well-founded and informed findings. Initial and Final Consensus Statement Preparation The pane! should use the collected! background material for the conference as the starting point of its discussion, rather than a draft consensus statement prepared by pane! members. It is inappropriate to have a draft consensus statement before a conference begins,6 as this may imply prejudgment of an issue mat 6A previous report to NIH recommended that "panelists should be assigned to questions and write first drafts of answers prior to the conference" or that the panel should meet before the conference and "write an advanced draft of the consensus statement" (Wortman and Vinokur, 1982~. The recommendations in that report were intended to reduce the burden on the panel during its executive session, when the consensus statement is written, and to provide the panel with additional time to produce an improved final consensus statement.
RECOMMENDATIONS FOR PROG~ F~70N 33 has not been fully evaluated. Such a statement may limit panel deliberations for or against particular positions rather than allow more open discussion. However, it is appropriate to present infor- mation (e.g., the speakers' positions in writing) to panelists prior to the conference, (see the recommendations in the section Preparation of Speakers, Data Synthesis, and Data Presentation earlier in this chapter). It also may be useful to describe a technology or practice and its uses in an objective background paper prepared for the panel. This paper could provide the basis for a preamble to their consensus statement and the descriptive narrative to which the pane} may refer in developing the consensus statement. Moreover, this information should be available to panelists in electronic (i.e., machine-readable) form. The pane! should be provided with all practical resources to fa- cilitate writing the consensus statement. Specifically, OMAR should proYia~e the best current computer support and audio-visual technol- °BY (e.g., words processing equipment that includes large screens or projection capability during the drafting of consensus statements and should consider employing a writer to support pane! working sessions and to assist the pane! in ensuring that the final consensus statement is presented clearly. The executive session during which the pane} writes the consen- sus statement has been described as '~grueling" (Mullen and Jacoby, 1985~. Given the existing time constraint and the potential diffi- culty of drafting consensus statements, the pane} should be provided with appropriate tools to lessen the burden of Weir work and reduce concern that a deadline forces panels into inadequately considered judgments.
