Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 18
Recommendations for
Program Function
PLANNIN G/PREPARATION
Topic Selection
The topic selection process should generate and select topics that
reflect the priorities of the broader health care system. Specifically,
OMAR should have a standing mechanism for actively soliciting
topic suggestions throughout the health care system on a regular
basis. This solicitation should include, but not be limited to, profes-
sional organizations, specialty societies, third-party payers, health
care providers, government agencies, and the general public. OMAR
should describe the process for generating potential topics to facili-
tate contributions by interested indlividuals and organizations. The
process of selecting topics should be formalized! and include partici-
pation by the aforementioned OMAR advisory council, OMAR stay,
and BID directors.
The selection of a topic is a critical determinant of the value of a
consensus conference. The NTH CDP has been criticized for exam-
ining topics that do not merit the evaluation or expenditure associ-
ated with the program. Thus, it has been suggested that "there
should be a mechanism for setting priorities among the approaches
to be assessed" (Perry, 1987~. The current process for selecting
18
OCR for page 19
RECOMMENDATIONS FOR PROGRAM FUNCTION
19
topics, as described in the OMAR Guidelines (OMAR, NTH, 1988),
relies largely on input from within NTH, without adequate represen-
tation of other segments of the health care system or consideration
of important nonbiomedical issues.
The active solicitation of topic suggestions on a regular basis will
assist OMAR in identifying the concerns of the health care system.
Examples of agencies and organizations that might provide topic
suggestions include, but are not limited to, the Agency for Health
Care Policy and Research, American Academy of Family Practitio-
ners, American College of Physicians, American College of Sur-
geons, American Hospital Association, American Medical Associa-
tion, American Nursing Association, Blue Cross and Blue Shield
Association, Centers for Disease Control, Council of Medical Spe-
cialty Societies and its member organizations, Department of Veter-
ans Affairs, Food and Drug Administration, Health Care Financing
Administration, Health Industry Manufacturers Association, Health
Insurance Association of America, National Institutes of Health, and
Pharmaceutical Manufacturers Association.
A formal process involving an OMAR advisory council, OMAR
staff, and BID directors should be used to set priorities among po-
tential topics. Consistent with the earlier discussion on the relation-
ship of consensus development to changing behavior, the criteria for
topic selection should be augmented to reflect an emphasis on the
potential to improve the effectiveness of clinical practice, as well as
the feasibility of achieving such results. For each potential topic, it
will be useful to identify the specific groups that most need or are
likely to use the consensus findings, the use that such groups might
make of the information, and the approximate magnitude of impact
that the findings are anticipated to have within a specified time
frame.
Evidlence on the state of clinical practice associated! with poten-
tial topics related! to existing technologies or practices should be
analyzed to help OMAR in selecting topics, the planning committee
in drafting questions, and the pane! in developing its statement.
In determining whether a potential CDP topic is controversial and
whether a conference on that topic is likely to be useful, OMAR
should consider the available information about pertinent practice
patterns as well as the current state of the applicable science (Kanouse
OCR for page 20
20
CONSENSUS DEVELOPME=AT THE NIH
et al., 1989~. The extent to which there is evidence about clinical
practice variation, potentially inappropriate utilization, and related
aspects of practice should be taken into account when choosing CDP
topics. Evidence should be examined on the extent to which prac-
tices are used, the circumstances under which they are used, and the
use of any particular relevant techniques (e.g., medical or surgical).
Clinical practice information will help planning committees to es-
tablish the scope of consideration for CDP topics and to target con-
ference questions. Such information may be necessary for determin-
ing whether matters of delivery or access, or related economic, so-
cial, or ethical issues, should be considered. Panels may use clinical
practice evidence for determining, in part, how their recommenda-
tions might be expected to affect clinician behavior (e.g., result in
the reduction, increase, or other modification in the use of a proce-
dure).
OMAR could review recent data collected for other purposes (e.g.,
Medicare claims data, data collected by Peer Review Organizations,
the National Disease and Therapeutic Index, or the National Ambu-
latory Medical Care Survey) (Kanouse et al., 1989~. OMAR should
coordinate this effort with AHCPR, which is to conduct research and
evaluation on the extent to which rates of utilization vary among
similar populations for particular diseases, disorders, and other heady
conditions; uncertainties exist on the effect of utilizing a particular
service or procedure; or inappropriate services and procedures are
provided (U.S. Congress, 1989~.
Some medical practices and technologies have been widely adopted,
despite He lack of reliable information about benefits and risks (Asch
and Lowe, 1984~. Even when evidence is not adequate to support
firm conclusions about safety and efficacy, NTH consensus develop-
ment conferences on management of important clinical problems
may be useful. For example, a consensus development conference
might evaluate an important, frequently used technology for which
evidence of safety and efficacy is inadequate for developing recom-
mendations about its appropriate use. Such a conference could iden-
tify specific areas for further research and would provide a more
tenable basis for health care delivery, payment, and related deci-
sions by informing clinicians, patients, provider organizations, arid
third-party payers that adequate evidence for wet/-grounded recom-
mendations on the technology's use is lacking.
OCR for page 21
RECOMMENDATIONS FOR PROGRAM FUNCTION
2
The evidence criterion for topic selection should be broadened to
include various types of scientifically sound evidence; in addition to
evidence on safety and efficacy, such evidence might include prac-
tice patterns and utilization, access, and the findings of other rele-
vant health services research.
Currendy, for a topic to be selected for consensus review, "the
topic must have an adequately defined and available base of scien-
tific information to answer me previously posed questions and to
resolve the controversies insofar as possible" (OMAR, NTH, 19883.
Given the above recommendations to broaden me scope of the pro-
gram and to consider evidence on the state of clinical practice in
selecting topics, me existing topic selection cnter~on regarding data
needs should be broadened.
Planning Committee Formation
The planning committee members should include members who
understand the clinical, methodologic, and societal issues related to
the conference topic. The planning committee should include repre-
sentatives of relevant BIDs, health care providers (physicians, nurses,
other health care professionals, and delivery organizations), third-
party payers, government agencies, health services researchers, epi-
d~emiologists, biostatisticians, economists, ethicists, patients, and the
general public, as relevant to the topic at hand.
The planning committee for each conference greatly influences
the outcome of a consensus development conference by drafting the
questions that define the scope and substance of the conference, by
nominating panelists who will respond to those questions, and by
nominating the speakers who will present the information on which,
in large part, the panelists win base their findings. Accordingly, the
membership of the planning committee should reflect a balance of
the parties interested in the selected topic in its entirety and possess
a comprehensive understanding of the conference topic (as broadly
defined above). However, achieving broader membership on the
planning committee should not compromise the efficiency of that
group, and OMAR should balance me needs for broad representation
OCR for page 22
22
CONSEI`ISUS DEVELOPMENT AT TlIE NIH
and effective group size when appointing planning committee mem-
bers.
The planning committee chair should be an individual who is not
employed by NIH. The planning committee chair should be selected
in consultation with OMAR staff, relevant BID directors, and the
OMAR advisory council recommended above.
As noted above, the planning committee shapes the consensus
development conference and can greatly influence its outcome. There
is a high degree of NIH staff member participation in the CDP. It is
desirable to complement NIH staff contributions to the process by
assigning the role of planning committee chair to an individual who
can broaden representation from other health care sectors.
Drafting Conference (~'e~tion~
The planning committee should publicly solicit questions concern-
ing a selected topic from a broad base of relevant organizations and
individuals, including OMAR advisory council members. This could
be accomplished through announcements in professional journals,
the Federal Register, and other notices and by contacting profes-
sional, profiler, industry, government, and patient organizations.
Conference questions should be "selected by means of a more
thorough search procedure" (Wortman and Vinokur, 1982~. The
questions posed to a CDP pane} should reflect a broad spectrum of
health care concerns about the conference topic. In order to under-
stand and include the perspectives of the various health care system
participants in the questions, representatives of these participants
should be consulted during the question preparation process.
Persons not involved in the process of drafting conference ques-
tions should review the questions to ensure that they are clearly
stated and will be understood by the consensus panel, conference
participants, and readers of the consensus statement.
PA list of the conference questions from the NIH consensus conferences held
between October 1987 and September 1989 is provided in Appendix C.
OCR for page 23
RECOMMENDATIONS FOR PROGRAM FUNCTION
23
Conference questions should be stated clearly to eliminate ambi-
guity or biased interpretation. Careful and precise definition of the
questions to be addressed is required to investigate the topic eff~-
ciently (Fink et al., 1984), to maximize the opportunity fovea valid
evaluation, and to avoid a predetermined conclusion (Oliver, 1985~.
Poorly worded questions can result in ambiguous or overly general-
ized recommendations Comas, 19X6~.
Speaker Selection
Suggestions for speakers should be solicited from a wide range of
knowledgeable persons in NIT, other government agencies, profes-
sional organizations, and other appropriate groups.
A more thorough search procedure should be used in selecting
speakers (Wortman and Vinokur, 1982~. Presentations by confer-
ence speakers should reflect the breadth of knowledge and diversity
of opinion about a topic. Solicitation of suggestions for conference
speakers will enhance the planning committee's ability to determine
when differences of opinion exist about a topic and to "include the
presentation of opposing data and interpretations" (OMAR, NIH,
19881. Broadening the process for selecting speakers will lessen the
potential for biased selection of speakers (Perry, 1987~.
Consensus Pane' Selection and Composition
The panel chair plays an important role in shaping the outcome of
consensus conferences (Wortman and Vinokur, 1982~. Thus, as
OMAR's guidelines state, the chair should be "knowledgeable and
prestigious" in the field of medical science under consideration, but
should not be "identified with strong advocacy positions regarding
the consensus topics or with research that might be presented"
(OMAR.~ NTH, 1988~. Further, the individual selected should have
strong leadership and group process skills. There is a risk that a
pane} chair may become too influential, particularly in light of the
pane} chair's participation in the planning committee. Panel chair
participation in the planning committee provides continuity of the
planning and implementation of conferences, enhances the pane}
chair's understanding of the process and the specific conference
objectives, and develops a stronger relationship between the chair
OCR for page 24
24
CONSENSUS DEKE:LOPMENTAT THE NIH
and OMAR staff. However, the objectivity of the process may be
compromised by the potential for one individual to exert excessive
control over the outcome of the process.
This committee did not reach agreement on whether the pane}
chair should be a member of the planning committee. Although a
majority of committee members indicated that the benefits of in-
volving the pane} chair in the planning process outweigh me poten-
tial risks, some did not. A number of committee members indicated
that OMAR should explore modified roles of the pane} chair (e.g.,
not having the pane} chair serve on the planning committee or intro-
ducing a professional facilitator to assist the pane} chair, as dis-
cussed below).
Suggestions for panelists should be solicitedF from a broad range
of organizations and individuals interested in the consensus topic.
Solicitation of suggestions for pane} members from both within
and outside of NIH will enhance the planning committee's ability to
achieve "balanced representation from various sectors of professional
and community life" (OMAR, NTH, 1988~. Achieving this represen-
tation will validate the pane} recommendations (Asch and Lowe,
1984~. As noted above, formal solicitation of suggestions for con-
ference participants may help to offset the potential for bias in the
process.
Conference panelists, when possible, should possess overlapping
expertise.
The breadth of panel expertise should be commensurate with the
issues being addressed by a conference. When assembling the con-
ference panel, the planning committee should carefully identify and
distinguish among the various disciplines required to address confer-
ence questions. For example, the related fields of biostatistics and
epidemiology may both need to be represented on a conference panel.
Achieving the necessary breadth of expertise must be accomplished
while maintaining a manageable group size for the pane} (Wortman
and Vinokur, 19X2~. Selection of panelists who possess more than
one area of expertise facilitates the balancing of group size and
group expertise. Further, there are potential disadvantages associ-
ated with representation of a discipline by a single expert. First, the
overall composition of the consensus pane} can influence the degree
OCR for page 25
RECOMMENDATIONS FOR PROGRAM FUNCTION
25
of participation by individual panelists in the consensus develop-
ment process; a single pane] member with a particular expertise
might be isolated during pane} deliberations. Second, sound deci-
sions are best achieved when the panel is not forced to defer to a
single individual on the pane} for crucial expertise. The selection of
panelists with multiple areas of expertise reduces the likelihood of a
pane} containing a solitary expert, without increasing the size of the
panel.
IMPLEMENTATION
Preparation of Speakers, Data Synthesis, and Data Presentation
Speakers should submit, in advance of the conference, brief state-
ments that outline their positions, reasoning, or findings on each
specific question they have been asked to address; the full text of the
key referenced articles on which they base their positions; and, pref-
erably, the full text of their presentations.
Panelists must deal with a large amount of evidence during the
consensus conference. Prior to the conference, panelists should re-
ceive the information to be presented at the conference in a well-
organized format. This will provide more time for them to evaluate
the information and prepare any questions for the conference speak-
ers. Providing these materials to the panelists before the conference
may help to diminish any undue influence of speakers' manners of
presentation or personal characteristics on the panels' evaluation of
the evidence being presented. Further, this information can serve as
reference material to Me pane} during its deliberations.
Whenever possible (i.e., when the nature of available data makes
this feasible), meta-analysis2 should be performed before the pane!
2Meta-analysis is a statistical method for obtaining quantitative findings, e.g., on
die effect size of a medical intervention, from multiple reports of primary studies on
a particular subject. From information obtained from each primary source, a synthe-
sis is made diet may produce a stronger conclusion than that which any of the
separate reports can provide. It is generally most appropriate when there are no
definitive studies on a topic and the nondefinitive studies are in some disagreement
(Bulpitt, 1988; Hunter, 1982; Louis et al., 1985; Pillemer and Light, 1980; Sacks et
al., 1987; Whacker, 1988~.
OCR for page 26
26
CONSENSUS DEVELOPME~ AT THE NIH
is convened in order to obtain more rigorous interpretations of mul-
tiple sources of evidence.
Evidence presented at a consensus development conference may
be complex as well as conflicting. Therefore, it is often difficult for
panelists to analyze and synthesize the information presented in the
short period of time normally provided. Use of data analytic and
reduction techniques (e.g., meta-analysis) has resulted in evidence
being "well integrated into both the deliberations and the consensus
statement" (lacoby, 19881. A critical element of meta-analysis is
that the process takes account of the quality of the information, as
well as information about the history, setting, types of patients, and
other factors that may explain differences across studies. Other
issues to be considered in reviewing the evidence include the preci-
sion of definition of the outcome being measured, the adequacy of
the study methodology and the degree to which it has been de-
scribed, the adequacy of sample size, the degree to which the char-
acteristics of the population studied and the activity being evaluated
have been described, and We degree to which results can be general-
ized (Public Health Service, 1989~.
Given the subject matter and types of questions posed for a par-
ticular topic, more than one meta-analysis may be prepared for a
particular consensus development conference. The findings of a
meta-analysis conducted on any particular research issue may vary
according to such factors as the thoroughness of the literature search
for research reports and analysts' decision rules about accepting and
grouping reports in a meta-analysis. Meta-analysis alone is unlikely
to fully address the major concerns of a consensus conference. It
does offer a rigorous means of analyzing and weighing evidence and
rendering more explicit areas of uncertainty in available research.
OMAR should employ available statistical and related methodol-
ogic techniques to synthesize the evidence into an understandable
format that will facilitate the assessment of evidence by the panel.
This can be accomplished by contracting with consultants who have
expertise with these techniques and some working knowledge of the
conference topic, or by developing the necessary staff expertise to
perform meta-analysis and to work with representatives of a particu-
lar BID who are familiar with the topic. A combination of experts
familiar with meta-analysis and with the topic area is likely to be
most desirable.
OCR for page 27
RECOMMENDATIONS FOR PROGRAM FUNCTION
Evidence should be presented to the pane] in a standardized for-
mat and be graded in a manner consistent with its quality.
27
The connection between evidence and recommendations has been
unsatisfactory for some consensus conferences (Ahrens, 1985; Perry,
1987~. As a result, some recommendations from the NTH COP may
have been premature or inadequately supported. The process of
evaluating evidence and deriving recommendations from it should
be explicit. As noted above' panelists should consider not only the
results and findings of the studies but also the strength of the study
design and the validity of the findings when developing consensus.
Although there are potential disadvantages as well as advantages3
associated with grading evidence, it can be an excellent means of
improving the process of evaluating evidence and deriving welI-
founded recommendations. For example, a system of grades of evi-
dence similar to the one used by the Canadian Task Force on the
Periodic Health Examination and the U.S. Preventive Services Task
Force might be appropnate.4
Potential advantages include the following.
· The process is explicit for speakers and panelists, reducing the extent to
which confusion about the quality of evidence, bias, and vested interest can
influence the recommendations.
· The process is explicit for members of the health care profession and the
public, so that the way evidence was handled can be appreciated and challenged.
Those who have different standards can apply them to the same evidence. Those
who think that important evidence was not considered can protest or incorporate
it themselves.
· When new or additional information becomes available, it can be evalu-
ated and, if appropriate, incorporated.
Potential disadvantages include the following.
· Any simple method of grading is necessarily arbitrary.
· Additional staff work is required, but this will be offset by the panel being
able to direct its energies to the most important evidence.
· Debate exists surrounding how the levels of evidence are to be translated
into grades of recommendations and how much evidence (if any) should be cited
along with recommendations.
· The standards for determining the strength of evidence will differ some-
what for different technologies (e.g., preventive, therapeutic, and diagnostic).
4The Canadian Task Force on the Periodic Health Examination and the U.S.
Preventive Services Task Force have used the following system for grading evi-
dence.
OCR for page 28
28
CONSENSUS DEVELOPAlE=AT THE NIH
Presentation of non-peer-reviewed original evidence should be
limited to those extraordinary instances in which the conference
pane] and planning committee chairs determine that compelling rea-
sons exist for its presentation to the panel. Consistent with the
preceding recommendation, any such evidence should be clearly
gradled.
The time available for the convening of panelists in consensus
conferences is limited. Staff and panelists therefore should be af-
forded ample time before a conference to carefully review and weigh
the considerable amount of evidence that may be presented for a
consensus conference topic. This process will enable panelists to
ascertain and compare the quality of studies and other data. Cer-
tainly, peer-reviewed evidence has been subject to a measure of
expert examination for quality that most panelists otherwise would
be unable to perform in the context of a consensus conference ses-
sion. Expert presentations made at the conferences of peer-reviewed
studies that have been examined by panelists prior to me conference
can focus on key issues of direct relevance to the conference, and
panelists will be better prepared to make useful inquiries.
The apparent drawback of limiting conference presentations to
previously available peer-reviewed evidence is that sufficient time
may not have been available for very recent studies to have been
I. Evidence obtained from at least one properly randomized, controlled trial.
II-1. Evidence obtained from well-desi~ned controlled trials without randomiza-
tion.
II-2. Evidence obtained from well-designed cohort or case-control analytic stud-
ies, preferably from more dean one center or research group.
II-3. Evidence obtained from multiple time series studies with or without inter-
vention. Dramatic results in uncontrolled experiments (such as the results of die
introduction of penicillin in Me 1940s) could also be regarded as this type of evi-
dence.
III. Opinions of respected authorities, based on clinical experience, descriptive
studies, or reports of expert committees (Goldbloom et al., 1989; Harris et al., 1989;
Sackett, 1989~.
Group III can be further classified into:
III-1. a group opinion with formal integrative technique,
III-2. a group opinion with informal consensus, and
III-3. an individual's opinion.
e
OCR for page 29
RECOMMENDATIONS FOR PROGRAA! FUNCrlON
29
peer reviewed and published. Even so, except under extraordinary
circumstances, evidence of unknown quality should not be intro-
duced to panelists for the first time in the limited format of a confer-
ence, where it would be difficult to ascertain how it affects the
balance of evidence already at hand. If compelling reasons exist for
introducing such evidence, every effort must be made prior to the
conference to subject it to grading and other critical review. Fur-
ther, as final consensus statements should reflect the evidentiary
basis of their findings, readers win be better prepared to consider
whether the findings of evidence peer reviewed after the conference
might alter the consensus statements' findings and whether a reevalu-
ation of the topic is in order.
OMAR's guideline that "speakers should be asked to confine their
presentations to the scientific topic that they have agreed to ad-
dress" should be strictly enforced by the pane! chair during the
conference. This guideline should be expanded to require speakers
to focus on the scientific evidence related to the question being
addressed, rather than relying on anecdote or opinion.
As noted above, a relatively short period of time is available
during the conference for presentation of evidence, and a great deal
of evidence must be considered by the panel. In order to make
efficient use of the available time arid to avoid burdening the pane}
with irrelevant evidence, speakers should be limited to appropriate
evidence in their presentations. Evidence considered as part of the
NTH CDP should be scientifically sound and not based on anecdote
· ~
Or oplmon.
Development of Consensus and Group Dynamics
A working definition of consensus should be made available to,
and agreed upon by, panels prior to initiating deliberations and
should be included in the consensus statement.
There is no standard definition of consensus; that is, it may con-
note unanimity, majority rule, or other levels or expressions of agree-
ment. A panel should specify an operational definition of consensus
(e.g., full agreement, majority agreement, voting, or mean score) to
OCR for page 30
30
CONSENSUS DEVELOPMENT AT THE NIH
ascertain the level of agreement during the deliberations and to re-
veal the existence of substantial minority opinions- (Fink et al., 19g4~.
The definition of consensus and level of agreement actually achieved
should be reflected in the consensus statement to provide an indica-
tion of the strength of the recommendations.5
OMAR should experiment with quantitative decision modeling as
a means of supporting the work of consensus panels.
A panel's charge to reach consensus within the allotted time may
be difficult. Although benefits are not yet fully known, OMAR is
encouraged to explore the use of different means of consensus de-
velopment support in order to improve the NTH CDP and provide
guidance for other consensus development efforts.
Decision modeling addresses well-focused questions that involve
substantial uncertainty in order to "provide a decision maker with an
explicit reproducible process that can help structure complex alter-
natives in a rational way" (McNeil and Pauker, 1984~. The basic
technique involves developing a formal, explicit model of questions
under consideration. Such a mode} provides language for represent-
ing conflicting data and conflicting objectives and can be explored
under a variety of alternative assumptions to determine whether or
not the choice of an optimal strategy is dependent on variations in
the data base (Pauker, 1986~.
Quantitative decision aids and their use of explicit processes to
assist in evaluating evidence and deriving recommendations may be
of value to panels in achieving consensus. Such tools mat facilitate
achieving "mathematical consensus" might provide a more cost-ef-
fective approach to consensus development and avoid the risk of
inadequately considered conclusions because of the potentially oner-
ous night sessions. Implementation of more formal definitions of
consensus would be facilitated by the use of these methods. Panels
should be provided with information and decision tools to support
their efforts. The program should experiment with these to deter-
SOne way to indicate the strength of recommendations would be for panels to
indicate the certainty of their recommendations qualitatively or quantitatively. The
"lowest common denominator" of acceptance is a potential problem for many is-
sues, and might be alleviated by providing a better indication of the degree of
consensus actually achieved. Readers may want to know, for example, whether
there is a near-unanimous opinion, with little variance, or whether the result reflects
an averaging of extremely diverse opinions.
OCR for page 31
RECOMMENDATIONS FOR PROGRAM FUNCTION
31
mine how they can be used most appropriately to serve the needs of
consensus development (e.g., decision analysts could be included on
the pane} or as speakers, or OMAR could contract for decision analy-
sis to be perfonned in support of the conference). However, if
quantitative decision aids are to be used, panels must be receptive to
their use and careful consideration must be given to how these tools
will be incorporated into the process. Further, use of these tools
will require considerable interaction between the individuals per-
forming the modeling and the pane} members.
OMAR should experiment with various resources (e.g., profes-
sional facilitator) and structures (e.g., varying ranges of conference
duration) to aid the efforts of panels to achieve a well-founded con-
sensus.
A professional facilitator may assist the pane} chair in managing
the consensus development process. The experience of working
groups indicates that the opportunity for full participation by all
members is desirable and is a function of delicate social balances
among the group members. As a fellow professional, the chair of
the group is part of the panel's social structure and is in competition
with the other members for time and influence. The measure of
control given to the chair (i.e., to determine who speaks and when)
tends to influence group synergy. An expert facilitator may be able
to gain more from the different viewpoints and knowledge bases or
skills of the various individuals than could be gained otherwise.
Specifically, the facilitator might assist the pane} chair in ensuring
participation by all panelists, preventing domination by strong per-
venalities on the panel, monitoring and allocating panel discussion
time, and keeping the pane} focused on the questions to be answered.
The limited time available to consensus panels may raise con-
ce~ns about the quality of the resulting consensus statements by
creating pressure for the panels to produce statements before they
have an adequate opportunity to review the available evidence and
reach well-founded conclusions. OMAR should continue to offer a
conference duration of between two and a half and four days to the
planning committee (Elliott, 1989~.
The pane] should take full account of alternative interpretations
or recommendations during the consensus development process. The
consensus statement should include well-reasoned! minority opinions
OCR for page 32
32
CONSENSUS DEVELOPME'AT TlIE NIH
when they exist, note when such minority opinions do not exist,
identify items that lack adequate evidence for a judgment to be made
(i.e., explicitly address what is not known about a topic), and iden-
tify specific areas for further research.
A repeated criticism of the program has been that minority views
are submerged or obscured (Perry, 1987~. According to OMAR,
there have only been two consensus statements that have included
minority opinions. Full consensus on all issues cannot always be
achieved, and valuable information can be produced from a confer-
ence that does not reach definitive conclusions.
Specific techniques should be considered to ensure that We pane}
does not override alternative interpretations of evidence or recom-
mendations. For example, one pane} member could be assigned to
critique the group process at intermediate points or to play the role
of "devil's advocate" (lanis, 1971~. Consensus should not be forced
or reported if none exists, and consensus statements should indicate
the degree of agreement achieved.
Approaches for improving group dynamics and development of
consensus are being developed and tested in many fields (Dalkey,
1969; Delbecq et al., 197S; TOM, 1985; Linstone and Turoff, 1975;
Olsen, 1982; Policy Research Incorporated, 1977; Porter-et al., 1980~.
OMAR can take a leadership role by looking within and beyond the
health field for alternative techniques and models used to engage
groups of experts in rendering well-founded and informed findings.
Initial and Final Consensus Statement Preparation
The pane! should use the collected! background material for the
conference as the starting point of its discussion, rather than a draft
consensus statement prepared by pane! members.
It is inappropriate to have a draft consensus statement before a
conference begins,6 as this may imply prejudgment of an issue mat
6A previous report to NIH recommended that "panelists should be assigned to
questions and write first drafts of answers prior to the conference" or that the panel
should meet before the conference and "write an advanced draft of the consensus
statement" (Wortman and Vinokur, 1982~. The recommendations in that report
were intended to reduce the burden on the panel during its executive session, when
the consensus statement is written, and to provide the panel with additional time to
produce an improved final consensus statement.
OCR for page 33
RECOMMENDATIONS FOR PROG~ F~70N
33
has not been fully evaluated. Such a statement may limit panel
deliberations for or against particular positions rather than allow
more open discussion. However, it is appropriate to present infor-
mation (e.g., the speakers' positions in writing) to panelists prior to
the conference, (see the recommendations in the section Preparation
of Speakers, Data Synthesis, and Data Presentation earlier in this
chapter). It also may be useful to describe a technology or practice
and its uses in an objective background paper prepared for the panel.
This paper could provide the basis for a preamble to their consensus
statement and the descriptive narrative to which the pane} may refer
in developing the consensus statement. Moreover, this information
should be available to panelists in electronic (i.e., machine-readable)
form.
The pane! should be provided with all practical resources to fa-
cilitate writing the consensus statement. Specifically, OMAR should
proYia~e the best current computer support and audio-visual technol-
°BY (e.g., words processing equipment that includes large screens or
projection capability during the drafting of consensus statements
and should consider employing a writer to support pane! working
sessions and to assist the pane! in ensuring that the final consensus
statement is presented clearly.
The executive session during which the pane} writes the consen-
sus statement has been described as '~grueling" (Mullen and Jacoby,
1985~. Given the existing time constraint and the potential diffi-
culty of drafting consensus statements, the pane} should be provided
with appropriate tools to lessen the burden of Weir work and reduce
concern that a deadline forces panels into inadequately considered
judgments.
Representative terms from entire chapter:
consensus development
