|
Items in cart [0] |
Questions? Call 888-624-8373 |
PAPERBACK PDF BOOK PDF CHAPTERS Free Resources |
||||||||||||||||
|
||||||||||||||||
|
|
|||||||||||||||
| ||||||||||||||||||||||||||||||
|
||||||||||||||||||||||||||||||
| Copyright © 2009. National Academy of Sciences. All rights reserved. Terms of Use and Privacy Statement |
Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 133
6
Implementing the New Approach
r ~
he previous chapters call on or-
ganizations that engage in risk analysis and characterization to do things
they do not routinely do: combine analysis with deliberation, broaden
the range of outcomes potentially subject to analysis, and broaden partici-
pation in activities that were previously restricted to analytic experts and
a few decision makers. It may seem to some readers that implementing
such an approach would be quite impractical because it would require a
major increase in the effort made to characterize risks at a time when the
responsible organizations are already overloaded and resources are stable
or shrinking. We believe, however, that when the effort is appropriately
scaled to match the needs of the decision at hand, it does not necessarily
require more time and money and that when it does, the potential ben-
efits are likely to outweigh the costs. This chapter discusses the issue of
practicality as it affects our approach to risk characterization. It then
discusses two keys to making our approach practical: diagnosing risk
decision situations in order to match the process to the needs of the situ-
ation and building the capability for implementation.
PRACTICALITY
There are legitimate concerns about the practicality of the analytic-
deliberative approach to risk characterization presented in Chapters 3, 4,
and 5. One concern is with the costs and benefits of expanding the con-
cept of risk analysis to include ecological and social outcomes as well as
133
OCR for page 134
134 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMO CRITIC SOCIETY
threats to human health and safety. An expanded domain for risk analy-
sis might require the responsible organizations to hire new kinds of ex-
perts and to support new and expensive analyses, some of them relying
on new techniques. It is reasonable to ask whether such additional activ-
ity can lead to better or more acceptable decisions and whether in a con-
text of restricted budgets, it might displace other, more essential, efforts.
Another concern is that bringing interested and affected parties into sci-
entific and technical discussions, such as about which analytic technique
to use, might introduce delay and confusion and allow nonscientific con-
cerns to impinge improperly on scientific decisions. A third concern is
that adding participants and increasing the number of issues that must be
considered can provide many opportunities for any interested or affected
party that might benefit from delaying a decision to find excuses for
delay.
Such adverse outcomes might indeed result from adopting our ap-
proach to risk characterization, and it is certainly true that expense, delay,
and confusion already plague risk characterizations. On balance, how-
ever, we believe the approach we propose is more likely to mitigate these
outcomes than cause them, especially when applied to major decisions
with potentially wide impacts.
Consider, for example, the potential for expensive and time-consum-
ing debate about the adequacy of risk analyses. Analyzing additional
dimensions of risk may seem to invite additional debate, but experience
shows that extended and unproductive debates have been prompted by
omissions in existing analyses. In large and complex decision exercises,
risk characterizations that do not consider ecological, social, or human
health outcomes that are important to some of the interested and affected
parties or that are based on a process that excludes key parties can lead to
court challenges and other activities that question the technical adequacy
of the analyses, when the actual concern is the process or risks that were
never analyzed. We believe this pattern has been a major cause of delay
in decisions about high-level radioactive waste disposal, siting waste
incinerators, and other intensely controversial risk decisions. A delibera-
tive process that ensures that the decision-relevant risk analyses are per-
formed the first time may reduce delay in such cases, rather than increas-
ing it. If a decision requires additional analyses to meet the major concerns
of important parties to the decision, the short-term expense of obtaining
the additional expertise may actually be an investment in longer term
savings of time and money.
In some cases, there may not even be additional expense. Delibera-
tions in advance of risk analysis may reduce the immediate costs of analy-
sis or increase its cost-efficiency by directing limited resources for analy-
sis to the most decision-relevant issues. For instance, incremental efforts
OCR for page 135
IMPLEMENTING THENEWAPPROACH
135
to reduce uncertainty in risk analyses could be directed toward those
uncertainties whose reduction might change some of the parties' under-
standings enough to affect their judgments about what should be done.
We believe that an old carpenter's adage applies to risk analysis and
characterization: measure twice, cut once; measure once, cut twice. Experi-
ence shows that analyses, no matter how thorough, that do not address
the decision-relevant questions and use reasonable assumptions can re-
sult in long and detailed criticism, great expense and delay in responding
to the criticism, and even rejection of a risk decision in court. These
hidden costs are especially likely to arise when decisions involve very big
stakes and major controversy. When the entire decision-making process
is taken into account, not only the costs of analysis, it often costs less to get
it right the first time.
It is reasonable to ask whether broadening deliberation, especially on
issues that have a strong technical component, such as selecting assump-
tions for risk analysis, will cause confusion and delay, particularly if the
added participants do not understand the technical issues at stake or the
language of the technical experts. This potential certainly exists, and
avoiding it imposes costs. To involve nonexpert participants meaning-
fully, efforts must be made to educate them technically or to find indi-
viduals who understand the technical issues and can represent the par-
ties' knowledge, perspectives, and concerns in a way that satisfies those
parties. The problem of meaningful participation and the costs of achiev-
ing it are most serious with parties that have not been well organized or
that lack resources to identify or hire their own experts. However, leav-
ing those parties out of meaningful deliberations that affect the risk char-
acterization has its own dangers to the quality of understanding and to
the acceptability of the ultimate decisions, as noted in Chapters 2 and 3.
These dangers are sufficient in our judgment to warrant experimental
efforts to provide resources to allow meaningful participation for parties
that could not otherwise join effectively in deliberations. Such experi-
mental efforts should be focused first on risk decisions that may seriously
affect the parties in question. They should be designed in consultation
with the parties to be assisted and carefully evaluated, with the collabora-
tion of those parties, as to both process and outcome.
Another concern about broadening analysis and deliberation comes
from a recurring problem with risk decisions in the United States. Some
parties use repeated requests for broader analysis or further deliberation
as a tactic to delay a decision or to advance their interests as they could
not in the decision process as originally organized. In some cases, a risk
characterization process that encourages broader analysis and delibera-
tion will invite such tactics. Yet sometimes those requests for delay are
tactical reactions to procedures that excluded some of the parties' chief
OCR for page 136
136 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY
concerns from consideration on an a priori basis. Delays of this type
would become less frequent under a broader concept of risk characteriza-
tion. Clearly, procedural safeguards are needed to reduce inappropriate
or avoidable delays. We encourage organizations to experiment with
different approaches to dealing with the problem of strategic delay. We
note, however, that in implementing the broader concept of risk charac-
terization, especially in decisions with wide impact, more of the effort of
risk analysis and characterization will be focused on substantive ques-
tions about risk and less on the adequacy of procedures. Furthermore,
broadly based deliberation offers a more efficient forum than the courts
for arriving at socially acceptable judgments about whether a request to
extend analysis or debate is necessary to the overall decision process.
We emphasize that our proposed approach adds to analysis, delibera-
tion, and participation only as appropriate to specific situations. We do
not propose wholesale analysis of every possible outcome, deliberation of
every analytical issue, or involvement of all interested and affected par-
ties in all steps leading to a risk characterization. Rather, we advocate
that these possibilities be uniformly considered and that as many addi-
tional activities included as is appropriate for the situation. For some of
the most contentious risk decisions, our approach indeed calls for very
extensive analysis, deliberation, and participation much more than is
envisioned in a linear approach that would only allow deliberation after a
risk analysis is supposedly complete. However, for most such decisions-
such as about high-level radioactive waste disposal, dioxin, and the like-
the linear approach has not produced the efficient process that was hoped
for, even after huge investments in risk analysis and characterization. For
the vast majority of risk decisions, our approach calls for much less than
the most extensive possible analytic-deliberative process, and may not
add much, if anything, to the current level of effort. We note, though, that
the design of simple, generic risk characterization procedures should pre-
sumptively involve the spectrum of interested and affected parties and
that established processes of this type should be periodically reviewed.
A final concern about the practicality of our approach is that forces
external to the analytic-deliberative process may sometimes preclude its
use. There may be organized political opposition to allowing some parties
a voice in risk analysis or to considering certain kinds of concerns in a risk
characterization. This sort of concern is highlighted by the California
Comparative Risk Project and the regulatory negotiation on disinfectant
by-products (described in Appendix A). In both cases, the conclusions of
an analytic-deliberative process with many of the features of our approach
was overturned in the larger political system. In California, the argument
was that considering certain kinds of risks to human welfare in risk rank-
ing would be unscientific. It is normal in a democracy for parties dissat
OCR for page 137
IMPLEMENTING THE NEWAPPROACH
137
isfied with a decision to seek redress elsewhere in the system. However,
when extensive efforts are made to involve the full range of interested
and affected parties in a deliberative process and individual interests
bypass the process, it is destructive of the search for deliberative solu-
tions. It may be necessary or valuable at times for the larger political
system to legitimate analytic-deliberative processes and thus make it more
costly for interests to bypass them. We have not considered the advan-
tages and disadvantages of different ways to legitimate these processes.
Among those that might be considered are declaring in advance that the
results of certain analytic-deliberative processes will be legally binding
on government agencies and establishing analytic-deliberative processes
or forums that would continue to participate in governance functions,
advising on the final decision, monitoring its implementation and its ef-
fects on the things at risk, and recommending changes in policy as appro-
priate. The latter approach has sometimes been used with local risk deci-
sions about such matters as landfills.
DIAGNOSIS: MATCHING THE PROCESS TO THE DECISION
A key to implementing our approach is to match the analytic-delib-
erative process to the needs of the risk decision. Doing this can be diffi-
cult, and not enough is known to justify any standard procedure for
matching. In this chapter, we offer some guidance on how to make that
match.
We find it useful to rely on an idea most often associated with medi-
cine, that of diagnosis. A government agency or other organization re-
sponsible for risk characterization begins with a diagnosis of the potential
hazard situation that is sometimes explicit (e.g., risk defined by law) and
sometimes implicit. Diagnosis includes, at minimum, ideas about the
nature of the hazard and the hazard situation, the purposes for which the
risk characterization will be used, the kinds of information that will prob-
ably be needed, and the kind of decision to be made. Diagnosis is typi-
cally implicit when an agency applies an existing routine, presuming it
adequate for the situation at hand. At other times, important elements of
the diagnosis are implicit, such as assumptions about which parties are
affected or which threats of harm deserve analysis. We recommend that
diagnosis be conducted explicitly far more often than is the current prac-
tice. Although a single organization may have responsibility for risk
characterization, diagnosis generally benefits from interactions of its staff
with scientists, policy makers, and interested and affected parties.
Diagnosis results in a provisional procedure for each step of the ana-
lytic-deliberative process leading to a risk characterization. These provi-
sional choices should be reconsidered, with input from the interested and
OCR for page 138
138 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMO CRITIC SOCIETY
affected parties, during the entire process. Because the best approach to a
risk characterization depends on the specific situation, the first element of
diagnosis is surveying what we call the risk decision landscape.
The Risk Decision Landscape
Risk decisions vary along many dimensions. Although it might be
desirable to reduce these to a few, as has been done with the qualitative
aspects of hazards (see Figure 2-5, p. 63), analogous systematic research
has not been done on risk decision situations. In our judgment, there is no
simple set of categories that can be confidently used to reduce the great
variety of risk decisions to a few types for the purpose of designing a few
standard approaches to informing the decisions. A similar conclusion has
been reached by Graham et al. (1988~. We believe, however, that it is
useful to consider a number of diagnostic questions before embarking on
the processes that lead to risk characterizations. We list some key ques-
tions below.
A good diagnosis can help practitioners narrow the range of appro-
priate courses of action and thus increase the chances of a rational and
socially acceptable outcome. For example, past experiences with decision
situations similar to the one at hand can provide some guidance on proce-
dures that may work well in the new situation. A good diagnosis can
make it easier to consider how a risk decision problem differs from those
made in the past and whether existing decision routines should be
changed.
Despite our caution about the feasibility of classifying risk decision
situations for the purpose of creating a small number of decision-making
routines, we think it is useful to keep in mind the following five categories
that occur with some frequency on the risk decision landscape. The first
category, unique and wide-impact decisions, encompasses those that are most
well known and most often controversial because they are one-time ac-
tions that affect a large portion of the country, a large number of people,
or have effects for a very long time. At the other end of the spectrum is
the category of routine and narrow-impact decisions, which are usually very
similar to previous decisions and involve a small geographic area, few
people, or primarily short-term effects. Somewhere in the middle of this
spectrum is a category that we term repeated, wide-impact decisions: like
unique and wide-impact decisions, they may have major effects over a
large area or large numbers of people, but like routine and narrow-impact
decisions they are very similar to previous decisions so that it is relatively
easy to anticipate the issues they will raise. Our last two categories do not
directly involve decisions about a specific potential hazard: generic hazard
characterizations and decisions about policiesfor risk analysis. Because these
OCR for page 139
IMPLEMENTING THE NEW APPROACH
139
two categories of activities are not designed to result in a specific decision
about a specific hazard, they may appear to be outside our framework for
an analytic-deliberal:ive process; however, we believe that they, too, can
often benefit from this process, beginning with diagnosis.
Unique and Wide-Impact Decisions
Unique and wide-impact decisions are one-time decisions of national
or even wider import that usually involve many kinds of interested and
affected parties and disparate perspectives on what is at risk. The para-
digmatic case is that of decisions associated with the Yucca Mountain site
for a permanent national repository for high-level radioactive waste. Be-
cause of the size of the stakes in the ultimate decision in such an instance,
risk characterization often needs to be based on extensive analysis and
deliberation with broad participation or representation of the spectrum of
interested and affected parties at every step of the process. Such decisions
present special challenges in planning and in carrying out an effective
analytic-deliberative process. The process considers risks on a large so-
cial and geographic scale, and it often extends over considerable time. It
may expand in ways (other topics, across distance) that were not foreseen
at the start. Because of the importance of the decisions, considerable
resources and time may be available to meet these challenges, so it may be
easier to meet the needs for breadth, inclusion, and attention to process
than when less weighty issues are at stake.
Routine and Narrow-Impact Decisions
In contrast, some risk characterization procedures support thousands
of routine and narrow-impact decisions each year. These may include
decisions to issue permits to release small amounts of effluents into air
and water, to approve building designs as adequately earthquake resis-
tant, to accept individuals as blood donors, and so forth. Although there
may be significant unresolved scientific issues underlying individual de-
cisions, an extended analytic-deliberative process for informing each one
would be impractical and might not always serve the overall public inter-
est. For this reason, it is reasonable to routinize the associated risk charac-
terizations.
When establishing routines, it may be useful to consider using a
broad-based analytic-deliberative process to devise a general procedure
that would be used for the individual decisions and then provide for an
appeals procedure for individual decisions as well as periodic public re-
view of the general process. The periodic review should probably in-
volve an analytic-deliberative process roughly as broadly based as the
OCR for page 140
140 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIETY
initial effort to create the routine. Periodic review may be useful for a
variety of reasons. Such review may be legally mandated; there may be a
change in management followed by calls to reconsider certain decisions;
occasionally, scientific breakthroughs may indicate a better scientific ba-
sis for future decisions; or there may be public calls for change resulting
from a process failure. An organization may also desire to change its
procedures because of concerns about representation in the process, inef-
ficiencies, or shrinking resources. Because of the potential for loss of
trust, the responsible organizations should consider planning for regular
review of the risk characterization routines used for informing major
classes of routine, quick decisions.
Many of the considerations that apply to setting up and conducting
the analytic-deliberative process leading to risk characterizations for
unique, wide-impact decisions apply also to reviews of routines for risk
analysis or characterization. There are a series of obvious questions to ask
in evaluating routine decision-making processes:
· Input and access: Do some parties have considerable input to deci-
sions while others have little or none? Are there sufficient resources for
adequate participation? Is the process closed because of claims regarding
confidentiality? If so, are there mechanisms for review?
· Decision quality: Are complaints regarding decisions justified? Are
there criteria for identifying bad decisions in the absence of complaints?
· Efficiency: Are there mechanisms for staff to identify and easily
point out inefficiencies in the process? Are some decisions quickly made
that should require more review and deliberation? Are some decisions
too slow?
· Trust and satisfaction: Do some of the interested and affected par-
ties express distrust about the process? Do some of the parties claim that
the decisions are arbitrary or unfair?
· Review: When was the last time the process was evaluated or re-
viewed? Are there established processes for evaluation or review? Is
there an appeals process? Is the review process broadly accepted?
· Resources: Is the process too costly? Does it have sufficient re-
sources? If not, are additional resources available?
Repeated, Wide-Impact Decisions
Repeated, wide-impact decisions are those that will be the subject of
widespread attention because of their possible effects but are sufficiently
similar to other decisions that some routinization seems possible. Typical
of these are decisions about approving siting and operating permits for
power plants and hazardous waste facilities and, in recent years, about
OCR for page 141
IMPLEMENTING THE NEW APPROACH
14
strategies for restoring ecosystems. The analyses for them are often local
or regional in scope and so may present fewer logistical limitations than
national cases since the interested and affected parties may have easier
access to the responsible organizations.
These decisions have aspects in common with the previous two cat-
egories of decision. They often present the potential to routinize aspects
of the analytic-deliberative process, yet they have wide impacts and some-
times high stakes. The responsible organizations should be alert to the
need, especially when the likely impact of a decision or the potential for
controversy is great, to design or modify aspects of the process to suit
unique needs of the particular decision. As with routine decisions, any
standard procedures should be periodically reviewed; as with unique,
wide-impact decisions, it is important to consider instituting broadly
based deliberative mechanisms in one or more of the tasks leading to a
risk characterization.
Generic Hazard and Dose-Response Characterizations
Generic hazard and dose-response characterizations are not designed
to inform any particular decision, but to serve as inputs for a class of
decisions. Examples include efforts to describe the health risks of dioxin,
the impacts of climate change on human and ecological health and the
economy, and the likelihood of airborne transmission of a particular dis-
ease. These efforts are abstracted from the context of any particular deci-
sion about a specific situation in which exposures take place, but they can
have far-reaching impacts and therefore deserve careful attention, similar
to that accorded risk characterizations associated with unique decisions.
A special problem with these risk characterizations is that absent a par-
ticular decision, it may be more difficult to identify the interested and
affected parties in advance and hence to arrive at the most appropriate
formulation of the problem. Typically, people in the institutions that
would clearly be affected (e.g., producers of a chemical) and those con-
cerned about the precedents that may be set or changed by these types of
decisions can be relied upon to participate, but representatives of the
more general public or environmental or community groups cannot, per-
haps because of lack of resources or limited expertise. Ways to broaden
participation in these exercises should be explored. Decisions about set-
ting standards, for instance, for exposure to a toxin or for the performance
of a piece of equipment, raise many of the same issues as generic hazard
characterizations (Fischhoff, 1984~.
1
OCR for page 142
142 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY
Decisions about Policies for Risk Analysis
Decisions about policies for risk analysis are procedural or method-
ological in nature, such as decisions about which dose-response model to
use in toxicological analysis, whether to routinely consider psychological
impacts in risk analyses, how to gather information about previously
unstudied outcomes, and so forth. These have already received consider-
able attention from an analytic standpoint from the National Research
Council (1994a). These decisions are like generic hazard characterizations
in that they are not tied to a specific risk situations but may affect many
substantive decisions. Consequently, it may be difficult for many of the
interested and affected parties to recognize their importance and to mobi-
lize resources to participate in the analytic-deliberative process. The re-
sponsible organizations may need to make special efforts to identify and
involve the parties and to ensure broad and balanced participation.
Diagnostic Steps and Questions
To diagnose a risk decision situation, we offer eight steps. Depending
on the risk decision, the effort involved may be very brief or rather exten-
sive. This section is written primarily with government agencies in mind,
but we believe it will also be useful to other organizations responsible for
analyzing and characterizing risks. We do not intend to create a new
bureaucratic procedure, but instead to reduce wasted effort through ad-
vance thought and planning. Figure 6-1 shows the diagnostic steps in-
volved in preparing to conduct an analytic-deliberative process to inform
a risk decision. It represents the fact that the steps are not necessarily
sequential all of them flow into the diagnosis and thus inform prelimi-
nary choices about how to execute the process.
1. Diagnose the Kind of Risk and the State of Knowledge
Every analytic-deliberative process sets boundaries as it begins to
consider the risk problem, define options, and examine consequences.
Considering these boundaries explicitly and systematically from the start
has the advantage of identifying mismatches between the boundaries that
the responsible organization is initially inclined to set and the demands of
the situation.
This phase of diagnosis begins with asking basic questions about the
hazard; see box on page 144~. Answers to the questions about who or
what is exposed have implications for who should participate in the ana-
lytic-deliberative process, including the possible need to find ways to
include the perspectives of parties that cannot speak for themselves (e.g.,
OCR for page 143
IMPLEMENTING THE NEW APPROACH
S. Summarize and discuss
diagnosis within
the organization
1 1
6. Plan for
organizational
needs
| 1. Diagnose the kid;
I of risk and state of knowledge
preliminary process .(~ DIAGNOSIS ). l
design
5. Estimate
resource needs
and timetable
FIGURE 6-1 Diagnostic steps for risk decision making.
143
2. Describe the
legal mandate
3. Describe the purpose
of the risk decision
4. Describe the affected
parties and likely public reactions
infants, future generations) or that lack sufficient expertise to be effective
participants. These answers may reveal highly exposed or susceptible
populations or suggest that those at greatest risk are not identifiable. If
risks appear on initial consideration to be inequitably distributed as a
function of race, gender, socioeconomic status, or other factors, the diag-
nostic effort should lead to a conclusion that the analytic-deliberative
process specifically address these issues and that the potentially affected
parties participate to ensure that the process is carried out in a way they
find competent and credible.
Answers to diagnostic questions about the nature of the harm should
reveal the kinds of human health effects and ecological impacts that may
need to be characterized and the various other kinds of possible adverse
outcomes that technical experts or interested and affected parties con-
sider important. The test for which harms to consider in the analytic-
deliberative process is a practical one: Which harms must a characteriza-
tion address for it to be accepted as sufficiently thorough
Answers to questions about the qualitative features of the hazard
should help in anticipating demands from potentially affected parties for
detailed analysis and information, as well as for opportunities to partici-
pate. Hazards that are high on dimensions associated with dread and
lack of knowledge (the upper-right quadrant in Figure 2-5) are especially
likely to generate such demands and to have ripple effects, partly because
one such event may signal or portend future, perhaps catastrophic mis
OCR for page 144
144 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY
~ Giagnost'~ Ouest'0~s About C
_
.~ ~ ~ ~ ~ ~
.~ ~ ~ ~ ~ ~ ~ ~ .
. ~ ~ ~ ~ ~ ~ ~ ~ ~ . . . . . . . . .. .
~ ~ ~ ~ - 0 at. ~..7 8.~.. Ace. I.... l~ 7
.~mS9. .... the. .. or ~.~.~ ben~t f~m
.~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ i, ~ it, . ~ ~ ~,~ i.. ~ ~ ~.~,~.~ ., ~...~. ~ ~ ~ ~.,~ . , aid,., ., ,~ ~ ~ ~ ,~ ., ~.~ ~ ~ ~} ~ , ~
thm 6 ~ ~ ~ tt b! ~ Ad
~Ixble~} ~ h hI
.. ~.~.~.~.~.~.~.~,~.~.~. ~. ~. I. ~., ~,, ~. I. , ~. ~.~. ~it if. , , ,. ~
I ~= Eng'nee~p~ses~tmay~l~cos
~ ~ ~.~.~.~.~,. ~.~.,~ ,.,. aid,. ~ ., ., ,,. ., ~, ~.~,~,~, ~.~, ~. ~
.~ ~m ~ ~9 ~ ~ ~ ~ x000~"~.'bOh8~'''...~'.''C'0~ - '''I''
~ ~ ~ ~ ~ ~ . ~ ~,~ if. ~ ~ ~ ~ , . ~. ~.,~ ~.~.~.~ ~.~. ~, ~ ~ . if ~ ~ ~ I,.,. . , ~ ~,~ ~,~ if. . ~.~ aid , ,
~ ~ ~ ... ~ . I. ~. ~. ~ . if if. ~ of ....... if. ~ ~ . ~ ~ ~ ~ If. ~.~.~. ~ if. ~ I ~ ~ ~ I if.
~ ~ ''I' ~"~ ~ ~ A'' 'Meg I'd ~ ~ ~ 'ECo'togImI 'I ~ ~ ~dd " I'm ~ ~ ~ ~ ~
~ de - of:: Mo ~
:: ~ ~:~:~ a: :: ~ If: ~ ::: :: ~ ::~ ::~ ~ ~ ~ :~ ~ ~
~ ~ ~ {x ~9~L f ~9~
::::::::: . $ . ::::::::::::::
: ::::: :::::::: ~:'~:~:~'~.:~.~. ~'~'~'~'~ ~ ~'~'~'~ ~''''.''~''~.'~''~''::: ':'' . ' .'.' ~ ~ 'I I'm :''':.. ~ ~ . :''''''' ..~: ~.''~'~.': O'er ~ ' .~ .' . ~ ~
.,',, ~,~,~ ,, ~,~,~M',,~e,,,,~m, ,~t,.,.~t,,~'~,.~t'.'.~'~'~ ?: ~'~'"'~2~'~'~'~'~'.'~
::::~::::::::::::::::: ::: :: :::: :: :::: :::: :::::: :::: ::::::::: :.: :::: :. ::.:: ::.::: ::: ::: ::: :.:: :::::. :::::: ::,:: ::::::::: : : ::::: :,::::: : . :: ::::: ::
:~ .~:~:~,:~.:~,~,~ .... : : , ~ : ~ : :. ~.' ~. ~ ,: :. .~ if. ~,~.~ ,::' . :.,~' ~.~ ~'~.'~ I: :.,' ..~ if'
061~ VOlU~S Ot =~um ~ m~ ~mD 1~c A! 3~00 x
~,~,~.~,~,~,~,~l,06.~,,8~.,~e hi,,.,, Debt ,. X6' X' ,,~..,! XiX' ,,~..~b .,: V~,,~.., ~.X..~,~,~,~,,,~
~'~'~'~'~'~.~'~'~'~'~ '~''.'~'."'.'~,''~'~'~,~,~''~'.,'~,~'~'~'.'~'~',','~'.'.',~'~'~'~'",~,' ~'~'~'~'., ~ ,, ':. 'I
. ~ ~ ~ ~ ~ ~ ~ ' ~ ~ ~ ' ~ ' ~ ~ ~ ~ ~ ~ ~ ' ~ ~ ~ ~ ~ ~ ' ~ ~ ~ , . . ' ~ . ' ' ~ ' ' ~ , ~ , ~ " ~ ~ , , ~ ~ . , ' ~ ~ ~ ' ~ ' ~ ' ~ ' ' ~ ~ ~ ' ' " " " ' ' ~ ' ~ ' ~ ' " ' " " ' ~ ~ " ' " " ' ~ ~ ~ ~ ~ ~ ~ ~ ' ~ " . ' ' " " ~ ~ ~ ~ ' ~ ~ ~ ' . " ' ~ ~ . ~ ~ ~ ~ ~ ~ . , , ~ , . . ~ ~ ~ ~ ~ ~ ~ . . , . , , , . ~ , .
1u'd'9m'e"'^ ab"~t Who i'S ''m2S''"""" ' ' " ' ' ' ' ' ' ' ' " ' ' ' ~
:.,. ,,' ' ' ~ ~ ~ ,~ ~ ~ ~ ~ ~ ~ ~ ~ , ~ ,,, .,,,. :.
~.~,~ .', ".:X,'',' ~.'.~,~'~ .~ ~.~ ~,~,~ ~.~,~,~ ~ ~:~,~,~,~ ~ ,~,~,~, ~
~ ~ ~e ~ _~2 ~y ,~ UniqUe 'detents *9 ~y 'and
'~'~,~"":~'~:'.. 'I."". ~ ,','".'.'.':. ".''~::"'.''~' ~'~'~:,"'~':',' i" ,::''" "~"','~':',''~"',:~"""":~::'~'~ ,'' ,"":'::: :,''"".,' ,'~'~'~',"":::,.'.~". "'I ': ":"":"':"':::"'"'. :":"
"'~''~' '','',"'.'',' ' 'a' "' ' 9~.,'~'~'R' ': ' ' ' I,) ' '.' ' ' ."" I' ' ' ' 'I' 9"""',".,~.'.'~'~"'~'~'~.~,.~"~'~,'',~"'.~.'.~'~'~"'.""~,~'~"".~.'~'~.~."~",~"'~,"~":".,
'-' ~'"'"~"'a'2''n"~"~"'''0""''''""''' ~ 'a' "'''''' A'"' "' '''''' '' '
' ~'~'~'~'~"'"~""'~'~""""''''""""X"~' "."""'""'. '"""""''~""'""'""""~'~'~'~""'".'.""""''"'''"'''"'~'.~""''."''"'"'',""'~'~',~".'"' i':"""",' ~'."'.~" "'
.~ ~.~ ~.~.~t,0.~d,~.~o other:.~9 ~ Am~.~y xOf92tSmS'~"~'O'x'"''0~t' ""''
i. ~:~:~ .'.: ~..""~',~.~'~,~:~':.~"': :: :." i,.,": :~':.~ :,:'::':'.'.: , ," . "~'.:~':'~ :.:,'~': .:,:" i:.': :".':':,:' '::.:'. .,',:.:':"'..".',:~.'~,: :'..," "I':,: I'm ":.'.:".:.
t'ng fit h
~.~ , ~ , . . ~.~ ..~. ~. ~ .~.~.~.~ ~ . ~.~.~.~.~.~ .~ ~ ~ i.- ~ ~ . ~ ~ ~ . ~ ~ ~ . ~ ~ . ~. ~ ~ ~ ~ .~
haps (Slovic, 1987). Such hazards thus may warrant extra analytic atten-
tion and deliberation.
Diagnosis should also consider questions about the state of knowl-
edge about the risk; see box on the next page. There are a few risk
decisions for which actuarial data provide a solid knowledge base (e.g.,
automobile fatalities), but the state of knowledge is more commonly char-
acterized by a mixture of expert judgment, inference, and uncertainty. In
such cases, due consideration should be given to how much incremental
knowledge additional analysis might buy and what it will cost in terms of
time and money, before making initial estimates of the best way to bal-
ance analysis and deliberation for the purposes of understanding specific
risk factors and clarifying uncertainties.
Answers to the diagnostic questions about consensus and possible
omissions from analysis can suggest what will be needed for a risk char-
acterization to meet the likely demands on it. What kinds of analysis will
the interested and affected parties demand? What sorts of quantitative
OCR for page 145
IMPLEMENTING THE NEW APPROACH
145
X'~t x'''"' ~i
~ ~ I tax
~ ,,. ~ . ~ ~ .., . ., ,, ~ ., ~ a. ... .. , , ~ . ~ , ~
. , ,, ~, , I. . ~. ~ ~ ~ I I,. ~ ~ I. ~ I ~
~ .~ . ~.~.~...~ .~...~.~.~ . I. .... .... I, ~.~..~ ~,~ .. ..~..~....~.~...~.~.~.~..~.~.~.~
~ ' ~ ' ~ ~ ~ ~ ~ ~ ~ ~ ' '~8' 'a ~ ~ it' 'I'd X' 'I ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ : ~Aft ~ BRA ~- ~ A A ~ ~ F:: - W ~ ~ ~ ~ ~a:: a:: :~: :~ V A. ~A ~ ~: ~
~ ~'~'~.~.'~.~'~.~"'~...~'.~'~.":~:'~ :'~..':".":.:'." :".'":':':.' .'"':':'': ~ '....":':'. "':.:"'.~:.'":'~ I':" ':':....:.''. ~':""~.."':':"':'.'.':':" ..:.:" "':'.",'... it.
'~':':':':'~:: :~
."'''."'.
:::::~:
'~'~'~"~'~'~'~'~'~""~''~"'~'~'~'~'~'''.'~''_'."''.'~'.'''"''''."'"" "'''''".""'''""".'.""'.""',"'''," "~"".''"',""'"~''',',".""''''"'."'."''""'.''"''""''"'"''"''" "."'""",
::::::
~.~''.~.'.~.."" .'.'.': :'.'' :'.':": :'."': ..~. : I': . .:."'.:.':"." ..:".'.~".:~'.:.:'~.:..'..:.~..': :...~,~':.:'~"':"'~"':"':.:':':.~':"':'.""'.".:"~"'~."~.":':.":"'."":.:':.'.':':':'.'~"'~"."
. ,. ,,.,..
. ~"''""~,''~,'~ ~,~ ,, ~ ,~ ~,~ ~ it, _, > ~ I, I,,,,, ~ ~ ~,, I, -, _ ~, ~, ,, ~,~ it, _ ,,, ,, . ,,,, _ ~ , ,. ,, ,, _
.',~' 'I'd ,',.,.'. ~,~ ~ ~'~ ~.~ ~ ~ .,. ~ ., ,.. ~ _.. I, _~ ...,,,_ _~..~ am,. aft,_, I, I, ~ I.. A, ~ ~,~,~ ~ ~ _, ~ _ ~ _ ,. I.. ~ _ ~ _ _
=
::::.''.,, i..'. ,~ .., ...', ..,,...-.. .,
.',"..~..:'.~""""."""'.~'.'.'...."."~'~,~,~,"', I" '.'"""' '"I ' I' 'I '~:~:~:~:
::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::
information will be needed on risk and uncertainty? Will there be a need
for qualitative descriptions of the state of scientific knowledge? What
kinds of expertise and which scientific perspectives will have to be in-
cluded for the characterization to achieve balance? The diagnosis should
result in preliminary answers to such questions.
While the answers to these diagnostic questions are only preliminary,
they can lead to a sounder and more credible analytic-deliberative pro-
cess that addresses, from the outset, most of the issues likely to be of
concern. When the process begins, of course, it may alter the organ-
ization's preliminary list of options and issues.
2. Describe the Legal Mandate
The legal obligations and legal environment surrounding the deci-
sion activity should be recognized or clarified at the beginning. Issues to
consider for a government agency include the agency's legislative mis-
sion; legal requirements affecting the decision-making process (e.g., Ad-
ministrative Procedure Act); the degree to which the decision-making
OCR for page 146
146 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOC - TIC SOCIETY
authority has been legally delegated; the demand for evidence in terms of
burden-of-proof requirements; expectations regarding legal challenges;
requirements for documentation of the process and for tangible analytic
outputs; and the legal responsibilities of interested parties both before
and after the decision.
Because of the importance of broadly based deliberation, an agency
should avoid taking the stance that a statute prohibits public input simply
because it gives the agency full decision-making responsibility. Rather, it
should develop a clear understanding of how much statutory discretion it
can exercise in order to listen to issues as needed without abdicating
responsibility. Agencies should avoid giving the impression that they
will prolong a process until all participants are satisfied or that listening
to the interested and affected parties is equivalent to a commitment to
decide in favor of those who testified.
3. Describe the Purpose of the Risk Decision
The responsible organization's staff should describe the stated and
implicit purposes of the decision-making activity" the type of decision
and general aims furthered by the activity, and the intended users of the
risk characterization. Different types of decisions may require different
types of knowledge and perspectives and hence require different partici-
pants in the analytic-deliberative process-both inside and outside the
organization. Consider the variety: Is the decision about risk analysis
technique (e.g., selection of default assumptions), about guidelines for
making inferences from data, about regulating an industrial process,
about setting an emissions standard, about taxing emissions and efflu-
ents, about establishing cancer potency values, about informing individu-
als at risk, about policy strategies, or about implementation? Different
kinds of decision also affect different parties, whose concerns the process
must satisfy and whose participation or representation it may require.
Staff should identify the types of decisions that will probably follow
the risk characterization and consider how the particular risk character-
ization activity will facilitate the decision and the overall aims of the
organization. It is important to also consider possible secondary and
tertiary impacts of the decision, which may be of more concern to inter-
ested and affected parties than the primary one.
4. Describe the Affected Parties and Likely Public Reactions
Diagnosis should consider the identity and likely positions and per-
spectives of the interested and affected parties. The first step is to provide
a tentative identification of the parties and any barriers there might be to
OCR for page 147
IMPLEMENTING THE NEW APPROACH
147
their effective involvement in the analytic-deliberative process. As al-
ready noted, the success of the process depends on the satisfaction and
appropriate participation of these parties. It may be difficult to achieve
meaningful participation of some of the parties in those parts of the pro-
cess involving discussion of complex technical issues because of lack of
expertise and funding of those parties, because they mistrust the respon-
sible organization and are unwilling to participate, or because they trust
the organization implicitly and see no need to participate. Government
agencies should resist the temptation to exclude parties whose views are
known to be different from those of agency administrators.
Diagnosis should consider the possibility that some affected parties
may resist participation because they believe they are more likely to
achieve their desired outcomes by some other strategy, such as a legal
challenge. Diagnosis should result in tentative recommendations on how
to address any such problems that seem likely to become significant. (We
discuss some of the possibilities in Chapter 3.)
Diagnosis should also consider the potential for controversy. Which
parties are likely to support or oppose a possible decision? How might
they try to exert influence indirectly? How strong is the public consensus
on the need to address the hazard? What type of press coverage can be
expected? It is important to assess the potential for "rough weather,"
consider the organization's possible responses to it, and lay the ground-
work for a strategy for addressing external pressures. Informal contact
with interested and affected parties can provide valuable insight on these
issues and on how to address them. In some cases, a reorientation of the
analytic-deliberative process can help reduce controversy or channel it
more productively.
The indicators of strong potential for public controversy include:
· There is a clear distinction between those people who benefit from
one of the decision options and those who do not. The potential is stron-
gest when the distinction reinforces preexisting social divisions (e.g., rich
versus poor, workers versus employers, regional or racial differences).
· Recent decisions that are similar to this one evoked controversy.
· The responsible organization suffers from a low level of public
trust or lacks a public constituency.
· The hazard is characterized by a high degree of dread or the poten-
tial for widespread, involuntary exposure.
· The issue presents news "hooks" that attract media attention or
vivid opportunities for use in larger debates (e.g., incinerator siting and
the debate on pollution prevention strategies; the presence of endangered
species).
OCR for page 148
148 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY
There are numerous other factors that may also be worth considering
(Foran et al., 1995~. The same factors also suggest the probable nature of
the controversy that the organization may face.
In our view, the effectiveness of a risk characterization will depend
significantly on the prevailing climate of public opinion. It is therefore
imperative for public officials to diagnose that climate so that they can
arrange for appropriate participation and direct scientists to address is-
sues likely to be raised as criticisms if they are omitted from the risk
characterization.
Conflicts over substance and process are closely related. Affected
parties that raise objections to substantive conclusions or omissions in a
risk characterization product are often also reacting to perceived failings
in the process. If people do not trust the process, they have little trust in
the outcome. Therefore, when public controversy can be anticipated,
diagnosis should pay particular attention to the concerns of the affected
parties and to including those parties in the process.
We emphasize again that understanding the potential for controversy
and designing an analytic-deliberative process accordingly are not enough
to prevent some unwanted outcomes. Deliberations across the range of
decision participants may fail to reach consensus and, sometimes, inter-
ested or affected parties may choose to exert influence indirectly and
outside the officially designated process, such as through litigation, legis-
lation, or mass media publicity. Even with good diagnosis and planning,
such reactions may occur, but they are likely to be less incapacitating.
5. Estimate Resource Needs and Timetable
Diagnosis should consider the internal and external time pressures
on the decision and the extent to which they are explicit. It should con-
sider the public health and other consequences of indecision, and if sig-
nificant, the risk-reduction potential of interim actions during a lengthy
decision-making process (Harris, 1990~. It should consider whether addi-
tional resources might be made available or whether resources might be
cut. The adequacy of resources depends in part on the range of expertise
needed for analysis and on the potential for controversy. Evaluating the
timeline in terms of the legislative, budgetary, and executive election
cycles is a must for complicated decisions by government agencies.
6. Plan for Organizational Needs
The diagnosis should result in a plan that specifies the organizational
support needed for the risk characterization. When a government agency
is responsible, it should consider the needs for coordination between the
OCR for page 149
IMPLEMENTING THE NEWAPPROACH
149
program unit given initial responsibility and other units within the
agency, other agencies, and other levels of government. It should explic-
itly consider the need for early coordination with units that maintain
regular contact with interested and affected parties (e.g., risk communica-
tion programs). Several diagnostic steps can benefit from the input of in-
house experts, as well as the informal input from interested and affected
parties themselves. The diagnostic process should consider the need for a
task force or some similar entity, focused on the particular risk decision,
that cuts across the usual organizational structure. If this sort of coordi-
nation is needed, it should be implemented early.
7. Develop a Preliminary Process Design
The diagnosis should result in a clear proposal for the steps of the
analytic-deliberative process, their sequence, expected iterations, partici-
pants, rules for closure and other decisions, and tangible products. The
plan should be open for discussion by the affected and interested parties
once the process begins, and it should be changeable as needed. It should
consider the legal and resource constraints on the process, where and
how affected and interested parties can participate, time commitments,
and overall time frame. It should also clearly specify whether tangible
products will be needed describing the risk and documenting the process.
8. Summarize and Discuss Diagnosis within the Organization
Both "risk managers" and "risk assessors" should be actively engaged
in all parts of the diagnosis. The discussion and review of the many
judgments involved in the diagnosis will help to surface potential prob-
lems within the organization, clarify the degree of commitment the orga-
nization should make to the activity, and ensure that the organization
enters the process with a consistent position on what it is willing to do in
terms of participation, deliberation, and other potentially contentious is
sues.
Conclusion
Diagnosis should result in a commitment within the responsible or-
ganization, among both staff and management, regarding the nature and
level of effort of the analytic-deliberative process leading to a risk charac-
terization. Especially when the risk analysis is expected to be complex
and difficult and there is likely to be polarization and politicization about
the risk decision, an understanding among both staff in the organization
and the interested and affected parties that the organization is committed
OCR for page 150
150 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMO CRITIC SOCIETY
to the activity is essential. The diagnostic effort should therefore result in
explicit expectations about the extent of the activity and the kinds of
support and constraints that will come from the organization.
The responsible officials should treat the diagnosis as tentative. One
of the greatest dangers of diagnosis is that it may convey a sense that the
problem formulation, the process design, and other aspects of the ana-
lytic-deliberative process are firmly established. Diagnostic efforts are
subject to the test of experience. An unwillingness to modify preliminary
decisions can undermine the larger purpose of making risk characteriza-
tion responsive to the emerging needs of the decision makers and the
interested and affected parties. Officials of the responsible organization
should always keep in mind that their goal is a process that leads to a
useful and credible risk characterization.
BUILDING ORGANIZATIONAL CAPABILITY
Implementing effective risk characterization requires appropriately
structured and staffed organizations and systematic efforts to improve
the knowledge base for designing and managing the analytic-deliberative
process that informs risk decisions.
Organizational Issues
Implementing a broadly based analytic-deliberative process for risk
characterization makes demands on an organization. It must assign re-
sponsibility and authority for diagnosing risk decision situations and for
implementing new analytic-deliberative processes, and it must create pro-
cedures for reviewing and approving its process decisions. The relevant
staff must understand the underlying concepts, which may require spe-
cial training. The organization must be prepared to respond appropri-
ately to requests to analyze dimensions of risk it has not analyzed before
and to acquire the necessary expertise to do so. It must also be prepared
to cope with the possibility of attempts by some of the interested and
affected parties to delay a decision, and it must develop a range of strate-
gies for reaching closure on decisions that affect the process leading to
risk characterization.
Broadening the process also requires new kinds of coordination be-
tween the organizational units normally responsible for risk analysis and
those responsible for interactions with interested and affected parties.
This coordination may require changes in organizational structure or pro-
cedures. It may be advisable in some situations to establish task forces or
working groups that cut across organizational units so as to involve all
the relevant units in the analytic-deliberative process, beginning with the
OCR for page 151
IMPLEMENTING THE NEW APPROACH
151
initial decision to characterize a risk, and to maintain communication
throughout.
For some organizations, a broad concept of risk characterization may
require changes of procedure to permit flexibility and judgment. Those
responsible for managing the process that leads to a risk characterization
must be allowed the flexibility necessary to match the process to the deci-
sion situation. Their organizations will need to develop ways to allow
that flexibility and at the same time guard against arbitrary decisions and
undue influence from interested parties. Replacing a reliance on stan-
dard procedures with a more flexible system will require care both in
assigning responsibility and in establishing safeguards. Perhaps most
important, organizations should develop mechanisms that provide feed-
back on their procedures so that they can be improved over time. Some
agencies consider reviewing deliberative innovations to be integral to
their success and improvement (Fisher, Pavlova, and Covello, 1991;
Young, Williams, and Goldberg, ~993; Crumbly, 1996~. (On organiza-
tional learning related to risk decision making, see Short and Clarke, 1992;
Chess, Tamuz, and Greenberg, 1995.)
Improving the Knowledge Base
Only a very limited knowledge base exists for guiding decisions about
the process that leads to risk characterization. Thus, organizations that
modify their standard procedures and adopt a carefully designed ana-
lytic-deliberative process must to a great extent find their own paths.
This situation can be improved over time if explicit and systematic efforts
are made to evaluate and learn from experience. Such efforts can help
organizations conserve resources and solve problems in at least three
ways:
· Gathering knowledge and feedback early and throughout the ana-
lytic-deliberative process allows mid-course corrections that save time
and money.
· Pretesting materials that summarize risk information reveals in
advance whether this information is understandable and useful for the
intended audiences.
· Retrospective analysis can suggest ways to improve future efforts
(Office of Cancer Communications, 1989; Fisher, Pavlova, and Covello,
1991).
Ideally, an organization involved in a major analytic-deliberative process
will devise systems of feedback and evaluation to inform it both during
and after the process. In addition, institutions that provide scientific
OCR for page 152
152 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIETY
support for many of these organizations, such as federal scientific agen-
cies and industry-based research institutes, should support systematic
efforts that build knowledge about analytic-deliberative processes that
may have general value for many organizations.
One of the difficulties in building knowledge about analytic-delibera-
tive process is defining criteria for success. We do not suggest that inter-
ested and affected parties or organizational managers must all be "happy"
at each step of the process for it to be considered successful. A more
realistic approach is to define criteria for success early on, at or before the
step of process design, using a process in which both interested and af-
fected parties and organizational staff participate (Rosener, 1981~. We
believe that asking questions like those listed below will yield valuable
insights that can be used to develop realistic expectations for the analytic-
deliberative process and to arrive at a working definition of success or
effectiveness.
Criterion
Measurement procedure
Getting the science right
Getting the right science
Getting the right participation
Getting the participation right
Developing accurate, balanced,
and informative synthesis
Ask risk analytic experts who represent the
spectrum of interested and affected parties to
judge the technical adequacy of the risk-
analytic effort
Ask representatives of the interested and
affected parties how well their concerns were
addressed by the scientific work that informed
the decision
Ask public officials and representatives of the
interested and affected parties if there were
other parties that should have been involved
Ask representatives of the parties whether they
were adequately consulted during the process;
if there were specific points when they could
have contributed but did not have the
opportunity
Ask representatives of the parties how well
they understand the bases for the decision;
whether they perceived any bias in information
coming from the responsible organization
Evaluation or feedback should take a form appropriate to the scale
and nature of the analytic-deliberative process: a resource-intensive risk
characterization will merit more rigorous and extensive evaluation than a
more limited one. Evaluation efforts may use quantitative or qualitative
methods and aspire to different degrees of rigor as the situation demands.
(For detailed discussion of principles and methods of evaluation for deci
OCR for page 153
IMPLEMENTING THE NEW APPROACH
153
signs involving broadly based participation, see, e.g., Sewell and Phillips,
1979; Rosener, 1981; Fisher, Pavlova, and Covello, 1991; Syme and Sadler,
1994.) Generally, conscious efforts at evaluation both during and after the
process are important for improving analytic-deliberative processes.
Feedback and evaluation can begin in the diagnosis phase, when the
responsible organization begins to define the resources it will need to
develop a risk characterization and to develop a preliminary process de-
sign. Process evaluations that seek data for midcourse corrections should
be common practice. The U.S. Department of Energy, for example, has
developed a practice in which citizens' advisory committees do self-as-
sessments on at least a yearly basis (Grumbly, 1996~. Surveys have also
been used to solicit feedback before and during efforts to involve commu-
nities in problem solving (e.g., Pflugh, no date). Informational materials
can be pretested for comprehensability and relevance and to ensure that
they deliver the intended message, as well as being technically accurate
(e.g., Office of Cancer Communications, 1989; Morgan et al., 1992~. We
strongly suggest that risk characterization messages developed for wide
distribution undergo pretesting with the intended audience.
Other innovative approaches may also be appropriate in certain situ-
ations. One is to conduct simulations to illuminate particular issues.
Simulations mimic actual decision processes but are conducted outside
the decision itself. They can be used to examine the ways that delibera-
tive groups formulate problems, identify the outcomes that require analy-
sis, and interpret scientific information. Simulations can be useful for
identifying potential problems with existing processes and for suggesting
particular approaches that might be tried in real decision contexts. (For
an illustrative example, see Hester et al., 1990~. Organizations may also
use quasi-experimental evaluation designs to compare different proce-
dures, for instance, for integrating analysis into deliberations, for discuss-
ing scientific information in a diverse deliberative group, for arriving at
particular judgments to be incorporated as assumptions in risk analyses,
or for reaching closure. In this approach, an organization treats its inno-
vations as experiments and studies their effects in comparison with other
procedures. An organization might also use what it believes will be an
exemplary process and gather data on it. If the first attempt is successful,
it might become a benchmark for future processes; regardless of its de-
gree of success, it can become a baseline for learning and improvement.
An organization can use informal feedback or combine formal and
informal evaluations. For example, it might organize broadly based advi-
sory groups to review the processes it has used to make the judgments at
each step leading to risk characterizations. The advisory groups could
consider whether it appears with hindsight and with the findings of evalu-
ation research that the analytic-deliberative process might have been more
OCR for page 154
154 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMO CRITIC SOCIETY
effective if conducted differently. An organization, with the help of its
advisory group, might then consider what lessons past experience holds
for future risk characterization efforts. Organizations might also build
and update libraries of case files that use standard categories to record the
processes used to make the judgments that affect its risk characteriza-
tions. These files would provide a basis for systematic case study re-
search on the organization's past experience and for review by an advi-
sory group.
CONCLUSION
A broadly based analytic-deliberative process may require organiza-
tions that characterize risks to engage in new and unfamiliar activities.
Although there are legitimate concerns about the practicality of making
such changes, we see good reason to do so, while testing the effects on the
overall process and implementing safeguards against attempts at tactical
delays. Although there will be initial increases in the cost in money and
time for some risk analyses and characterizations, the overall costs of risk
decision making may actually decrease. It may cost less in the long run to
do it right the first time.
Successful implementation depends on matching the analytic-delib-
erative process to the needs of the decision. This requires a clear under-
standing of the decision milieu. Although there is no standard procedure
for doing this, organizations can benefit by asking a series of diagnostic
questions when they plan the process and by keeping their diagnoses
flexible and responsive to information that emerges during the process.
Implementation may also require organizational efforts at staffing and
training and organizational changes to permit the necessary coordination
among units and to allow flexibility in the processes informing risk deci-
sions. There is also a need for evaluation research to improve the analyti-
cal-deliberative processes on which risk characterization depends.
Representative terms from entire chapter:
affected parties
