Click for next page ( 134


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 133
6 Implementing the New Approach r ~ he previous chapters call on or- ganizations that engage in risk analysis and characterization to do things they do not routinely do: combine analysis with deliberation, broaden the range of outcomes potentially subject to analysis, and broaden partici- pation in activities that were previously restricted to analytic experts and a few decision makers. It may seem to some readers that implementing such an approach would be quite impractical because it would require a major increase in the effort made to characterize risks at a time when the responsible organizations are already overloaded and resources are stable or shrinking. We believe, however, that when the effort is appropriately scaled to match the needs of the decision at hand, it does not necessarily require more time and money and that when it does, the potential ben- efits are likely to outweigh the costs. This chapter discusses the issue of practicality as it affects our approach to risk characterization. It then discusses two keys to making our approach practical: diagnosing risk decision situations in order to match the process to the needs of the situ- ation and building the capability for implementation. PRACTICALITY There are legitimate concerns about the practicality of the analytic- deliberative approach to risk characterization presented in Chapters 3, 4, and 5. One concern is with the costs and benefits of expanding the con- cept of risk analysis to include ecological and social outcomes as well as 133

OCR for page 133
134 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMO CRITIC SOCIETY threats to human health and safety. An expanded domain for risk analy- sis might require the responsible organizations to hire new kinds of ex- perts and to support new and expensive analyses, some of them relying on new techniques. It is reasonable to ask whether such additional activ- ity can lead to better or more acceptable decisions and whether in a con- text of restricted budgets, it might displace other, more essential, efforts. Another concern is that bringing interested and affected parties into sci- entific and technical discussions, such as about which analytic technique to use, might introduce delay and confusion and allow nonscientific con- cerns to impinge improperly on scientific decisions. A third concern is that adding participants and increasing the number of issues that must be considered can provide many opportunities for any interested or affected party that might benefit from delaying a decision to find excuses for delay. Such adverse outcomes might indeed result from adopting our ap- proach to risk characterization, and it is certainly true that expense, delay, and confusion already plague risk characterizations. On balance, how- ever, we believe the approach we propose is more likely to mitigate these outcomes than cause them, especially when applied to major decisions with potentially wide impacts. Consider, for example, the potential for expensive and time-consum- ing debate about the adequacy of risk analyses. Analyzing additional dimensions of risk may seem to invite additional debate, but experience shows that extended and unproductive debates have been prompted by omissions in existing analyses. In large and complex decision exercises, risk characterizations that do not consider ecological, social, or human health outcomes that are important to some of the interested and affected parties or that are based on a process that excludes key parties can lead to court challenges and other activities that question the technical adequacy of the analyses, when the actual concern is the process or risks that were never analyzed. We believe this pattern has been a major cause of delay in decisions about high-level radioactive waste disposal, siting waste incinerators, and other intensely controversial risk decisions. A delibera- tive process that ensures that the decision-relevant risk analyses are per- formed the first time may reduce delay in such cases, rather than increas- ing it. If a decision requires additional analyses to meet the major concerns of important parties to the decision, the short-term expense of obtaining the additional expertise may actually be an investment in longer term savings of time and money. In some cases, there may not even be additional expense. Delibera- tions in advance of risk analysis may reduce the immediate costs of analy- sis or increase its cost-efficiency by directing limited resources for analy- sis to the most decision-relevant issues. For instance, incremental efforts

OCR for page 133
IMPLEMENTING THENEWAPPROACH 135 to reduce uncertainty in risk analyses could be directed toward those uncertainties whose reduction might change some of the parties' under- standings enough to affect their judgments about what should be done. We believe that an old carpenter's adage applies to risk analysis and characterization: measure twice, cut once; measure once, cut twice. Experi- ence shows that analyses, no matter how thorough, that do not address the decision-relevant questions and use reasonable assumptions can re- sult in long and detailed criticism, great expense and delay in responding to the criticism, and even rejection of a risk decision in court. These hidden costs are especially likely to arise when decisions involve very big stakes and major controversy. When the entire decision-making process is taken into account, not only the costs of analysis, it often costs less to get it right the first time. It is reasonable to ask whether broadening deliberation, especially on issues that have a strong technical component, such as selecting assump- tions for risk analysis, will cause confusion and delay, particularly if the added participants do not understand the technical issues at stake or the language of the technical experts. This potential certainly exists, and avoiding it imposes costs. To involve nonexpert participants meaning- fully, efforts must be made to educate them technically or to find indi- viduals who understand the technical issues and can represent the par- ties' knowledge, perspectives, and concerns in a way that satisfies those parties. The problem of meaningful participation and the costs of achiev- ing it are most serious with parties that have not been well organized or that lack resources to identify or hire their own experts. However, leav- ing those parties out of meaningful deliberations that affect the risk char- acterization has its own dangers to the quality of understanding and to the acceptability of the ultimate decisions, as noted in Chapters 2 and 3. These dangers are sufficient in our judgment to warrant experimental efforts to provide resources to allow meaningful participation for parties that could not otherwise join effectively in deliberations. Such experi- mental efforts should be focused first on risk decisions that may seriously affect the parties in question. They should be designed in consultation with the parties to be assisted and carefully evaluated, with the collabora- tion of those parties, as to both process and outcome. Another concern about broadening analysis and deliberation comes from a recurring problem with risk decisions in the United States. Some parties use repeated requests for broader analysis or further deliberation as a tactic to delay a decision or to advance their interests as they could not in the decision process as originally organized. In some cases, a risk characterization process that encourages broader analysis and delibera- tion will invite such tactics. Yet sometimes those requests for delay are tactical reactions to procedures that excluded some of the parties' chief

OCR for page 133
136 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY concerns from consideration on an a priori basis. Delays of this type would become less frequent under a broader concept of risk characteriza- tion. Clearly, procedural safeguards are needed to reduce inappropriate or avoidable delays. We encourage organizations to experiment with different approaches to dealing with the problem of strategic delay. We note, however, that in implementing the broader concept of risk charac- terization, especially in decisions with wide impact, more of the effort of risk analysis and characterization will be focused on substantive ques- tions about risk and less on the adequacy of procedures. Furthermore, broadly based deliberation offers a more efficient forum than the courts for arriving at socially acceptable judgments about whether a request to extend analysis or debate is necessary to the overall decision process. We emphasize that our proposed approach adds to analysis, delibera- tion, and participation only as appropriate to specific situations. We do not propose wholesale analysis of every possible outcome, deliberation of every analytical issue, or involvement of all interested and affected par- ties in all steps leading to a risk characterization. Rather, we advocate that these possibilities be uniformly considered and that as many addi- tional activities included as is appropriate for the situation. For some of the most contentious risk decisions, our approach indeed calls for very extensive analysis, deliberation, and participation much more than is envisioned in a linear approach that would only allow deliberation after a risk analysis is supposedly complete. However, for most such decisions- such as about high-level radioactive waste disposal, dioxin, and the like- the linear approach has not produced the efficient process that was hoped for, even after huge investments in risk analysis and characterization. For the vast majority of risk decisions, our approach calls for much less than the most extensive possible analytic-deliberative process, and may not add much, if anything, to the current level of effort. We note, though, that the design of simple, generic risk characterization procedures should pre- sumptively involve the spectrum of interested and affected parties and that established processes of this type should be periodically reviewed. A final concern about the practicality of our approach is that forces external to the analytic-deliberative process may sometimes preclude its use. There may be organized political opposition to allowing some parties a voice in risk analysis or to considering certain kinds of concerns in a risk characterization. This sort of concern is highlighted by the California Comparative Risk Project and the regulatory negotiation on disinfectant by-products (described in Appendix A). In both cases, the conclusions of an analytic-deliberative process with many of the features of our approach was overturned in the larger political system. In California, the argument was that considering certain kinds of risks to human welfare in risk rank- ing would be unscientific. It is normal in a democracy for parties dissat

OCR for page 133
IMPLEMENTING THE NEWAPPROACH 137 isfied with a decision to seek redress elsewhere in the system. However, when extensive efforts are made to involve the full range of interested and affected parties in a deliberative process and individual interests bypass the process, it is destructive of the search for deliberative solu- tions. It may be necessary or valuable at times for the larger political system to legitimate analytic-deliberative processes and thus make it more costly for interests to bypass them. We have not considered the advan- tages and disadvantages of different ways to legitimate these processes. Among those that might be considered are declaring in advance that the results of certain analytic-deliberative processes will be legally binding on government agencies and establishing analytic-deliberative processes or forums that would continue to participate in governance functions, advising on the final decision, monitoring its implementation and its ef- fects on the things at risk, and recommending changes in policy as appro- priate. The latter approach has sometimes been used with local risk deci- sions about such matters as landfills. DIAGNOSIS: MATCHING THE PROCESS TO THE DECISION A key to implementing our approach is to match the analytic-delib- erative process to the needs of the risk decision. Doing this can be diffi- cult, and not enough is known to justify any standard procedure for matching. In this chapter, we offer some guidance on how to make that match. We find it useful to rely on an idea most often associated with medi- cine, that of diagnosis. A government agency or other organization re- sponsible for risk characterization begins with a diagnosis of the potential hazard situation that is sometimes explicit (e.g., risk defined by law) and sometimes implicit. Diagnosis includes, at minimum, ideas about the nature of the hazard and the hazard situation, the purposes for which the risk characterization will be used, the kinds of information that will prob- ably be needed, and the kind of decision to be made. Diagnosis is typi- cally implicit when an agency applies an existing routine, presuming it adequate for the situation at hand. At other times, important elements of the diagnosis are implicit, such as assumptions about which parties are affected or which threats of harm deserve analysis. We recommend that diagnosis be conducted explicitly far more often than is the current prac- tice. Although a single organization may have responsibility for risk characterization, diagnosis generally benefits from interactions of its staff with scientists, policy makers, and interested and affected parties. Diagnosis results in a provisional procedure for each step of the ana- lytic-deliberative process leading to a risk characterization. These provi- sional choices should be reconsidered, with input from the interested and

OCR for page 133
138 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMO CRITIC SOCIETY affected parties, during the entire process. Because the best approach to a risk characterization depends on the specific situation, the first element of diagnosis is surveying what we call the risk decision landscape. The Risk Decision Landscape Risk decisions vary along many dimensions. Although it might be desirable to reduce these to a few, as has been done with the qualitative aspects of hazards (see Figure 2-5, p. 63), analogous systematic research has not been done on risk decision situations. In our judgment, there is no simple set of categories that can be confidently used to reduce the great variety of risk decisions to a few types for the purpose of designing a few standard approaches to informing the decisions. A similar conclusion has been reached by Graham et al. (1988~. We believe, however, that it is useful to consider a number of diagnostic questions before embarking on the processes that lead to risk characterizations. We list some key ques- tions below. A good diagnosis can help practitioners narrow the range of appro- priate courses of action and thus increase the chances of a rational and socially acceptable outcome. For example, past experiences with decision situations similar to the one at hand can provide some guidance on proce- dures that may work well in the new situation. A good diagnosis can make it easier to consider how a risk decision problem differs from those made in the past and whether existing decision routines should be changed. Despite our caution about the feasibility of classifying risk decision situations for the purpose of creating a small number of decision-making routines, we think it is useful to keep in mind the following five categories that occur with some frequency on the risk decision landscape. The first category, unique and wide-impact decisions, encompasses those that are most well known and most often controversial because they are one-time ac- tions that affect a large portion of the country, a large number of people, or have effects for a very long time. At the other end of the spectrum is the category of routine and narrow-impact decisions, which are usually very similar to previous decisions and involve a small geographic area, few people, or primarily short-term effects. Somewhere in the middle of this spectrum is a category that we term repeated, wide-impact decisions: like unique and wide-impact decisions, they may have major effects over a large area or large numbers of people, but like routine and narrow-impact decisions they are very similar to previous decisions so that it is relatively easy to anticipate the issues they will raise. Our last two categories do not directly involve decisions about a specific potential hazard: generic hazard characterizations and decisions about policiesfor risk analysis. Because these

OCR for page 133
IMPLEMENTING THE NEW APPROACH 139 two categories of activities are not designed to result in a specific decision about a specific hazard, they may appear to be outside our framework for an analytic-deliberal:ive process; however, we believe that they, too, can often benefit from this process, beginning with diagnosis. Unique and Wide-Impact Decisions Unique and wide-impact decisions are one-time decisions of national or even wider import that usually involve many kinds of interested and affected parties and disparate perspectives on what is at risk. The para- digmatic case is that of decisions associated with the Yucca Mountain site for a permanent national repository for high-level radioactive waste. Be- cause of the size of the stakes in the ultimate decision in such an instance, risk characterization often needs to be based on extensive analysis and deliberation with broad participation or representation of the spectrum of interested and affected parties at every step of the process. Such decisions present special challenges in planning and in carrying out an effective analytic-deliberative process. The process considers risks on a large so- cial and geographic scale, and it often extends over considerable time. It may expand in ways (other topics, across distance) that were not foreseen at the start. Because of the importance of the decisions, considerable resources and time may be available to meet these challenges, so it may be easier to meet the needs for breadth, inclusion, and attention to process than when less weighty issues are at stake. Routine and Narrow-Impact Decisions In contrast, some risk characterization procedures support thousands of routine and narrow-impact decisions each year. These may include decisions to issue permits to release small amounts of effluents into air and water, to approve building designs as adequately earthquake resis- tant, to accept individuals as blood donors, and so forth. Although there may be significant unresolved scientific issues underlying individual de- cisions, an extended analytic-deliberative process for informing each one would be impractical and might not always serve the overall public inter- est. For this reason, it is reasonable to routinize the associated risk charac- terizations. When establishing routines, it may be useful to consider using a broad-based analytic-deliberative process to devise a general procedure that would be used for the individual decisions and then provide for an appeals procedure for individual decisions as well as periodic public re- view of the general process. The periodic review should probably in- volve an analytic-deliberative process roughly as broadly based as the

OCR for page 133
140 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIETY initial effort to create the routine. Periodic review may be useful for a variety of reasons. Such review may be legally mandated; there may be a change in management followed by calls to reconsider certain decisions; occasionally, scientific breakthroughs may indicate a better scientific ba- sis for future decisions; or there may be public calls for change resulting from a process failure. An organization may also desire to change its procedures because of concerns about representation in the process, inef- ficiencies, or shrinking resources. Because of the potential for loss of trust, the responsible organizations should consider planning for regular review of the risk characterization routines used for informing major classes of routine, quick decisions. Many of the considerations that apply to setting up and conducting the analytic-deliberative process leading to risk characterizations for unique, wide-impact decisions apply also to reviews of routines for risk analysis or characterization. There are a series of obvious questions to ask in evaluating routine decision-making processes: Input and access: Do some parties have considerable input to deci- sions while others have little or none? Are there sufficient resources for adequate participation? Is the process closed because of claims regarding confidentiality? If so, are there mechanisms for review? Decision quality: Are complaints regarding decisions justified? Are there criteria for identifying bad decisions in the absence of complaints? Efficiency: Are there mechanisms for staff to identify and easily point out inefficiencies in the process? Are some decisions quickly made that should require more review and deliberation? Are some decisions too slow? Trust and satisfaction: Do some of the interested and affected par- ties express distrust about the process? Do some of the parties claim that the decisions are arbitrary or unfair? Review: When was the last time the process was evaluated or re- viewed? Are there established processes for evaluation or review? Is there an appeals process? Is the review process broadly accepted? Resources: Is the process too costly? Does it have sufficient re- sources? If not, are additional resources available? Repeated, Wide-Impact Decisions Repeated, wide-impact decisions are those that will be the subject of widespread attention because of their possible effects but are sufficiently similar to other decisions that some routinization seems possible. Typical of these are decisions about approving siting and operating permits for power plants and hazardous waste facilities and, in recent years, about

OCR for page 133
IMPLEMENTING THE NEW APPROACH 14 strategies for restoring ecosystems. The analyses for them are often local or regional in scope and so may present fewer logistical limitations than national cases since the interested and affected parties may have easier access to the responsible organizations. These decisions have aspects in common with the previous two cat- egories of decision. They often present the potential to routinize aspects of the analytic-deliberative process, yet they have wide impacts and some- times high stakes. The responsible organizations should be alert to the need, especially when the likely impact of a decision or the potential for controversy is great, to design or modify aspects of the process to suit unique needs of the particular decision. As with routine decisions, any standard procedures should be periodically reviewed; as with unique, wide-impact decisions, it is important to consider instituting broadly based deliberative mechanisms in one or more of the tasks leading to a risk characterization. Generic Hazard and Dose-Response Characterizations Generic hazard and dose-response characterizations are not designed to inform any particular decision, but to serve as inputs for a class of decisions. Examples include efforts to describe the health risks of dioxin, the impacts of climate change on human and ecological health and the economy, and the likelihood of airborne transmission of a particular dis- ease. These efforts are abstracted from the context of any particular deci- sion about a specific situation in which exposures take place, but they can have far-reaching impacts and therefore deserve careful attention, similar to that accorded risk characterizations associated with unique decisions. A special problem with these risk characterizations is that absent a par- ticular decision, it may be more difficult to identify the interested and affected parties in advance and hence to arrive at the most appropriate formulation of the problem. Typically, people in the institutions that would clearly be affected (e.g., producers of a chemical) and those con- cerned about the precedents that may be set or changed by these types of decisions can be relied upon to participate, but representatives of the more general public or environmental or community groups cannot, per- haps because of lack of resources or limited expertise. Ways to broaden participation in these exercises should be explored. Decisions about set- ting standards, for instance, for exposure to a toxin or for the performance of a piece of equipment, raise many of the same issues as generic hazard characterizations (Fischhoff, 1984~. 1

OCR for page 133
142 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY Decisions about Policies for Risk Analysis Decisions about policies for risk analysis are procedural or method- ological in nature, such as decisions about which dose-response model to use in toxicological analysis, whether to routinely consider psychological impacts in risk analyses, how to gather information about previously unstudied outcomes, and so forth. These have already received consider- able attention from an analytic standpoint from the National Research Council (1994a). These decisions are like generic hazard characterizations in that they are not tied to a specific risk situations but may affect many substantive decisions. Consequently, it may be difficult for many of the interested and affected parties to recognize their importance and to mobi- lize resources to participate in the analytic-deliberative process. The re- sponsible organizations may need to make special efforts to identify and involve the parties and to ensure broad and balanced participation. Diagnostic Steps and Questions To diagnose a risk decision situation, we offer eight steps. Depending on the risk decision, the effort involved may be very brief or rather exten- sive. This section is written primarily with government agencies in mind, but we believe it will also be useful to other organizations responsible for analyzing and characterizing risks. We do not intend to create a new bureaucratic procedure, but instead to reduce wasted effort through ad- vance thought and planning. Figure 6-1 shows the diagnostic steps in- volved in preparing to conduct an analytic-deliberative process to inform a risk decision. It represents the fact that the steps are not necessarily sequential all of them flow into the diagnosis and thus inform prelimi- nary choices about how to execute the process. 1. Diagnose the Kind of Risk and the State of Knowledge Every analytic-deliberative process sets boundaries as it begins to consider the risk problem, define options, and examine consequences. Considering these boundaries explicitly and systematically from the start has the advantage of identifying mismatches between the boundaries that the responsible organization is initially inclined to set and the demands of the situation. This phase of diagnosis begins with asking basic questions about the hazard; see box on page 144~. Answers to the questions about who or what is exposed have implications for who should participate in the ana- lytic-deliberative process, including the possible need to find ways to include the perspectives of parties that cannot speak for themselves (e.g.,

OCR for page 133
IMPLEMENTING THE NEW APPROACH S. Summarize and discuss diagnosis within the organization 1 1 6. Plan for organizational needs | 1. Diagnose the kid; I of risk and state of knowledge preliminary process .(~ DIAGNOSIS ). l design 5. Estimate resource needs and timetable FIGURE 6-1 Diagnostic steps for risk decision making. 143 2. Describe the legal mandate 3. Describe the purpose of the risk decision 4. Describe the affected parties and likely public reactions infants, future generations) or that lack sufficient expertise to be effective participants. These answers may reveal highly exposed or susceptible populations or suggest that those at greatest risk are not identifiable. If risks appear on initial consideration to be inequitably distributed as a function of race, gender, socioeconomic status, or other factors, the diag- nostic effort should lead to a conclusion that the analytic-deliberative process specifically address these issues and that the potentially affected parties participate to ensure that the process is carried out in a way they find competent and credible. Answers to diagnostic questions about the nature of the harm should reveal the kinds of human health effects and ecological impacts that may need to be characterized and the various other kinds of possible adverse outcomes that technical experts or interested and affected parties con- sider important. The test for which harms to consider in the analytic- deliberative process is a practical one: Which harms must a characteriza- tion address for it to be accepted as sufficiently thorough Answers to questions about the qualitative features of the hazard should help in anticipating demands from potentially affected parties for detailed analysis and information, as well as for opportunities to partici- pate. Hazards that are high on dimensions associated with dread and lack of knowledge (the upper-right quadrant in Figure 2-5) are especially likely to generate such demands and to have ripple effects, partly because one such event may signal or portend future, perhaps catastrophic mis

OCR for page 133
144 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY ~ Giagnost'~ Ouest'0~s About C _ .~ ~ ~ ~ ~ ~ .~ ~ ~ ~ ~ ~ ~ ~ . . ~ ~ ~ ~ ~ ~ ~ ~ ~ . . . . . . . . .. . ~ ~ ~ ~ - 0 at. ~..7 8.~.. Ace. I.... l~ 7 .~mS9. .... the. .. or ~.~.~ ben~t f~m .~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ i, ~ it, . ~ ~ ~,~ i.. ~ ~ ~.~,~.~ ., ~...~. ~ ~ ~ ~.,~ . , aid,., ., ,~ ~ ~ ~ ,~ ., ~.~ ~ ~ ~} ~ , ~ thm 6 ~ ~ ~ tt b! ~ Ad ~Ixble~} ~ h hI .. ~.~.~.~.~.~.~.~,~.~.~. ~. ~. I. ~., ~,, ~. I. , ~. ~.~. ~it if. , , ,. ~ I ~= Eng'nee~p~ses~tmay~l~cos ~ ~ ~.~.~.~.~,. ~.~.,~ ,.,. aid,. ~ ., ., ,,. ., ~, ~.~,~,~, ~.~, ~. ~ .~ ~m ~ ~9 ~ ~ ~ ~ x000~"~.'bOh8~'''...~'.''C'0~ - '''I'' ~ ~ ~ ~ ~ ~ . ~ ~,~ if. ~ ~ ~ ~ , . ~. ~.,~ ~.~.~.~ ~.~. ~, ~ ~ . if ~ ~ ~ I,.,. . , ~ ~,~ ~,~ if. . ~.~ aid , , ~ ~ ~ ... ~ . I. ~. ~. ~ . if if. ~ of ....... if. ~ ~ . ~ ~ ~ ~ If. ~.~.~. ~ if. ~ I ~ ~ ~ I if. ~ ~ ''I' ~"~ ~ ~ A'' 'Meg I'd ~ ~ ~ 'ECo'togImI 'I ~ ~ ~dd " I'm ~ ~ ~ ~ ~ ~ de - of:: Mo ~ :: ~ ~:~:~ a: :: ~ If: ~ ::: :: ~ ::~ ::~ ~ ~ ~ :~ ~ ~ ~ ~ ~ {x ~9~L f ~9~ ::::::::: . $ . :::::::::::::: : ::::: :::::::: ~:'~:~:~'~.:~.~. ~'~'~'~'~ ~ ~'~'~'~ ~''''.''~''~.'~''~''::: ':'' . ' .'.' ~ ~ 'I I'm :''':.. ~ ~ . :''''''' ..~: ~.''~'~.': O'er ~ ' .~ .' . ~ ~ .,',, ~,~,~ ,, ~,~,~M',,~e,,,,~m, ,~t,.,.~t,,~'~,.~t'.'.~'~'~ ?: ~'~'"'~2~'~'~'~'~'.'~ ::::~::::::::::::::::: ::: :: :::: :: :::: :::: :::::: :::: ::::::::: :.: :::: :. ::.:: ::.::: ::: ::: ::: :.:: :::::. :::::: ::,:: ::::::::: : : ::::: :,::::: : . :: ::::: :: :~ .~:~:~,:~.:~,~,~ .... : : , ~ : ~ : :. ~.' ~. ~ ,: :. .~ if. ~,~.~ ,::' . :.,~' ~.~ ~'~.'~ I: :.,' ..~ if' 061~ VOlU~S Ot =~um ~ m~ ~mD 1~c A! 3~00 x ~,~,~.~,~,~,~,~l,06.~,,8~.,~e hi,,.,, Debt ,. X6' X' ,,~..,! XiX' ,,~..~b .,: V~,,~.., ~.X..~,~,~,~,,,~ ~'~'~'~'~'~.~'~'~'~'~ '~''.'~'."'.'~,''~'~'~,~,~''~'.,'~,~'~'~'.'~'~',','~'.'.',~'~'~'~'",~,' ~'~'~'~'., ~ ,, ':. 'I . ~ ~ ~ ~ ~ ~ ~ ' ~ ~ ~ ' ~ ' ~ ~ ~ ~ ~ ~ ~ ' ~ ~ ~ ~ ~ ~ ' ~ ~ ~ , . . ' ~ . ' ' ~ ' ' ~ , ~ , ~ " ~ ~ , , ~ ~ . , ' ~ ~ ~ ' ~ ' ~ ' ~ ' ' ~ ~ ~ ' ' " " " ' ' ~ ' ~ ' ~ ' " ' " " ' ~ ~ " ' " " ' ~ ~ ~ ~ ~ ~ ~ ~ ' ~ " . ' ' " " ~ ~ ~ ~ ' ~ ~ ~ ' . " ' ~ ~ . ~ ~ ~ ~ ~ ~ . , , ~ , . . ~ ~ ~ ~ ~ ~ ~ . . , . , , , . ~ , . 1u'd'9m'e"'^ ab"~t Who i'S ''m2S''"""" ' ' " ' ' ' ' ' ' ' ' " ' ' ' ~ :.,. ,,' ' ' ~ ~ ~ ,~ ~ ~ ~ ~ ~ ~ ~ ~ , ~ ,,, .,,,. :. ~.~,~ .', ".:X,'',' ~.'.~,~'~ .~ ~.~ ~,~,~ ~.~,~,~ ~ ~:~,~,~,~ ~ ,~,~,~, ~ ~ ~ ~e ~ _~2 ~y ,~ UniqUe 'detents *9 ~y 'and '~'~,~"":~'~:'.. 'I."". ~ ,','".'.'.':. ".''~::"'.''~' ~'~'~:,"'~':',' i" ,::''" "~"','~':',''~"',:~"""":~::'~'~ ,'' ,"":'::: :,''"".,' ,'~'~'~',"":::,.'.~". "'I ': ":"":"':"':::"'"'. :":" "'~''~' '','',"'.'',' ' 'a' "' ' 9~.,'~'~'R' ': ' ' ' I,) ' '.' ' ' ."" I' ' ' ' 'I' 9"""',".,~.'.'~'~"'~'~'~.~,.~"~'~,'',~"'.~.'.~'~'~"'.""~,~'~"".~.'~'~.~."~",~"'~,"~":"., '-' ~'"'"~"'a'2''n"~"~"'''0""''''""''' ~ 'a' "'''''' A'"' "' '''''' '' ' ' ~'~'~'~'~"'"~""'~'~""""''''""""X"~' "."""'""'. '"""""''~""'""'""""~'~'~'~""'".'.""""''"'''"'''"'~'.~""''."''"'"'',""'~'~',~".'"' i':"""",' ~'."'.~" "' .~ ~.~ ~.~.~t,0.~d,~.~o other:.~9 ~ Am~.~y xOf92tSmS'~"~'O'x'"''0~t' ""'' i. ~:~:~ .'.: ~..""~',~.~'~,~:~':.~"': :: :." i,.,": :~':.~ :,:'::':'.'.: , ," . "~'.:~':'~ :.:,'~': .:,:" i:.': :".':':,:' '::.:'. .,',:.:':"'..".',:~.'~,: :'..," "I':,: I'm ":.'.:".:. t'ng fit h ~.~ , ~ , . . ~.~ ..~. ~. ~ .~.~.~.~ ~ . ~.~.~.~.~.~ .~ ~ ~ i.- ~ ~ . ~ ~ ~ . ~ ~ ~ . ~ ~ . ~. ~ ~ ~ ~ .~ haps (Slovic, 1987). Such hazards thus may warrant extra analytic atten- tion and deliberation. Diagnosis should also consider questions about the state of knowl- edge about the risk; see box on the next page. There are a few risk decisions for which actuarial data provide a solid knowledge base (e.g., automobile fatalities), but the state of knowledge is more commonly char- acterized by a mixture of expert judgment, inference, and uncertainty. In such cases, due consideration should be given to how much incremental knowledge additional analysis might buy and what it will cost in terms of time and money, before making initial estimates of the best way to bal- ance analysis and deliberation for the purposes of understanding specific risk factors and clarifying uncertainties. Answers to the diagnostic questions about consensus and possible omissions from analysis can suggest what will be needed for a risk char- acterization to meet the likely demands on it. What kinds of analysis will the interested and affected parties demand? What sorts of quantitative

OCR for page 133
IMPLEMENTING THE NEW APPROACH 145 X'~t x'''"' ~i ~ ~ I tax ~ ,,. ~ . ~ ~ .., . ., ,, ~ ., ~ a. ... .. , , ~ . ~ , ~ . , ,, ~, , I. . ~. ~ ~ ~ I I,. ~ ~ I. ~ I ~ ~ .~ . ~.~.~...~ .~...~.~.~ . I. .... .... I, ~.~..~ ~,~ .. ..~..~....~.~...~.~.~.~..~.~.~.~ ~ ' ~ ' ~ ~ ~ ~ ~ ~ ~ ~ ' '~8' 'a ~ ~ it' 'I'd X' 'I ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ ~ : ~Aft ~ BRA ~- ~ A A ~ ~ F:: - W ~ ~ ~ ~ ~a:: a:: :~: :~ V A. ~A ~ ~: ~ ~ ~'~'~.~.'~.~'~.~"'~...~'.~'~.":~:'~ :'~..':".":.:'." :".'":':':.' .'"':':'': ~ '....":':'. "':.:"'.~:.'":'~ I':" ':':....:.''. ~':""~.."':':"':'.'.':':" ..:.:" "':'.",'... it. '~':':':':'~:: :~ ."'''."'. :::::~: '~'~'~"~'~'~'~'~'~""~''~"'~'~'~'~'~'''.'~''_'."''.'~'.'''"''''."'"" "'''''".""'''""".'.""'.""',"'''," "~"".''"',""'"~''',',".""''''"'."'."''""'.''"''""''"'"''"''" "."'""", :::::: ~.~''.~.'.~.."" .'.'.': :'.'' :'.':": :'."': ..~. : I': . .:."'.:.':"." ..:".'.~".:~'.:.:'~.:..'..:.~..': :...~,~':.:'~"':"'~"':"':.:':':.~':"':'.""'.".:"~"'~."~.":':.":"'."":.:':.'.':':':'.'~"'~"." . ,. ,,.,.. . ~"''""~,''~,'~ ~,~ ,, ~ ,~ ~,~ ~ it, _, > ~ I, I,,,,, ~ ~ ~,, I, -, _ ~, ~, ,, ~,~ it, _ ,,, ,, . ,,,, _ ~ , ,. ,, ,, _ .',~' 'I'd ,',.,.'. ~,~ ~ ~'~ ~.~ ~ ~ .,. ~ ., ,.. ~ _.. I, _~ ...,,,_ _~..~ am,. aft,_, I, I, ~ I.. A, ~ ~,~,~ ~ ~ _, ~ _ ~ _ ,. I.. ~ _ ~ _ _ = ::::.''.,, i..'. ,~ .., ...', ..,,...-.. ., .',"..~..:'.~""""."""'.~'.'.'...."."~'~,~,~,"', I" '.'"""' '"I ' I' 'I '~:~:~:~: :::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::::: information will be needed on risk and uncertainty? Will there be a need for qualitative descriptions of the state of scientific knowledge? What kinds of expertise and which scientific perspectives will have to be in- cluded for the characterization to achieve balance? The diagnosis should result in preliminary answers to such questions. While the answers to these diagnostic questions are only preliminary, they can lead to a sounder and more credible analytic-deliberative pro- cess that addresses, from the outset, most of the issues likely to be of concern. When the process begins, of course, it may alter the organ- ization's preliminary list of options and issues. 2. Describe the Legal Mandate The legal obligations and legal environment surrounding the deci- sion activity should be recognized or clarified at the beginning. Issues to consider for a government agency include the agency's legislative mis- sion; legal requirements affecting the decision-making process (e.g., Ad- ministrative Procedure Act); the degree to which the decision-making

OCR for page 133
146 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOC - TIC SOCIETY authority has been legally delegated; the demand for evidence in terms of burden-of-proof requirements; expectations regarding legal challenges; requirements for documentation of the process and for tangible analytic outputs; and the legal responsibilities of interested parties both before and after the decision. Because of the importance of broadly based deliberation, an agency should avoid taking the stance that a statute prohibits public input simply because it gives the agency full decision-making responsibility. Rather, it should develop a clear understanding of how much statutory discretion it can exercise in order to listen to issues as needed without abdicating responsibility. Agencies should avoid giving the impression that they will prolong a process until all participants are satisfied or that listening to the interested and affected parties is equivalent to a commitment to decide in favor of those who testified. 3. Describe the Purpose of the Risk Decision The responsible organization's staff should describe the stated and implicit purposes of the decision-making activity" the type of decision and general aims furthered by the activity, and the intended users of the risk characterization. Different types of decisions may require different types of knowledge and perspectives and hence require different partici- pants in the analytic-deliberative process-both inside and outside the organization. Consider the variety: Is the decision about risk analysis technique (e.g., selection of default assumptions), about guidelines for making inferences from data, about regulating an industrial process, about setting an emissions standard, about taxing emissions and efflu- ents, about establishing cancer potency values, about informing individu- als at risk, about policy strategies, or about implementation? Different kinds of decision also affect different parties, whose concerns the process must satisfy and whose participation or representation it may require. Staff should identify the types of decisions that will probably follow the risk characterization and consider how the particular risk character- ization activity will facilitate the decision and the overall aims of the organization. It is important to also consider possible secondary and tertiary impacts of the decision, which may be of more concern to inter- ested and affected parties than the primary one. 4. Describe the Affected Parties and Likely Public Reactions Diagnosis should consider the identity and likely positions and per- spectives of the interested and affected parties. The first step is to provide a tentative identification of the parties and any barriers there might be to

OCR for page 133
IMPLEMENTING THE NEW APPROACH 147 their effective involvement in the analytic-deliberative process. As al- ready noted, the success of the process depends on the satisfaction and appropriate participation of these parties. It may be difficult to achieve meaningful participation of some of the parties in those parts of the pro- cess involving discussion of complex technical issues because of lack of expertise and funding of those parties, because they mistrust the respon- sible organization and are unwilling to participate, or because they trust the organization implicitly and see no need to participate. Government agencies should resist the temptation to exclude parties whose views are known to be different from those of agency administrators. Diagnosis should consider the possibility that some affected parties may resist participation because they believe they are more likely to achieve their desired outcomes by some other strategy, such as a legal challenge. Diagnosis should result in tentative recommendations on how to address any such problems that seem likely to become significant. (We discuss some of the possibilities in Chapter 3.) Diagnosis should also consider the potential for controversy. Which parties are likely to support or oppose a possible decision? How might they try to exert influence indirectly? How strong is the public consensus on the need to address the hazard? What type of press coverage can be expected? It is important to assess the potential for "rough weather," consider the organization's possible responses to it, and lay the ground- work for a strategy for addressing external pressures. Informal contact with interested and affected parties can provide valuable insight on these issues and on how to address them. In some cases, a reorientation of the analytic-deliberative process can help reduce controversy or channel it more productively. The indicators of strong potential for public controversy include: There is a clear distinction between those people who benefit from one of the decision options and those who do not. The potential is stron- gest when the distinction reinforces preexisting social divisions (e.g., rich versus poor, workers versus employers, regional or racial differences). Recent decisions that are similar to this one evoked controversy. The responsible organization suffers from a low level of public trust or lacks a public constituency. The hazard is characterized by a high degree of dread or the poten- tial for widespread, involuntary exposure. The issue presents news "hooks" that attract media attention or vivid opportunities for use in larger debates (e.g., incinerator siting and the debate on pollution prevention strategies; the presence of endangered species).

OCR for page 133
148 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY There are numerous other factors that may also be worth considering (Foran et al., 1995~. The same factors also suggest the probable nature of the controversy that the organization may face. In our view, the effectiveness of a risk characterization will depend significantly on the prevailing climate of public opinion. It is therefore imperative for public officials to diagnose that climate so that they can arrange for appropriate participation and direct scientists to address is- sues likely to be raised as criticisms if they are omitted from the risk characterization. Conflicts over substance and process are closely related. Affected parties that raise objections to substantive conclusions or omissions in a risk characterization product are often also reacting to perceived failings in the process. If people do not trust the process, they have little trust in the outcome. Therefore, when public controversy can be anticipated, diagnosis should pay particular attention to the concerns of the affected parties and to including those parties in the process. We emphasize again that understanding the potential for controversy and designing an analytic-deliberative process accordingly are not enough to prevent some unwanted outcomes. Deliberations across the range of decision participants may fail to reach consensus and, sometimes, inter- ested or affected parties may choose to exert influence indirectly and outside the officially designated process, such as through litigation, legis- lation, or mass media publicity. Even with good diagnosis and planning, such reactions may occur, but they are likely to be less incapacitating. 5. Estimate Resource Needs and Timetable Diagnosis should consider the internal and external time pressures on the decision and the extent to which they are explicit. It should con- sider the public health and other consequences of indecision, and if sig- nificant, the risk-reduction potential of interim actions during a lengthy decision-making process (Harris, 1990~. It should consider whether addi- tional resources might be made available or whether resources might be cut. The adequacy of resources depends in part on the range of expertise needed for analysis and on the potential for controversy. Evaluating the timeline in terms of the legislative, budgetary, and executive election cycles is a must for complicated decisions by government agencies. 6. Plan for Organizational Needs The diagnosis should result in a plan that specifies the organizational support needed for the risk characterization. When a government agency is responsible, it should consider the needs for coordination between the

OCR for page 133
IMPLEMENTING THE NEWAPPROACH 149 program unit given initial responsibility and other units within the agency, other agencies, and other levels of government. It should explic- itly consider the need for early coordination with units that maintain regular contact with interested and affected parties (e.g., risk communica- tion programs). Several diagnostic steps can benefit from the input of in- house experts, as well as the informal input from interested and affected parties themselves. The diagnostic process should consider the need for a task force or some similar entity, focused on the particular risk decision, that cuts across the usual organizational structure. If this sort of coordi- nation is needed, it should be implemented early. 7. Develop a Preliminary Process Design The diagnosis should result in a clear proposal for the steps of the analytic-deliberative process, their sequence, expected iterations, partici- pants, rules for closure and other decisions, and tangible products. The plan should be open for discussion by the affected and interested parties once the process begins, and it should be changeable as needed. It should consider the legal and resource constraints on the process, where and how affected and interested parties can participate, time commitments, and overall time frame. It should also clearly specify whether tangible products will be needed describing the risk and documenting the process. 8. Summarize and Discuss Diagnosis within the Organization Both "risk managers" and "risk assessors" should be actively engaged in all parts of the diagnosis. The discussion and review of the many judgments involved in the diagnosis will help to surface potential prob- lems within the organization, clarify the degree of commitment the orga- nization should make to the activity, and ensure that the organization enters the process with a consistent position on what it is willing to do in terms of participation, deliberation, and other potentially contentious is sues. Conclusion Diagnosis should result in a commitment within the responsible or- ganization, among both staff and management, regarding the nature and level of effort of the analytic-deliberative process leading to a risk charac- terization. Especially when the risk analysis is expected to be complex and difficult and there is likely to be polarization and politicization about the risk decision, an understanding among both staff in the organization and the interested and affected parties that the organization is committed

OCR for page 133
150 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMO CRITIC SOCIETY to the activity is essential. The diagnostic effort should therefore result in explicit expectations about the extent of the activity and the kinds of support and constraints that will come from the organization. The responsible officials should treat the diagnosis as tentative. One of the greatest dangers of diagnosis is that it may convey a sense that the problem formulation, the process design, and other aspects of the ana- lytic-deliberative process are firmly established. Diagnostic efforts are subject to the test of experience. An unwillingness to modify preliminary decisions can undermine the larger purpose of making risk characteriza- tion responsive to the emerging needs of the decision makers and the interested and affected parties. Officials of the responsible organization should always keep in mind that their goal is a process that leads to a useful and credible risk characterization. BUILDING ORGANIZATIONAL CAPABILITY Implementing effective risk characterization requires appropriately structured and staffed organizations and systematic efforts to improve the knowledge base for designing and managing the analytic-deliberative process that informs risk decisions. Organizational Issues Implementing a broadly based analytic-deliberative process for risk characterization makes demands on an organization. It must assign re- sponsibility and authority for diagnosing risk decision situations and for implementing new analytic-deliberative processes, and it must create pro- cedures for reviewing and approving its process decisions. The relevant staff must understand the underlying concepts, which may require spe- cial training. The organization must be prepared to respond appropri- ately to requests to analyze dimensions of risk it has not analyzed before and to acquire the necessary expertise to do so. It must also be prepared to cope with the possibility of attempts by some of the interested and affected parties to delay a decision, and it must develop a range of strate- gies for reaching closure on decisions that affect the process leading to risk characterization. Broadening the process also requires new kinds of coordination be- tween the organizational units normally responsible for risk analysis and those responsible for interactions with interested and affected parties. This coordination may require changes in organizational structure or pro- cedures. It may be advisable in some situations to establish task forces or working groups that cut across organizational units so as to involve all the relevant units in the analytic-deliberative process, beginning with the

OCR for page 133
IMPLEMENTING THE NEW APPROACH 151 initial decision to characterize a risk, and to maintain communication throughout. For some organizations, a broad concept of risk characterization may require changes of procedure to permit flexibility and judgment. Those responsible for managing the process that leads to a risk characterization must be allowed the flexibility necessary to match the process to the deci- sion situation. Their organizations will need to develop ways to allow that flexibility and at the same time guard against arbitrary decisions and undue influence from interested parties. Replacing a reliance on stan- dard procedures with a more flexible system will require care both in assigning responsibility and in establishing safeguards. Perhaps most important, organizations should develop mechanisms that provide feed- back on their procedures so that they can be improved over time. Some agencies consider reviewing deliberative innovations to be integral to their success and improvement (Fisher, Pavlova, and Covello, 1991; Young, Williams, and Goldberg, ~993; Crumbly, 1996~. (On organiza- tional learning related to risk decision making, see Short and Clarke, 1992; Chess, Tamuz, and Greenberg, 1995.) Improving the Knowledge Base Only a very limited knowledge base exists for guiding decisions about the process that leads to risk characterization. Thus, organizations that modify their standard procedures and adopt a carefully designed ana- lytic-deliberative process must to a great extent find their own paths. This situation can be improved over time if explicit and systematic efforts are made to evaluate and learn from experience. Such efforts can help organizations conserve resources and solve problems in at least three ways: Gathering knowledge and feedback early and throughout the ana- lytic-deliberative process allows mid-course corrections that save time and money. Pretesting materials that summarize risk information reveals in advance whether this information is understandable and useful for the intended audiences. Retrospective analysis can suggest ways to improve future efforts (Office of Cancer Communications, 1989; Fisher, Pavlova, and Covello, 1991). Ideally, an organization involved in a major analytic-deliberative process will devise systems of feedback and evaluation to inform it both during and after the process. In addition, institutions that provide scientific

OCR for page 133
152 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIETY support for many of these organizations, such as federal scientific agen- cies and industry-based research institutes, should support systematic efforts that build knowledge about analytic-deliberative processes that may have general value for many organizations. One of the difficulties in building knowledge about analytic-delibera- tive process is defining criteria for success. We do not suggest that inter- ested and affected parties or organizational managers must all be "happy" at each step of the process for it to be considered successful. A more realistic approach is to define criteria for success early on, at or before the step of process design, using a process in which both interested and af- fected parties and organizational staff participate (Rosener, 1981~. We believe that asking questions like those listed below will yield valuable insights that can be used to develop realistic expectations for the analytic- deliberative process and to arrive at a working definition of success or effectiveness. Criterion Measurement procedure Getting the science right Getting the right science Getting the right participation Getting the participation right Developing accurate, balanced, and informative synthesis Ask risk analytic experts who represent the spectrum of interested and affected parties to judge the technical adequacy of the risk- analytic effort Ask representatives of the interested and affected parties how well their concerns were addressed by the scientific work that informed the decision Ask public officials and representatives of the interested and affected parties if there were other parties that should have been involved Ask representatives of the parties whether they were adequately consulted during the process; if there were specific points when they could have contributed but did not have the opportunity Ask representatives of the parties how well they understand the bases for the decision; whether they perceived any bias in information coming from the responsible organization Evaluation or feedback should take a form appropriate to the scale and nature of the analytic-deliberative process: a resource-intensive risk characterization will merit more rigorous and extensive evaluation than a more limited one. Evaluation efforts may use quantitative or qualitative methods and aspire to different degrees of rigor as the situation demands. (For detailed discussion of principles and methods of evaluation for deci

OCR for page 133
IMPLEMENTING THE NEW APPROACH 153 signs involving broadly based participation, see, e.g., Sewell and Phillips, 1979; Rosener, 1981; Fisher, Pavlova, and Covello, 1991; Syme and Sadler, 1994.) Generally, conscious efforts at evaluation both during and after the process are important for improving analytic-deliberative processes. Feedback and evaluation can begin in the diagnosis phase, when the responsible organization begins to define the resources it will need to develop a risk characterization and to develop a preliminary process de- sign. Process evaluations that seek data for midcourse corrections should be common practice. The U.S. Department of Energy, for example, has developed a practice in which citizens' advisory committees do self-as- sessments on at least a yearly basis (Grumbly, 1996~. Surveys have also been used to solicit feedback before and during efforts to involve commu- nities in problem solving (e.g., Pflugh, no date). Informational materials can be pretested for comprehensability and relevance and to ensure that they deliver the intended message, as well as being technically accurate (e.g., Office of Cancer Communications, 1989; Morgan et al., 1992~. We strongly suggest that risk characterization messages developed for wide distribution undergo pretesting with the intended audience. Other innovative approaches may also be appropriate in certain situ- ations. One is to conduct simulations to illuminate particular issues. Simulations mimic actual decision processes but are conducted outside the decision itself. They can be used to examine the ways that delibera- tive groups formulate problems, identify the outcomes that require analy- sis, and interpret scientific information. Simulations can be useful for identifying potential problems with existing processes and for suggesting particular approaches that might be tried in real decision contexts. (For an illustrative example, see Hester et al., 1990~. Organizations may also use quasi-experimental evaluation designs to compare different proce- dures, for instance, for integrating analysis into deliberations, for discuss- ing scientific information in a diverse deliberative group, for arriving at particular judgments to be incorporated as assumptions in risk analyses, or for reaching closure. In this approach, an organization treats its inno- vations as experiments and studies their effects in comparison with other procedures. An organization might also use what it believes will be an exemplary process and gather data on it. If the first attempt is successful, it might become a benchmark for future processes; regardless of its de- gree of success, it can become a baseline for learning and improvement. An organization can use informal feedback or combine formal and informal evaluations. For example, it might organize broadly based advi- sory groups to review the processes it has used to make the judgments at each step leading to risk characterizations. The advisory groups could consider whether it appears with hindsight and with the findings of evalu- ation research that the analytic-deliberative process might have been more

OCR for page 133
154 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMO CRITIC SOCIETY effective if conducted differently. An organization, with the help of its advisory group, might then consider what lessons past experience holds for future risk characterization efforts. Organizations might also build and update libraries of case files that use standard categories to record the processes used to make the judgments that affect its risk characteriza- tions. These files would provide a basis for systematic case study re- search on the organization's past experience and for review by an advi- sory group. CONCLUSION A broadly based analytic-deliberative process may require organiza- tions that characterize risks to engage in new and unfamiliar activities. Although there are legitimate concerns about the practicality of making such changes, we see good reason to do so, while testing the effects on the overall process and implementing safeguards against attempts at tactical delays. Although there will be initial increases in the cost in money and time for some risk analyses and characterizations, the overall costs of risk decision making may actually decrease. It may cost less in the long run to do it right the first time. Successful implementation depends on matching the analytic-delib- erative process to the needs of the decision. This requires a clear under- standing of the decision milieu. Although there is no standard procedure for doing this, organizations can benefit by asking a series of diagnostic questions when they plan the process and by keeping their diagnoses flexible and responsive to information that emerges during the process. Implementation may also require organizational efforts at staffing and training and organizational changes to permit the necessary coordination among units and to allow flexibility in the processes informing risk deci- sions. There is also a need for evaluation research to improve the analyti- cal-deliberative processes on which risk characterization depends.