Click for next page ( 156


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 155
Principles for Risk Characterization Government agencies and other organizations are increasingly making decisions in which they explicitly consider risks of harm to human health, safety, and well-being and to nonhuman organisms and ecological systems along with the other con- siderations that enter into those decisions. In so doing, they have relied increasingly on analytic techniques developed to establish a solid factual basis of understanding of those risks. Such risk analyses sometimes yield information of. a type or in a form not directly useful to decision makers. The term risk characterization is commonly used to describe efforts to make the state of knowledge relevant to a risk decision intelligible to decision participants who may or may not be expert in the techniques of risk analysis. We undertook this study to advise federal government agencies and others on ways to improve those efforts. Our study has led to a conception of risk characterization as the prod- uct of a decision-driven, analytic-deliberative process and to a set of prin- ciples for organizing the process. The purpose of risk characterization is to improve the understanding of risk among public officials and inter- ested and affected parties in a way that leads to better and more widely accepted risk decisions. 1. Risk characterization should be a decision-driven activity, directed toward informing choices and solving problems. Scientific efforts in support of risk analysis have sometimes been criti 155

OCR for page 155
156 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIETY cized for being of little help for decision making, even when they have added to scientific knowledge. Effective risk characterization must accu- rately translate the best available information about a risk into language nonspecialists can understand. But it must also do more. It must address the right questions-the ones that the various participants in risk deci- sions want answered as a basis for making choices and it must give those parties an understanding of the many facets of risk. Good risk characterization results from a process that not only gets the science right-that is, involves an adequate level of scientific inquiry and analy- sis but also gets the right science-that is, directs that analysis to the most decision-relevant questions. Risk characterization serves the needs not only of the designated decision makers but also of the spectrum of parties that participate in risk decisions. Although risk characterizations are often completed for the benefit only of an organization's decision maker, it is important to recognize that vari- ous other parties have a right to participate in the decision and may do so, either before or after the organization acts. These parties include legisla- tors, judges, industry groups, environmentalists, citizens' groups, and a variety of others. Acceptance of risk decisions by the interested and affected parties is usually critical to their implementation. Satisfactory risk characterization processes and products provide all the decision par- ticipants with the information they need to make informed choices, in the form in which they need it. A risk characterization that fails to address their questions is likely to be criticized as irrelevant or incompetent, re- gardless of how carefully it addresses the questions it selects for attention. Risk characterization should not be an activity added at the end of risk analysis; rather, its demands should largely determine the scope and nature of risk analysis. It is well recognized that risk characterization depends on good scientific analysis. It is not so well appreciated that risk analysis depends on risk characterization: that the need for characterization to be decision relevant determines which analyses are worth doing. Risk char- acterization requires a solid scientific base, but it will fail if it does not incorporate the knowledge and perspectives of the various participants in decisions and seriously address the issues they see as critical. Conse- quently, risk analysis should not proceed very far as the task only of analysts. The scientific agenda should also be guided by and periodically recalibrated on the basis of input from the interested and affected parties. 2. Coping with a risk situation requires a broad understanding of the relevant losses, harms, or consequences to the interested and affected parties.

OCR for page 155
PRINCIPLES FOR RISK CHARACTERIZATION 157 Risk analyses are often restricted to the examination of a narrow set of outcome conditions: certain human health hazards and sometimes a nar- row range of effects on ecosystems or on economic interests. Such a focus is sometimes defended on the ground that the outcomes examined are the most serious ones that meet some a priori definition of "risk." This is not an appropriate justification because risk characterization is a goal-directed activity the success of which depends on the satisfaction of decision par- ticipants. Relevance to a decision, and therefore to a risk characterization, cannot be determined a priori by a formal definition of risk. The out- comes that should be considered relevant depend on the decision. Risk characterizations should, when appropriate, address social, economic, ecological, and ethical outcomes as well as consequences for human health and safety. Human health and safety are often the only important conse- quences to the interested and affected parties, but for some decisions, other outcomes are as important or even more so. Some of the interested and affected parties will feel inadequately informed unless these other outcomes are addressed. A wide range of outcomes is amenable to sys- tematic analysis, although many of them require additional expertise to that needed for assessing outcomes for human health and safety. Even when a decision-relevant outcome cannot be analyzed in a systematic and replicable way, it should still be addressed in the risk characterization so as to avoid leaving the impression that the outcome has been judged to have zero risk. Risk characterizations should, when appropriate, address outcomes for particular populations in addition to risks to whole populations, maximally ex- posed individuals, or other standard affected groups. Depending on the deci- sion at hand, adequately informed choice may require that risks be char- acterized for certain specially exposed or vulnerable populations, such as children, members of particular occupational groups, residents of highly exposed areas, those with increased susceptibility due to genetic or envi- ronmental factors, people with compromised health, and groups defined by race, ethnicity, or income. Adequate risk characterization depends on incorporating the perspectives and knowledge of the spectrum of interested and Affected partiesfrom the earliest phases of the effort to understand the risks. If a risk characterization is to illuminate the relevant facets of a risk decision and be credible to the interested and affected parties, it must address what these parties believe may be at risk in the particular situation, and it must incorporate their specialized knowledge. Often, the best way to do this is by the active involvement or representation of the parties. The breadth of analysis and the appropriate extent of involvement or

OCR for page 155
158 UNDERSTANDING RISK: INFORMING DECISIONSINA DEMOCRATIC SOCIETY representation required for satisfactory risk characterization is situation depen- dent. Many risk characterizations can be satisfactorily completed by ana- lyzing only a few outcomes and with little direct involvement of inter- ested and affected parties, but others require a more inclusive and more participatory approach. The level of effort required is situation specific. What is always required is that the necessary breadth of analysis, charac- terization, and involvement be considered explicitly for each step of the process and not made by default. 3. Risk characterization is the outcome of an analytic-deZibera- tive process. Its success depends critically on systematic analy- sis that is appropriate to the problem, responds to the needs of the interested and affected parties, and treats uncertainties of importance to the decision problem in a comprehensible way. Success also depends on deliberations that formulate the deci- sion problem, guide analysis to improve the decision partici- pants' understanding, seek the meaning of analytic findings and uncertainties, and improve the ability of interested and affected parties to participate effectively in the risk decision process. The analytic-deliberative process must have an appro- priately diverse participation or representation of the spectrum of interested and affected parties, of decision makers, and of specialists in risk analysis, at each step. Risk characterization requires a sound scientific base, supported by systematic analysis. Of critical importance is maintaining the integrity of the analytic process; in particular, protecting it from political and other pressures that may attempt to influence findings or their characterization so as to bias outcomes. Analysis, like all of risk characterization, should be decision driven and aimed at a comprehensive understanding of relevant factors. Analy- sis includes not only the use of systematic methods from the physical, mathematical, and health sciences, but also, whenever relevant for under- standing, analytic methods from the social sciences, ethics, and law. The best available analytic methods should be used, whether quantitative or qualitative, within limits of effort determined by the degree of detail or precision appropriate for the decision. Analysis should address the ques- tions that decision participants need to consider in order to make informed choices, which may require conducting analyses beyond the ordinary scope of risk analysis. Simple and narrowly focused analytic procedures may be appropriate, however, for large numbers of routine decisions if such procedures are justified by prior analysis and delibera- tion and are subject to appropriate review (see Chapter 6~.

OCR for page 155
PRINCIPLES FOR RISK CHARACTERIZATION 159 Uncertainty should receive clear and comprehensible treatment in risk characterization. Participants in risk decisions need to understand both the magnitude and the character of uncertainty: for example, whether it is due to inherent randomness, lack of knowledge, or disagreements of theories, models, values, or perspectives. Useful analytic methods exist for characterizing certain types of uncertainty. They should be focused on the uncertainties that matter most to the decision. Moreover, they should be used with great care. Pervasive and persistent cognitive biases, as well as a variety of social factors, can exert pressure toward misper- ceiving uncertainty, even when it has been carefully analyzed. The evalu- ation of uncertainty can enlighten the decision process, identifying those studies and data collection efforts that can most effectively reduce the uncertainties that matter. Deliberation is as critical to risk characterization as analysis, although its importance has been underappreciated. Deliberation is needed to frame, and where necessary reframe, the decision problem, define the fundamental questions that risk characterization needs to address, set the research agenda, decide who will participate in the effort to build under- standing, identify the relevant information, settle on ways to gather the information, select assumptions to use when data are insufficient, and arrive at judgments about the degree of reliance that should be attached to the results of risk analyses and about the amount and kind of uncer- tainty these results contain. For potentially controversial risk decisions, deliberation should involve the spectrum of interested and affected par- ties to bring the analysis into better alignment with the parties' needs for information, choose more realistic and satisfactory assumptions based on specialized knowledge the affected parties may uniquely possess, and subject analyses to critical review from a fuller range of perspectives. Broadly based, appropriately participatory deliberation benefits under- standing by ensuring that analysis draws on the full range of relevant knowledge and perspectives available in the society, and it benefits the decision process by making it more inclusive and more credible, further- ing democratic norms. Deliberation in the context of risk characterization, even when highly participatory, differs from what is usually called "public participation" in three major ways. First, it precedes agency proposals and action: it is aimed at improving understanding of risk situations, as distinct from taking action on them. Second, because the deliberation is intended to improve understanding, the involvement of knowledgeable experts as well as "the public" is essential throughout. Third, deliberation is not merely a forum in which interested citizens can be heard, but a sympo- sium in which risk experts, public officials, and the various interested and affected parties can interact as equally valid contributors.

OCR for page 155
160 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIE~ Effective deliberation can affect the acceptance of risk characteriza- tions. It can clarify the areas of consensus and disagreement among inter- ested and affected parties about how the problem is framed, how data are interpreted, and what further analysis is needed, thus focusing the char- acterization on the issues agreed to be most critical to a risk decision. It can promote mutual exchange of information among interested and af- fected parties and increase mutual understanding. It can reduce prob- lems of mistrust if the responsible organization involves the spectrum of parties and responds to the participants' suggestions about the risk analy- sis. It can limit conflict by arriving at substantive and procedural agree- ments, such as about which assumptions to use in analysis or which tech- nical consultants to select. And it may help the participants learn ways to interact productively that they can employ in future analytic-deliberative processes. Organizations should startfrom the presumption that both analysis and deliberation will be needed at each step leading to a risk characterization. We are not advocating that explicit deliberation occur at every step of every pro- cess. It is sometimes appropriate, for example, to conduct generic delib- erations to arrive at analytic procedures that will then be used routinely in a large number of subsequent risk characterizations. However, ad- equate justification should be given for restricting deliberation. An orga- nization may show, for instance, that sufficient deliberation for the cur- rent purpose was done in establishing the appropriateness of an analytic routine being followed. Organizations should stars from the presumption that the spectrum of interested and affected parties will be involved in deliberations in each step lead- ing to a risk characterization. Organizations should consider ways to broaden participation and accommodate demands for it; the burden for justification should be placed on those who would restrict participation rather than on those who would broaden it. Full participation at each step is particularly important when the contemplated risk decision is ex- pected to be controversial or to have widespread and potentially serious effects on many people or areas. . Broadly based' appropriately participatory deliberation does not necessar- ily mean the inclusion of every interested and affected individual and group. Depending on the purpose of the deliberation, appropriately broad par- ticipation may be achievable through the use of surrogates or representa- tives who bring to the table knowledge, perspectives, and concerns of the parties that are relevant to the issue at hand. Broadly based, appropriately participatory deliberation will sometimes require that resources be provided to some of the interested and affected parties. Some of the parties to some risk decisions cannot afford the time, the

OCR for page 155
PRINCIPLES FOR RISK CHARACTERIZATION 161 travel, or the technical assistance they need to participate meaningfully in particular deliberations, and it will be necessary to provide such resources in order to obtain the benefits that deliberation can provide for informing risk decisions and increasing their acceptance. An elective analytic-deZiberative process depends on explicit attention to process design, especially concerning the deliberation and its integration with analysis. The deliberative parts of the process require as much advance planning as the analytic ones. Planning should consider who should be involved, the form the process might take, resource needs, timing, and the coordination of deliberation and analysis. 4. Those responsible for a risk characterization should begin by developing a provisional diagnosis of the decision situation so that they can better match the analytic-deliberati~re process leading to the characterization to the needs of the decision, par- ticularly in terms of level and intensity of effort and representa- tion of parties. Risk situations vary along many dimensions, and the same analytic- deliberative process is not appropriate for all risk characterizations. In particular, the level of effort that should go into problem formulation, process design, and the other elements of the analytic-deliberative pro- cess and into securing appropriately broad participation is situation dependent. Responsible organizations should seek to match the level of effort to the needs of the task. Past experience shows that government agencies and other organizations are more likely to err on the side of inadequate participation and too-narrow deliberation, a bias that should be reversed. Nevertheless, we do not advocate unlimited efforts to broaden the process. Diagnosis helps organizations use their resources efficiently and ef- fectively. An organization may decide that a decision can be informed quite adequately by following an existing standard procedure for charac- terizing risks. However, if this diagnosis is incorrect, the organization may generate avoidable ill will and make it difficult to get the needed participation later on. Organizations will do better to err on the side of broadening participation, beginning at the stage of problem formulation, when it may be possible to determine whether a simple procedure for the rest of the process will meet the needs of the parties. For many decisions, a simple, generic risk characterization procedure will suffice. Many risk decisions do not require extended attention to problem formulation, process design, and so forth within the analytic-deliberative process. With many routine regulatory approvals, for example, it is suffi

OCR for page 155
162 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMOCRATIC SOCIETY cient to design a single generic risk characterization process that can be used for a large number of specific decisions. However, when a simple or routine procedure is being contemplated, careful consideration should be given to the process design at the outset, the deliberation about process should presumptively involve the spectrum of interested and affected parties, and a commitment should be made to reconsider the procedure from time to time to ensure the adequacy and appropriateness of the routine and to check on the adequacy of representation of the parties. That is, a routine for informing risk decisions should be periodically re- viewed through an analytic-deliberative process. An inappropriate or inflexible decision to use a narrow, routinized, or nonparticipatory analytic-deliberative process for risk characterization can un- dermine the decision-making process. Some of the most contentious risk controversies have centered on claims of inadequate analytic attention to valid concerns or failure to meaningfully involve some of the interested and affected parties. Examples include controversies over siting hazard- ous waste facilities and over pesticide spraying in residential areas. Ex- plicit attempts to diagnose the risk decision situation and design the ana- lytic-deliberative process accordingly can make risk characterizations more credible and thereby reduce controversy. 5. The analytic-deliberative process leading to a risk character ization should include early and explicit attention to problem formulation; representation of the spectrum of interested and affected parties at this early stage is imperative. Some of the worst examples of risk decision making have roots in the way that problems were formulated for risk analysts. Difficulties arise predictably, and conflicts are exacerbated, when a large-scale analytical effort is addressed to a problem that some of the interested and affected parties do not recognize as the relevant one for the decision. It is therefore extremely important for the organizations responsible for risk decisions to investigate whether there are or might be competing definitions of the risk problem. Risk characterization can be fairly straightforward if the interested and affected parties agree on which issues deserve analysis; if they do not agree, it is often worth making special efforts at the outset to engage them in deliberation about what should be analyzed. We do not imply that extensive efforts at participatory problem defi- nition are always warranted. However, failure to make such efforts when they are appropriate can be extremely costly. When there are major dif- ferences among the parties in their understandings of the decision prob- lem, it is a serious mistake to proceed without addressing these differ- ences. When problem formulation is given short shrift, there may be a

OCR for page 155
PRINCIPLES FOR RISK CHARACTERIZATION 163 loss of understanding, a loss of credibility for the responsible organiza- tion, a serious and possibly avoidable escalation of controversy, delay or paralysis in the decision process, and subsequent economic and social costs of delay to government and society. 6. The analytic-deliberative process should be mutual and re- cursive. Analysis and deliberation are complementary and must be integrated throughout the process leading to risk character- ization: deliberation frames analysis, analysis informs delib- eration, and the process benefits from feedback between the two. As already noted, a typical criticism of risk characterizations is that the underlying analysis failed to pay adequate attention to questions of central concern to some of the interested and affected parties. This is not so much a failure of analysis as a failure to integrate it with broadly based deliberation: the analysis was not framed by adequate understanding about what should be analyzed. Risk characterization can also make the opposite sort of error, for example, by addressing decision options that careful analysis would show to be impracticable. This would not be so much a failure of deliberation as a failure to inform deliberation with good analysis. Risk characterization fails when analysis is not properly guided to address the informational needs of participants in risk deci- sions; it also fails when deliberation is not adequately informed by analy s~s. Analysis and deliberation ideally improve each other. Analysis en- hances deliberation by informing discussions with facts, predictions, and basic understanding of risk-generating processes. Deliberation enhances analysis in several ways. Deliberation among technical experts can help clarify areas of consensus and dispute and the underlying reasons. Delib- eration among public officials, analysts, and interested and affected par- ties can define needs for analysis and improved understanding. And deliberation can bring new information and new perspectives to analysis. Risk characterization benefits from mutual and recursive interaction be- tween analysis and deliberation and between analytic specialists and the other decision participants. Both analysis and deliberation have a place in each step leading up to risk characterization: formulating the problem, designing the process, selecting options and outcomes, generating and interpreting information, and synthesizing the state of knowledge. The organizations responsible for risk characterization need to give special attention to the role of delib- eration in each of these steps: whose input would advance the task, how

OCR for page 155
164 UNDERSTANDING RISK: INFORMING DECISIONS INA DEMO CRITIC SOCKET that input should be elicited, how it should be informed by analysis, and how it should feed into further analysis. The interplay between analysis and deliberation sometimes gives rea- son to revisit past decisions. Deliberation may identify an additional policy option whose effects on the risks need to be analyzed, or an analy- sis may identify a previously unrecognized aspect of a hazard, so that the meaning of risk information needs to be reinterpreted. In these and other ways, the analytic-deliberative process is recursive, recovering old ground, but with improved understanding. 7. Each organization responsible for making risk decisions should work to build organizational capability to conform to the principles of sound risk characterization. At a minimum, it should pay attention to organizational changes and staff train- ing efforts that might be required, to ways of improving prac- tice by learning from experience, and to both costs and benefits in terms of the organization's mission and budget. Organizations may experience difficulties in following these prin- ciples, particularly in regard to increasing input from some interested and affected parties, involving nonscientists in deliberations about risk analy- sis, broadening the range of adverse outcomes to consider in risk analysis, more fully integrating analysis and deliberation, and doing anything that appears to prolong the decision process or increase its complexity. We are sensitive to concerns about cost and delay, but note in response the massive cost and delay that have sometimes resulted when a risk situa- tion was inadequately diagnosed, a problem misformulated, key parties excluded, or analysis not integrated with deliberation. We believe that following the above principles can reduce delay and cost as much as or more than it increases them. As a general matter, we believe it is critical for organizations to have the capability to organize the full range of analytic-deliberative processes, including the broadly participatory ones that risk situations sometimes warrant. Organizations differ too much from one another to allow us to make any universal recommendations about how to establish and main- tain this capability, but we offer several points for organizations to con- sider: Having the capability to organize thefull range of analytic-deliberative processes may require special efforts to train staff. Training may be warranted to introduce concepts such as broadly based, appropriately participatory deliberation, integration of analysis and deliberation, and social and ethi- cal risk. It may also be useful for establishing good working relationships

OCR for page 155
PRINCIPLES FOR RISK CHARACTERIZATION 165 between agency units that have not previously collaborated successfully, but that must do so to integrate analysis and deliberation. It may be necessary to acquire analytic expertise with regard to ecological, social, economic, or ethical outcomes. Experts in analyzing human health risks are not usually expert in analyzing these other outcomes. Addi- tional experts should be involved when these outcomes are important to a risk decision. Having the capability to organize the full range of analytic-deliberative processes may require organizational changes. For example, it may be advis- able in some situations to establish task forces or working groups that cut across units of the organizational structure so as to involve risk analysts, policy makers, risk communication specialists, and others in the analytic- deliberative process, beginning with the initial diagnosis of the problem. It may also be helpful in some agencies to make organizational changes that facilitate creating such cross-cutting groups. Future risk characterizations will benef~tfrom organizations' evaluations of their current activities. Experience provides a good base for learning how to better diagnose risk decision situations and what works in each type of situation. Organizations involved in analytic-deliberative pro- cesses should devise systems of feedback and evaluation to inform them both during and after these processes, and institutions that provide scien- tific support for many such organizations, such as federal scientific agen- cies and industry-based research institutes, should support systematic efforts that build knowledge about analytic-deliberative processes and that may have general value for many organizations. Organizations that characterize risk should work with interested and affected parties to de- fine criteria for evaluating the process leading to risk characterization. They should also consider implementing explicit practices to promote systematic learning from their efforts to inform and make risk decisions. These might include establishing broadly participatory panels or advi- sory groups to review past analytic-deliberative processes, building li- braries of case files that use standard protocols for describing past efforts and their outcomes so that experience can provide a basis for learning, conducting formal evaluation research projects to understand and learn from the ways the outcomes of analytic-deliberative processes are af- fected by how the processes are organized, and using simulations and quasi-experimental research to gain deeper understanding of the ana- lytic-deliberative process. The breadth offocus and participation of analytic-deliberative processes should be considered in terms of the potential benefits and costs to an organization's mission and budget, and to society. There are obvious poten- tial costs to making risk characterization broader in terms of who partici- pates, which risks are examined, and how extensively deliberation is inte

OCR for page 155
166 UNDERSTANDING RISK: INFORMING DECISIONS IN A DEMOCRATIC SOCIETY grated into the process. These costs are mainly in terms of time and money. The potential benefits, though sometimes less obvious and im- mediate, may be considerable for controversial or wide-impact decisions. They may include decreases in the time and money it takes to reach a final decision (even if it takes more time and money to reach the organization's policy decision); improved credibility for the organization; and more widely accepted decisions. In opting for a broader analytic-deliberative process, an organization may be required to accept monetary and other tangible costs to gain nonmonetary and intangible benefits, immediate costs to avoid greater future costs, or administrative costs to gain societal benefits. We are confident that a conscious and careful application of an ana- lytic-deliberative approach will lead to better risk characterizations and better risk decisions.