Improving Environmental Decision Processes
Federal agencies should support a program of research in the decision sciences addressed to improving the analytical tools and deliberative processes necessary for good environmental decision making. This research effort would have three components: (a) developing useful criteria to characterize and evaluate the quality of environmental decisions, (b) developing and testing formal decision science tools for structuring decision processes, and (c) building and testing concepts and practice for broadly based analytic-deliberative processes. Basic research on decision processes in individuals, groups, and organizations provides an essential foundation for this science priority. The National Science Foundation has supported such research in the past, sometimes in conjunction with the U.S. Environmental Protection Agency (EPA), and we expect support for basic decision research to continue. Our emphasis here is on research that would employ and advance basic understanding for the practical objective of improved environmental decision making.
THE RESEARCH NEED
Individuals, organizations, and ultimately societies, through their choices, have significant effects on the natural environment. The large human footprint on the Earth and the potential for huge mistakes make it imperative that the major choices are well informed and adequately considered. Decisions affecting environmental processes, however, are among the most challenging facing humanity because of the following collection of attributes that environmental choices usually share:
Structural complexity: choices affect phenomena that operate at multiple scales; decision-making entities also exist at multiple scales, not necessarily matched to those of the phenomena; and many different kinds of expertise are required to understand the issues.
Multiple, conflicting, and uncertain values: people affected by the choices have deeply held values often tied to spiritual, cultural, stewardship, or equity concerns that they are unwilling to negotiate or trade off; people differ in their value priorities; and sometimes their values seem to shift unexpectedly.
Long time horizons: the consequences of choices made now may extend for decades or longer.
Open-access structure: it is often difficult to exclude people from using or polluting a resource, putting that resource at considerable risk of overuse and decline (see Chapter 3).
Incomplete and uncertain knowledge: the consequences of choice options may be unknown or in dispute among scientists; they may also be dependent on ongoing processes of social or environmental changes that are also little understood.
High stakes: the long-term implications of the wrong choice for environment and society may be profound.
Time pressure: decisions must be made without waiting for scientific certainty or agreement on values.
These points are well recognized by observers of environmental decision processes (e.g., Funtowicz and Ravetz, 1992; National Research Council, 1996; Dietz and Stern, 1998; Renn, 2003). A further challenge is to address the linked nature of environmental processes and environmental decisions across time scales, physical scales, and institutional scales. Decisions made at one scale can be transformed or undermined by processes at other scales, which must therefore be taken into account. Researchers have only recently given serious consideration to this challenge to environmental decision making and management (Cash and Moser, 2000; Young, 2002; Berkes, 2002; also see Chapter 3).
In addition, environmental choices are affected by decision makers’ attention to various environmental or other aspects of the choices. Individuals’ apparent preferences shift depending on how choices are framed and on their interpretations of and affective reactions to information (Tversky and Kahneman, 1981; Kahneman, Ritov, Jacowitz, and Grant, 1993; Slovic, 1995), and the apparent priorities of organizations and governments shift as a function of how interested parties shape decision agendas (Kingdon, 1987).
Decisions of such difficulty require a variety of inputs. Elected representatives, who are normally entrusted with making value choices, rarely
have sufficient expertise to make well-informed decisions, but scientific and technical experts are not well suited or trusted to address the value issues. To meet the challenges, scientists have developed numerous analytical tools to inform decision makers about the functioning of environmental systems and the likely consequences of available choices. Mathematical models of complex systems represent the multiple layers and linkages that constitute environmental systems and forecast the consequences of interventions in them. Risk analysis techniques characterize undesired outcomes and the uncertainties that surround them and estimate their probabilities of occurrence. Various techniques based in economics quantify outcomes in terms of costs and benefits, compare outcomes occurring at different times in the future, and aggregate the outcomes facing individuals into measures of net societal outcome. Spatial analysis and mapping techniques combined with ground-based or remote observation represent environmental change and its effects.
Such analytical tools help address several of the challenges of environmental decision making, but not all. In particular, they often fail to meet the challenges of value conflict and uncertainty. Value choices are often hidden in the simplifying assumptions of analytic techniques, and the assumed values may not be universally shared. Overreliance on analytic tools without adequate consideration of their limiting assumptions can sometimes heighten mistrust of governments and their experts and make it difficult to get public acceptance of public policy decisions.
Another problem that often arises with environmental analysis is a failure to address key decision-relevant questions. For example, a billion-dollar research program to assess the cancer risks of dioxin not only failed to resolve the scientific issues but also may not have been asking the right question, which, for many affected people, concerned the overall health risks to groups exposed to multiple hazardous chemicals, not just the cancer risks of dioxin exposure. In all the likely decision contexts in which these risks matter, dioxin is only one among many hazardous chemicals involved and cancer is only part of the problem (National Research Council, 1996). Several billions were spent to characterize the risks of leakage of radioactive materials from proposed repository sites for high-level radioactive waste, but no comprehensive appraisal was done to compare these risks with the risks of continuing current practices of temporary waste disposal (National Research Council, 1995; 2001a). Assessments of acid precipitation, including the U.S. National Acid Precipitation Action Program and similar assessments in several other countries, have been criticized for overemphasizing the collection of new data and not doing enough to interpret existing data to understand their implications for societal decisions (U.S. Office of Science and Technology Policy, 1983; U.S. Office of Technology Assessment, 1984; Oversight Review Board, 1991; Cowling, 1992;
Cowling and Nilsson, 1995). In short, when science is gathered to inform environmental decisions, it is often not the right science. Among the consequences are heightened social conflict, delayed decisions, and mistrust.
Because of these pervasive problems at the junction of environmental analysis and decision making, several authoritative studies have recommended processes that integrate analysis with broadly based deliberative processes involving the range of parties interested in or affected by the decisions (e.g., National Research Council, 1996; Presidential-Congressional Commission on Risk, 1997; Canadian Standards Association, 1997; Royal [UK] Commission on Environmental Protection, 1998). The goal is to put analysis more directly into the service of those who may be affected by decisions. In these analytic-deliberative processes, participants with diverse perspectives and values contribute to decision making in many ways, including defining the environmental decisions that require analysis, framing the scientific analyses needed to gain insight into the decisions, and interpreting the results to illuminate the decisions at hand (National Research Council, 1996).
Public agencies in the United States and elsewhere are increasingly committed to an analytic-deliberative approach to environmental understanding. For example, EPA has made extensive efforts to implement and improve “science-based environmental stakeholder processes” in support of its decisions (U.S. Environmental Protection Agency, 2001). The U.S. Climate Change Science Program, possibly the largest single environmental science program in the federal government and one involving 13 federal agencies, adopted “decision support” as one of four core approaches to meeting the program’s goals in its current strategic plan (U.S. Climate Change Science Program, 2003). Among the implications of this emphasis are “early and continuing involvement of stakeholders … in defining key science and observation questions” and “transparent public review of analysis questions, methods, and draft results” (pp. 111-112).
Good environmental decision making requires both improved understanding of human-environment interactions and improved understanding and management of decision-making processes. The research we recommend will complement past efforts in the former area with expanded effort in the latter, using a decision science approach. Its central purpose is to identify and continually improve techniques for guiding and organizing practical environmental decision processes so that they achieve more of the ideal qualities of good decisions.
WHAT IS A DECISION SCIENCE APPROACH?
A decision science approach analyzes decisions and the processes for making them (e.g., Raiffa, 1968; von Winterfeldt and Edwards, 1986;
Morgan and Henrion, 1990; Keeney and Raiffa, 1993; Kleindorfer, Kunreuther, and Shoemaker, 1993; Clemen, 1996). It considers the objectives of decision making and ways to evaluate decisions and decision processes against those objectives. It identifies the questions and kinds of information needed for good decision processes, and it develops knowledge about which decision processes are likely to produce desired outcomes.
Basic research in decision sciences has focused primarily on unitary decision makers. Normative decision theory concerns hypothetical rational decision makers; behavioral decision theory concerns actual individuals (see the more detailed discussion in Appendix B). A smaller body of research has addressed the additional complications that arise when decisions are made in groups or when they must take into account the different and often conflicting perspectives and values of the people a decision will affect. There is relevant work in the social psychology of small-group decisions (Levine and Moreland, 1998) and in studies of organizational decisions, mainly in business (March, 1997). There are also numerous case studies of actual environmental decision making, but few of these have systematically applied conceptual frameworks (e.g., Renn, Webler, and Wiedemann, 1995; Beierle and Cayford, 2002).
This priority emphasizes research on what has been called prescriptive decision making (Bell, Raiffa, and Tversky, 1988), which we define as a science-based practice concerned with helping people to make good decisions. As already noted, our focus on the prescriptive presumes the continuation of basic research efforts on decision making that provide an essential foundation for prescriptive research. A decision science approach to practical decision making begins by identifying the elements of a responsible and competent decision-making process. For example, an ideal decision process has been defined on normative grounds as one that includes the elements listed in Box 2-1. These elements include some that are strongly dependent on participants’ values (V) and some for which information from scientific and technical analysis (T) can provide participants with essential insight regardless of their values. Actual decisions vary widely in terms of how closely they approach these ideals.
Many analysts argue that environmental decision processes should be iterative to accommodate changing human desires and the changing state of knowledge about the effects of environmental choices. This is the notion of learning over time embodied in concepts of adaptive management and governance (Holling, 1978; Gunderson, Holling, and Light, 1995; Lee, 1993, Dietz et al., 2003; National Research Council, 1999a, 2004b).
From a decision science standpoint, good environmental decisions consider both physical and social phenomena—environmental processes, the available options, the effects of different options on environmental and social conditions, and so forth—and human values. Information about
V = Achieving the ideal is strongly dependent on incorporating participants’ values.
T = Scientific and technical analysis provide essential insight for achieving the ideal.
SOURCE: Adapted from Hammond, Keeney, and Raiffa (1999).
phenomena is often obtained from environmental scientists, health specialists, engineers, economists, and other experts on those phenomena. Information on values can legitimately come from a wide of array of interested parties. The need for both kinds of information is worth underlining because judgments about phenomena and about values are often intertwined, not least in the minds of analysts.
Good decisions require competent and socially acceptable ways to integrate information about phenomena with information about values (Hammond, Keeney, and Raiffa, 1999; Payne, Bettman, and Johnson, 1993; Kleindorfer et al., 1993). Decision science has developed some systematic techniques for doing this integration in ways that can be applied to environmental decision processes (Slovic and Gregory, 1999).
As already noted, environmental policy additionally involves diverse, conflicting, and changing values; substantial scientific uncertainty and ignorance; and often mistrust among participants. Consequently, judgments about information can be hotly contested. There may be disagreement about which technical information is needed and what its practical significance is, how to interpret uncertain or disputed information, how to make tradeoffs between desired outcomes, whether seeking more information will be worth the cost, and even about the nature of the decision to be made (National Research Council, 1994a, 1996:Chapter 2).
Decision science can help improve decision processes by making these
judgments more explicit and structuring the ways participants in decision processes make and consider their judgments. It can also help parse disagreements so that decision participants can distinguish those that might be resolved by more information and clear thinking from those that may also require bargaining, compromise, or other resolution strategies. Decision science approaches have long been applied to a variety of environmental decisions (e.g., U.S. Nuclear Regulatory Commission, 1975; Lewis et al., 1975; Crouch and Wilson, 1982; Travis, 1988; Cohrsen and Covello, 1989; Rodricks, 1992; Suter, 1993) and used in training environmental policy analysts (e.g., Morgan and Henrion, 1990; Clemen, 1996). For further discussion of its use in environmental policy, see National Research Council (1996, 2002d) and U.S. Office of Management and Budget (2003).
Box 2-2 lists some characteristics of good public-sector environmental decision processes that have been proposed on the basis of previous research. These aspects of decision process are considered important for several reasons: normative (they allow affected parties to exercise democratic rights), substantive (they generate better alternatives and choices), and instrumental (they increase the likelihood of timely implementation) (Fiorino, 1990).
A major challenge in applying a decision science approach to environmental decisions is the linked nature of these decisions. Prescriptive decision theory will need to be expanded from its past emphasis on one-time
SOURCES: Webler (1995); National Research Council (1996).
decisions with defined boundaries to decisions that require linkages among administrative or institutional levels that have implications at many physical scales, that have long time horizons, and that involve iteration. The application of decision science to problems of long-term adaptive management or governance is an important area for contributions. At the other extreme, applying decision science to time-pressured decisions, as during crisis, also presents an important challenge and opportunity.
AREAS OF RESEARCH
Developing Criteria of Decision Quality
The quality of an environmentally significant decision cannot appropriately be defined by its outcomes because those outcomes may be highly dependent on factors that are unknown or uncontrollable when the decision is made. A decision may be well informed and well considered given what is known but may lead to unfortunate results because the system was not fully understood or because of the outcomes of key uncertainties. Thus, it is necessary to have internal criteria for judging the quality of decisions—criteria that are not dependent on ultimate outcomes. It is reasonable to expect that, on average, higher quality decisions (those based on the best available information, careful evaluation, adequate consideration of uncertainties and plausible worst cases, and so forth) are more likely than lower quality decisions to lead to desired outcomes in an uncertain world. It also seems reasonable to expect that higher quality decisions will on average be more widely accepted. There is evidence, however, from energy decision making of a perverse and inverse relationship between the quality of decisions and their acceptance (Craig, Gadgil, and Coomey, 2002). The empirical relationships among different indicators of decision quality is a worthwhile research question.
Researchers have proposed numerous internal criteria for decision quality. Some of these are presented in Boxes 2-1 and 2-2. The problem of defining decision quality for practical environmental decisions, however, has not received the level of research attention it deserves. Both the normative and the behavioral traditions in decision science have difficulty with this problem. The main difficulty in applying normative decision theory is that, in realistic situations, one cannot assess every consequence of every possible alternative and evaluate them all against the values of each decision participant. Decision makers seek the best practice not in the abstract, but under constraints of real people’s cognitive capabilities, legislative mandates, limited time and resources, social conflict, and so forth: choices must be made and defended regarding what to include and what to exclude (for further discussion, see Clemen, 1996; National Research Council, 1996;
U.S. Office of Management and Budget, 2003). Researchers in the behavioral tradition typically do not address issues of decision quality, in part because their approaches emphasize characterization rather than evaluation and improvement.
The recommended research would build on recent efforts to develop empirically supported knowledge about ways to organize high-quality decision-making processes under realistic constraints (Renn et al., 1995; National Research Council, 1996; Beierle and Cayford, 2002; Webler, Tuler, and Krueger, 2002; Renn, 2003). It would inform the design of environmental decision processes by addressing questions such as these:
What criteria do people use to evaluate decision quality? To what extent do these criteria differ for people from different cultural, socioeconomic, or educational backgrounds or for people representing different positions in environmental controversies? To what extent do they depend on past experience with related decisions?
Which characteristics of decision processes are associated with judgments of decision quality by the participants or outside observers? Which characteristics are associated with confidence in decisions? With acceptance of decisions?
How do different ways of organizing decision processes fare in terms of the attention given to the elements of normatively good decision process (for example, do processes that involve more different stakeholders do a better job of identifying all relevant objectives, as is often claimed)?
How do different levels of attention to particular elements of good decision process affect assessments of overall decision quality?
How do different resolutions of trade-offs in the decision process (e.g., between getting more information and deciding quickly, or between broader representation and efficiency of decision) affect various indicators of decision quality?
To what extent do interventions designed to ensure that decisions address certain elements of good decision processes result in more positive assessments of decision quality? Which elements are most important to those judgments under which conditions?
When decisions are highly constrained in terms of time, attention, or legal requirements, which elements of good public decision processes are most critical to the quality of the decisions?
Are decisions of higher normative quality associated with preferred social and environmental outcomes?
How can research results concerning good decision processes and ways to promote them best be disseminated to the users of these results (e.g., government agencies, stakeholder groups, corporations, partnership groups)?
This research might be conducted by various methods, including structured comparisons of naturally occurring cases, simulation, modeling, and quasi-experimental field research. Because environmental decisions present the full range of difficulties in decision making outlined above, they provide a good test bed for research on decision quality more generally.
Developing Formal Tools for Structuring Decision Processes
Behavioral decision research shows that individual decision makers typically omit key elements of good decision processes and that their decisions suffer as a result (Slovic, Fischhoff, and Lichtenstein, 1977). People respond to complex tasks by using their judgmental instincts to simplify them in ways that seem adequate to the problem at hand. They respond to probabilistic information or questions involving uncertainty with predictable biases that often ignore or distort important information (Kahneman, Slovic, and Tversky, 1982). They have difficulty clarifying objectives (March, 1978), identifying all viable alternatives (Keeney, 1992), and structuring decision tasks (Simon, 1990). When asked to consider value trade-offs or select among alternatives, they employ heuristic reasoning processes that are susceptible to a variety of contextual or task-related influences (Payne et al., 1993). Hence, there are many reasons to expect that, on their own, individuals (including experts) will often fall short of the normative ideal in making choices about complex issues involving uncertainties and value trade-offs.
Decision-making groups can, in principle, identify more elements of any decision than individuals and correct for individual members’ errors, thus producing better decisions than individuals. Behavioral research on group decision processes indicates, however, that this potential is not necessarily realized in practice. Individuals who have relevant information that is not widely shared in the group must get others to take this information seriously. Whether this happens is highly contingent, depending for instance on the individual’s social status (Hastie et al., 1983; David and Turner, 1996), the group’s norms (e.g., a value on originality opens the group to individuals’ information, a value on agreement closes it; e.g., Moscovici, 1985), and the tactics the individual uses to propose the ideas (Turner, 1991). Moreover, social processes in groups sometimes lead to premature closure on a common perspective that ignores contrary information, resulting in tendencies toward “group think” (Janis and Mann, 1977) or group polarization (Kaplan and Miller, 1987).
The decision sciences have developed a variety of tools to structure decisions and help decision makers and decision-making groups better approximate ideals of good decision processes. The recommended research will refine these tools and apply them more widely as a basis for improving
environmental decisions. To illustrate the possibilities, we briefly describe progress on developing tools for three purposes: clarifying decision participants’ values and preferences concerning alternatives, understanding and thinking through uncertainties and disagreements about the implications of choice options, and assisting in making collective choices when different individuals have conflicting understandings and competing preferences.
Clarifying Values and Preferences
The values and preferences people express regarding complex and unfamiliar environmental goods vary considerably according to how they are elicited (Payne et al., 1993; Slovic, 1995). Hence, the task of clarifying preferences is one of helping people construct their preferences rather than simply revealing them. Tools concerned with preference construction and elicitation include formal methods based on precepts of measurement theory and decision theory and wisdom gleaned from applied experience. Perhaps the most well-known of these formal approaches is termed multiattribute trade-off analysis, which involves an interview between an analyst and a decision participant (Keeney, 1992). The result is a mathematical statement comprising a utility or value function that could be used to evaluate every possible alternative within the range of consequences used in the interview process.
The advantages of formal techniques of preference construction are that the judgments involved are made explicit, that the value information can be used in many ways to help clarify the decision process, and that decision makers in collective choice situations can learn a great deal through joint efforts to clarify preferences. The disadvantages are also substantial: the questions involved are difficult to answer and require decision makers to make their inchoate feelings explicit, the questioning process may be confusing, the process can be cognitively and analytically demanding, and it may not be clear how the results will be used. This approach has other drawbacks, including the lack of trained people to implement such preference elicitation approaches, and the lack of a rigorous way to combine individuals’ utility functions into guidance for collective decisions, such as the kind of social welfare function provided by welfare economics.
In part as a response to these drawbacks, several other approaches to preference elicitation have been developed and tested by researchers in order to make the task cognitively simpler, more transparent, or more closely matched to particular decisions. These include the analytic hierarchy process developed by Saaty (1980, 1991); strategies that focus on a choice among a set of possible policy alternatives to address a given environmental question (McDaniels and Thomas, 1999); methods of clarifying preferences based on judgments of what would constitute “even swaps”
(Hammond, Keeney, and Raiffa, 1999); and techniques that emphasize key aspects of the decision problem, such as values (“value-focused thinking,” Keeney, 1992), particular objectives, or finding an alternative that provides acceptable performance across all the objectives (satisficing) (Payne et al., 1993). Such approaches are discussed in more detail in Appendix B.
Understanding Uncertainties and Disagreements
Decision scientists have been developing analytical tools and approaches for characterizing uncertainties. These include methods of eliciting and making use of probabilistic judgments or other sources of information about uncertainty, methods of combining probabilistic estimates through simulation, and methods of characterizing several different sources of uncertainty at once, all as a basis for estimating the effects of decision options (Morgan and Henrion, 1990; Cullen and Frey, 1999). Decision researchers have also experimented with methods of conveying information from these methods to decision participants as a basis for better understanding, deliberation, and decision making. Morgan and Henrion (1990) provide a comprehensive review of such methods and how they are applied for complex problems.
Although probability estimation remains the major approach to characterizing uncertainties, other methods are being developed that involve less demanding judgments. Some are based on fuzzy set theory (Zadeh, 1965), on the presumption that highly precise probabilistic judgments are often unnecessary. Scenarios offer another widely applied approach to characterizing uncertainties for environmental decisions (Waack, 1985a, 1985b), although relatively little research has been conducted on their efficacy as a means to generate an appropriate comprehension of uncertainty (see Chapter 6; Moss and Schneider, 2000).
Recent research using influence diagrams (a generic tool with wide application in model building, problem structuring, probability elicitation, knowledge mapping, and many other contexts) helps reveal the mental models of decision participants and the sources of some of their disagreements (Howard, 1989; Clemen, 1996). Influence diagrams can reveal differences in the understandings of lay and expert participants or between participants with different stakes in the decision (Morgan, Fischhoff, Bostrom, and Atman, 2002). They can help participants understand the bases of disagreements and perhaps see ways to resolve them.
Assisting in Collective Choice
One of the most difficult challenges in environmental decision making is how to arrive at a societal preference in a collective decision context.
Since the writing of Arrow (1963), decision researchers have recognized that there is no unique rule for aggregating the ordinal preferences of individuals with different values across a range of alternatives. Research is warranted on a variety of techniques that may be useful for informing judgments about societal preference. Voting approaches could possibly provide a means of directly eliciting preferences and under specific rules provide affording a basis for aggregation, although such approaches are highly affected by how questions are framed, the set of alternatives, and the choice of aggregation rules (Brams and Fishburn, 2002).
New information technologies may also provide useful tools for expression of individual preference, even if a generalized rule for social choice on the basis of expressed ordinal preference may be impossible. Computer-based tools for knowledge and value elicitation may provide widely applicable approaches to obtaining high-quality judgments about subjective probabilities of consequences and the values people associate with different consequences. Problem-structuring tools such as influence diagrams may have enormous potential in conjunction with advanced information technologies.
We are not recommending new research related to benefit-cost analysis, even though this approach is widely used to address the key issue of arriving at a social choice. We have two reasons for not doing so. One is that this line of research has its own momentum and seems less in need of increased research support than other, less developed areas. The current state of concepts and practice for benefit-cost analysis are discussed in several sources (e.g., Cropper and Oates, 1992; Zerbe and Dively, 1994; Morgenstern, 1997; Boardman, Greenberg, Vining, and Weimer, 2000; U.S. Office of Management and Budget, 1996; and Freeman, 1993).
The other is that benefit-cost analysis is in some respects antithetical to the research program recommended here because it decides by assumption how to address at least two important issues that we think need to be decided empirically. It assumes that social value is nothing more or less than the sum of the values individuals express in markets or market-like contexts, and it assumes that the values of different kinds of consequences (for employment, endangered species, sacred spaces, etc.) can be compared by reducing them to a single monetary metric.
There is growing literature documenting difficulties with these assumptions (Kelman, 1981; Morgan, Kandlikar, Risbey, and Dowlatabadi, 1999; Lave, 1996), particularly for large-scale problems involving long time horizons, nonmarginal changes, deeply held values, and equity issues. Moreover, because these assumptions are sometimes not shared by people affected by environmental decisions, attempts to employ them on actual environmental policy decisions have proved controversial and divisive (National Research Council, 1989, 1996). A large literature on perceptions of
justice and injustice, although not directly addressed to environmental issues, makes it clear that for people in the United States and several other countries, concepts of just decisions do not reduce to the choice that is best for the individual making the judgment and that individuals’ normative judgments about whether decisions are just can engender predictable emotional reactions (e.g., anger, resentment) that it may be risky for collective decision processes to ignore (see, e.g., Tyler and Smith, 1998; Miller, 2001; Mikula, 2003; Skitka and Crosby, 2003).
The research recommended here would investigate ways to structure decision processes, develop empirical understanding of the effects of various decision rules and analytical assumptions, and identify ways to structure decisions that help actual decision processes more closely approach normative ideas of good decision making. Research on formal tools for structuring decision processes might address such questions as these:
How can formal methods for improving decisions be made understandable and cognitively tractable for participants in complex environmental decisions? How can such methods be applied in real-world decision settings? How are the decisions affected?
To what extent and under what conditions do the benefits of formal approaches to decision making outweigh their costs in time, money, and effort?
How can judgments about the nature and likelihood of a range of outcomes be made more routine and workable through the use of information technologies? Do approaches such as influence diagrams and elicitation of subjective probability lead to clearer and more accurate understanding of uncertainty?
How can learning be built into these formal tools through the potential for updating over time?
How can methods for structuring decisions be applied effectively when decision processes overlap and involve multiple agencies, levels of organization, and sectors that jointly affect environmental outcomes?
What systematic methods for aggregation of preferences can be developed and implemented in realistic environmental decision settings that do not require the strict assumptions of social benefit-cost analysis?
How can risk communication methods be used to make the results of efforts to clarify preferences and uncertainty intelligible and useful to key decision makers and affected parties?
Which values matter to individuals in important generic decision situations (e.g., purchase of energy services, housing, transportation, and consumer durables)? Can decision-aiding approaches help consumers by structuring the values and uncertainties in these choices as well as their links to other broader level decisions?
Creating Effective Analytic-Deliberative Processes
As already noted, several authoritative studies recommend that public policy decisions affecting environmental and associated public health risks be organized in ways that integrate analysis with broadly based deliberative processes involving the range of parties interested in or affected by the decisions. These studies conclude that better decisions can result when analysis is organized for decision relevance by giving decision participants a guiding role: “deliberation frames analysis, analysis informs deliberation, and the process benefits from feedback between the two” (National Research Council, 1996:6).
Many government agencies in the United States and elsewhere have made commitments to using broadly participatory processes involving analysis and deliberation to make or support environmental policy decisions and many have tried to implement those commitments (see, e.g., Beierle and Cayford, 2002; Leach, Pelkey, and Sabatier, 2002; Bradbury et al., 2003; Kasemir et al., 2003). Nevertheless, the quality of these decisions is only beginning to be evaluated and the knowledge base for selecting the best process for a specific decision remains weak. By the late 1990s it was possible to demonstrate the potential of analytic deliberation, to identify some of the factors likely to affect its success, and to show that the best process depends on the situation. But because systematic analyses based on data from multiple cases are only beginning to appear (Jones and Klein, 1999; Beierle and Cayford, 2002; Leach et al., 2002; Bradbury, Branch, and Malone, 2003) and because most of these studies are restricted to specific decision contexts, understanding has not progressed to the point at which science-based input can be given to the design of processes affecting types of decision that have not yet been studied. As a result, organizations that convene such processes have been limited to improvising on the basis of the judgments of experienced practitioners and extrapolation from available case studies.
This situation is ripe for change. In recent years, researchers have begun to apply consistent methods to the study of multiple analytic-deliberative processes (e.g., Ashford and Rest, 1999; Beierle and Cayford, 2002; Leach et al., 2002; Bradbury et al., 2003). Such studies have the potential to demonstrate generalities that apply across contexts and to specify ways in which outcomes are context-dependent. These studies, together with advances in theory and conceptualization (e.g., Renn, Webler, and Wiedemann, 1995; National Research Council, 1996; Beierle and Cayford, 2002; Renn, 2003), are making it possible to build much more nuanced understanding of desired outcomes, such as how decision quality and legitimacy are affected by the ways collective environmental decision-making processes are organized (e.g., whether and how the parties are represented, what resources they
have available, how their input is structured, how decision makers are constrained in using external input).
With this base of concepts and empirical knowledge, researchers are now poised to draw on preexisting bodies of basic social and behavioral science research that are clearly relevant to the design of environmental decision processes. These include not only decision research, as already noted, but research on small-group processes (Moscovici, 1985; Levine and Moreland, 1998; Mendelberg, 2002), perceptions of justice and fairness (e.g., Tyler and Smith, 1998; Mikula, 2003), democratic deliberation and civic participation (e.g., Fishkin, 1991; Elster, 1998; Dryzek, 2000; Ostrom, 1990); organizational change (e.g., Scott, 1992; Chess, 1999), communications research (National Research Council, 1989; McComas, 2001), and conflict resolution (e.g., Druckman, Broome, and Korper, 1988; Rubin, Pruitt, and Kim, 1994; Fisher, 1997; Bingham and Langstaff, 2003). With clearer conceptual frameworks for examining environmental decisions, findings from these separate, older lines of research can be linked to the study of environmental decisions and can generate new and fruitful hypotheses to explore.
An ongoing study on public participation in environmental assessment and decision making at the National Research Council is synthesizing knowledge in this rapidly moving field and the associated fields of basic social and behavioral science and developing recommendations for research and practice (see http://www7.nationalacademies.org/hdgc/Public_Participation.html). An organized research community is beginning to emerge that can generate the knowledge needed for a science-based practice of process design for environmental policy decisions. This research can improve the ability of decision-making organizations to deal in a competent and credible way with environmental complexity, incomplete and uncertain knowledge, diversity of human values and interests, and the other realities that make this field of decision making so difficult.
Research to build effective analytic-deliberative processes could address such questions as:
What are good indicators for key attributes of success for analytic-deliberative processes, such as decision quality, legitimacy, and improved decision capacity?
How are these outcomes affected by the ways in which the processes are organized, the range and diversity of people involved, the rules used for deliberating and reaching conclusions, the ways technical information is organized and made available, and the environmental, social, organizational, and legal contexts of the decision at hand?
What are effective ways to make technical analyses transparent
to a wide range of decision participants, some of whom lack technical training?
How can decision-analytic techniques for preference elicitation, characterizing uncertainty, and aggregating preferences be used to best advantage in broadly based analytic-deliberative processes?
How can decision processes be organized to ensure that all sources of relevant information, including the local knowledge of nonscientists, are gathered and appropriately considered?
How can analytic-deliberative decision processes be organized to reach closure effectively and with broad acceptance, especially when the processes involve a diversity of perspectives and interests? What tests could be applied to decisions and decision processes to support claims that they are ready for closure?
RATIONALE FOR THE SCIENCE PRIORITY
Likelihood of scientific advances. The recommended research can yield significant scientific advances by building on several recent developments in understanding. Recent efforts to identify and assess several elements of decision quality (e.g., Webler et al., 2002; Beierle and Cayford, 2002) have established the groundwork for much improved understanding and measurement of this concept. Substantial recent work on decision-analytic tools for structuring decision processes, conducted mainly in laboratory and simulation settings, provides a basis for developing these tools further and testing and comparing their usefulness in realistic settings involving multiple and diverse participants. Recent theoretical, conceptual, and empirical work on analytic-deliberative processes and the increasing development of a self-identifying community of researchers and practitioners has set the stage for rapid progress through conceptually coherent empirical research on the design and study of processes for informing environmental decisions through analytic deliberation.
Continuing interest at the National Science Foundation in research on environment and decision making bodes well for scientific advances. Through its programs on decision, risk, and management science and human dimensions of global change and its initiatives on coupled human and natural systems and environmental research and education, the foundation is bringing together researchers from multiple disciplines with shared interests in environmental decision processes. These venues for communication are likely to provide good test beds for new research ideas.
Potential value. The long history of inadequately informed and incompletely deliberated environmental decisions, as well as the cost in delay, decisional gridlock, social conflict, and mistrust of government, make clear
the importance and value of finding more competent and legitimate ways to organize the processes that lead to public policy decisions affecting environmental quality. Moreover, the increasingly widespread practice among federal agencies and other governmental and nongovernmental entities of opening environmental decision-making processes to a range of stakeholders and potentially affected parties has raised the stakes for managing decision-making processes well. A decision science approach can increase the likelihood of success with such processes.
Likelihood of use. The increasingly widespread use in government of participatory processes requiring both analysis and broadly based deliberation indicates the potential demand for scientifically informed guidance on how to make decision processes work better. Despite the public commitments of various government agencies to openness, however, significant barriers remain to the use of results from the recommended scientific research on decision-making processes. These include commitment to standard procedures or past practices, perceptions of statutory constraints, and a shortage of organizational capability to conform to principles of sound process design (National Research Council, 1996). Key decision makers may not recognize that it is possible to put the design of decision processes on a scientific footing. Perhaps the most serious barrier to use of the results of the recommended research lies in the potential unwillingness of some decision makers to delegate responsibility for the design of decision processes or to involve a full range of affected parties in decision making in a serious way for fear that the ultimate decision might not fit their preconceived ideas or serve interests they wish to promote.
Despite such potential barriers, many environmental agencies clearly have backed their stated commitments to better and more open decision processes with significant investments of time, money, and institutional reputation, for example, in seeking out and responding to the input of a variety of stakeholders in these processes. Some have also shown serious interest in designing these processes with the help of sound knowledge. Such agencies are likely to take research results seriously if researchers are given incentives to disseminate their findings and if good lines of communication are established between researchers and practitioners. To make best use of research results, decision-making organizations should create internal incentives and assign responsibility within the organization for incorporating the best science into the deliberative part of decision making, and not only the analytical part. To the extent that these efforts are successful in some public-sector organizations, they are likely to diffuse to others over time.