Improving Environmental Decision Processes
Robin Gregory and Timothy McDaniels
Environmental decision processes refer to the ways by which individuals, groups, and organizations—and ultimately societies—go about making choices that have implications for the natural environment. Individual decisions about consumption patterns, energy use, the relative importance of different water and air quality objectives, and trade-offs such as those among recreation options all have important environmental consequences. Choices by groups concerning alternative uses of the local resource base, including how such decisions will be made and who will participate, affect not only environmental but also economic, social, and cultural options for communities and regions. Choices by organizations and by corporations about which products to use, produce, and market; how to manage waste products; and how to incorporate learning and make investments over time all may have significant environmental implications. This includes a vast array of choices by municipal, state, national, and international agencies and governments that also have environmental significance, including regulations, land-use rules, standards for transportation and energy policies, guidelines for the extraction of renewable and nonrenewable resources, international agreements on climate change, and countless others. Considering how to go about making these decisions raises issues relating to the construction of social norms and values, cognition and emotion, analysis and discourse, information and informed consent, and the ability to appropriately frame and address difficult trade-offs. Together, these issues serve to shape how environmental decisions are now made and also set a frame-
work for the consideration of improvements in environmental decision making as the result of future research in the social and behavioral sciences.
Sound environmental decision making reflects the theory and practice of general principles for good decision making. These include commonsense steps, such as defining the problem correctly or identifying specific objectives that will be used to assess the pros and cons of alternatives, as well as being attentive to the limits of scientific knowledge, recognizing influences of the regulatory and political context, and the like. To a large degree, this same mix of common sense and awareness of constraints would help to define good decision making in other policy areas such as health protection or space exploration or economic development.
In addition, several characteristic aspects of environmental decision making muddy the theory and complicate the application of decision making to environmental problems. One of these is the importance of scientific knowledge to making good decisions. A second is that the needed scientific knowledge encompasses both the natural sciences and the social sciences. It is well known that the quality of communication between these two groups of professionals is often poor. A third characteristic of many environmental decisions is the level of uncertainty and disagreement associated with the science. Finally, environmental decision making often incorporates scientific and expert understanding and analysis within an explicitly deliberative and political context (National Research Council, 1996) in which technical experts work alongside community residents, representatives of local or state governments, and consultants or members of nongovernmental organizations and other interest groups. As a result, there is a need to combine the knowledge of scientists with that of local residents and resource users in ways that are readily understood by diverse groups of stakeholders and to use processes that help these same groups to make good decisions.
In this appendix, we link two primary fields of study, one based in the decision sciences and the other in environmental policy, and draw from a variety of other disciplines, including psychology, economics, political science, and geography. Rather than offering a comprehensive review of these contributions, we identify some key insights and promising areas of research that can be useful for improving decisions made by individuals, groups, and organizations that may affect the natural environment. We emphasize topics that have the potential for improving environmental decisions within the next decade. In each of these topic areas, substantial progress already has been made, but much more can be done.
LEVELS OF ENVIRONMENTAL POLICY DECISIONS
Environmental decisions include management strategies or levels of funding for activities that either directly affect components of the natural
environment, such as forests, oceans, wildlife, inland waterways, or fisheries, or influence health and lifestyle issues by, for instance, managing toxic wastes or air or water pollution. Yet the scope of environmental decisions is far broader because of the relationships among environmental management choices and economic, social, health, and cultural values. Environmental choices influence local and national economies, the geographic distribution of employment opportunities, people’s health and longevity, the social structure of communities, and, to an underappreciated degree, the economic and social fabric of the country. Moreover, economic, social, and health choices hold important implications for management of the environment. Many of the environmental challenges facing society, ranging from climate change to species diversity to genetically modified crops to soil contamination, stem directly or indirectly from decisions made primarily on the basis of economic, social, and political concerns rather than environmental considerations.
Environmental decisions extend across time scales, physical scales, and institutional scales. Although it is commonplace to observe that issues of environmental sustainability have dimensions that extend from the local to the global, relatively little of the writing on environmental decision making actually addresses how such linkages should be addressed in deliberation, assessment, analysis, and management (Cash and Moser, 2000). Good environmental decision making requires processes that link and balance values and technical information about impacts across these multiple scales, over long time horizons. This is not an easy task.
Beyond the complexity introduced by multiple scales, environmental decisions remain among the most difficult, multifaceted, multidisciplinary questions facing society. They are fraught with abundant uncertainties, value conflicts, long time horizons, high stakes, severe organizational and institutional constraints, and many deep levels of emotion.
We have organized the discussion around who it is that faces the decision and makes a choice: individuals, groups, and organizations. As noted in the following discussion, we acknowledge that some issues (e.g., adaptive management policies as a concern for organizations) fit quite neatly into one category whereas other issues (e.g., difficulties in making trade-offs, the role of time) cut across all three levels.
Research and Practice on Environmental Policy Choices by Individuals
Much of the research conducted by decision scientists over the past 50 years has addressed questions of individual decision making from two primary perspectives. One is normative, as reflected in the domain of subjective expected utility theory, which is built on a set of rational axioms that show the conditions for optimal choices in light of uncertainties and mul-
tiple objectives (Keeney and Raiffa, 1993). Subjective expected utility theory is widely recognized as applying to idealized decision making because it says very little about “how to do it”: that is, the processes by which these concepts could best be implemented in practice. The second perspective is descriptive, examining how individuals actually make choices in the face of complexity, limited time and information, and the need to balance concerns for accuracy and effort (Kahneman, Slovic, and Tversky, 1982; Payne, Bettman, and Johnson, 1992). Descriptive research has provided valuable insights into unaided human processes of perception, judgment, and choice, but says little about how one should design or conduct decision processes so as to make better choices. A third perspective, of prescriptive decision aiding (Bell, Raiffa, and Tversky, 1988), has received far less attention despite the obvious importance of helping people to make better decisions and choices and the robust research finding that, when left to their own devices, people “systematically violate the principles of rational decision making” (Slovic, Fischhoff, and Lichtenstein, 1976:169).
Individuals facing environmental choices need to first determine what is being asked of them and then figure out their preferred response, communicating it in an appropriate manner (to ensure that their “vote” will count and be entered correctly). This involves identifying their values in terms of those concerns (environmental, economic, social, health) that might be affected by the reasonable set of actions or policy choices. Researchers in the decision sciences have emphasized that this process generally occurs in the context of significant uncertainty, concerning both what the individual might want (Slovic, 1995) and what the consequences of different decisions might be (Morgan and Henrion, 1990).
Behavioral decision research, in particular reflecting the descriptive findings by Kahneman and Tversky (e.g., Tversky and Kahneman, 1981), has emphasized that a variety of heuristics and biases, such as anchoring on first impressions (without sufficient later adjustment) or paying undue attention to more salient aspects of a choice, may systematically influence the decisions made by individuals, with the result that choices may differ from those that these same individuals might, upon reflection, prefer (Kahneman et al., 1982). Findings by decision scientists (principally cognitive psychologists and behavioral economists) also introduce the idea that preferences for many environmental, social, and economic choices are constructed rather than simply revealed by a measurement practice (Slovic, 1995). If people are familiar with a choice and know what they want (e.g., because they have learned through repeated trials), then it might not matter much how they are asked. But when faced with unfamiliar decisions or trade-offs across novel options (neither of which is uncommon in the realm of environmental policy choices), individuals may not know their own preferences and therefore are required to construct (as well as articulate) their values in
the context of a specific choice. This will be accomplished by building from an underlying set of more basic values, guided in part by the cues provided by the elicitor or the judgmental setting. The reality of constructed preferences means that the analyst must ensure that such cues come from relevant sources (e.g., experiences with similar types of goods) rather than irrelevant ones (e.g., unintentional nonverbal hints).
The construction of preferences and, in turn, observed choices and judgments will be influenced by affective and emotional considerations as well as cognitive processes. Research by neurologists such as Damasio (1994) and decision scientists such as Loewenstein (1996) demonstrates that a key predictor of a person’s valuation of an item will be their general assessment of the positive or negative affect associated with the good. Affective considerations that come into play as part of a decision-making process, reflecting the individual and the task as well as the interaction between them, influence relative judgments of salience that, in turn, influence multidimensional evaluations of options (Slovic, Finucane, Peters, and MacGregor, 2002). Mellers, Richards, and Birnbaum (1992), for example, showed that the weights in integrative valuation tasks are inversely proportional to their variance, a result of the greater affective impression on judgments made by attributes whose variance is smaller.
Research and Practice on Environmental Policy Choices by Groups
Deliberative processes that seek to obtain responsible input to environmental decisions by involving small (10 to 25-person) groups in discussions about the choice of a preferred environmental policy have become very popular over the past 15 years. Guidance has been provided by a variety of official and quasi-official publications and bodies, including the National Research Council (1996) and the Presidential-Congressional Commission on Risk (1997). Similar initiatives have been undertaken in Canada (Canadian Standards Association, 1997), the UK (Royal Commission on Environmental Protection, 1998, and other countries (e.g., Australia, The Netherlands, Germany).
Successful deliberation requires a combination of at least three elements (Gregory, Fischoff, and McDaniels, 2004). The first is agreement among participants on the ground rules for participation; this involves both bringing the set of legitimate stakeholders to the table and keeping them there with appropriate rules for dialogue, analysis, and addressing disputes. The second element is a process for aiding decision making by group members that provides a context for creating effective understanding. Third, successful deliberation requires techniques for integrating the views of participants, for translating opinions into values, and for communicating effectively with decision makers. Addressing these three elements effectively
requires an understanding of both analytical components, in terms of distinguishing technical (factual) and value-based issues and explicitly addressing sources of uncertainty (relating either to facts or values) as part of selected evaluation approaches, and behavioral components, in terms of helping participants to understand the issues and express their conclusions clearly.
A wide variety of formats, principles, methods, and techniques have been proposed, employed, and analyzed to varying degrees as a basis for conducting deliberative processes. Renn, Webler, and Wiedemann (1995), for example, discuss approaches ranging from citizen juries to stakeholder negotiations. Beierle and Cayford (2002) and Chess and Purcell (1999) discuss the evaluation of such processes. The range of potential models is vast and, as discussed in more detail below, few empirical comparisons of different approaches have been conducted.
Our experience is that the methods of decision analysis, which typically have been applied to individual choices (Hammond, Keeney, and Raiffa, 1999), provide a particularly useful model for guiding group deliberations that address public policy questions. The distinguishing feature of decision analytic approaches is that they are sensitive to human judgment. People who might be influenced by a decision are asked to develop a clear statement of their relevant values, in terms of what matters in the context of the decision alternatives and operative constraints. Technical experts are asked to provide information about the consequences of these options as well as the associated uncertainty (e.g., probability distributions, degrees of belief); individuals are then asked to weigh the various objectives in terms of their relative importance, in the context of the specific problem under consideration. Both computational and judgmental methods are used to combine these components, using the precepts of multiattribute utility theory (Keeney and Raiffa, 1993) to summarize across concerns and provide recommendations to policy makers.
Research and Practice on Environmental Policy Choices by Organizations
Although behavioral decision theory has focused on the individual, helpful insights have been gained into the behavior of organizations and the ways in which they make decisions and address multidimensional choices (March, 1997). Much of this work is descriptive, with substantial progress made in recent years on understanding how organizations change in response to new ideas and stimuli. Social learning theory, which identifies ways in which organizations transform knowledge into action, has been one of the main themes in research on organizational theory over the past two decades (Levitt and March, 1988; Argote, 1999).
A widely discussed application of social learning in the context of environmental decision making is adaptive management (Holling, 1978; Lee, 1993). The premise is simple: because there are profound uncertainties in resource management decisions, policy actions should be regarded as experiments and a positive value should be attached to characteristics of management plans such as flexibility, learning, and monitoring or feedback. The idea of adaptive management has common-sense appeal, and the underlying concept of science-based policies as testing hypotheses and embracing failures is basic to the scientific method. Yet significant institutional, political, and organizational barriers have blocked many of the more innovative plans to pursue adaptive management strategies (Gunderson, Holling, and Light, 1995).
Another important concern for institutions, in the both private and public sectors, involves consideration of the costs and benefits that stem from actions (projects, programs, or policies) that occur at different points in time. To facilitate these intertemporal comparisons, organizations typically calculate the present value of future gains and future losses; on the basis of prescriptions that follow from the expected utility model, the usual practice is to employ a single, invariant rate of discount. This practice, however, is not supported by behavioral studies; as discussed further in the section “Approaches to Aid Organizational Decisions on Environmental Policy Changes” below, the topic is one of many areas receiving significant attention from researchers.
APPROACHES TO AID INDIVIDUAL DECISIONS ON ENVIRONMENTAL POLICY CHOICES
We noted above that the field of decision sciences has benefited from several syntheses that provide solid assessments of the field. Influential books were written by Kleindorfer, Kunreuther, and Schoemaker (1993), who provide a broad review of the field of decision sciences at the individual, group, and societal levels, and by Payne et al. (1992), who address individual decision making from the viewpoint of behavioral decision research. Mellers, Schwartz, and Cooke (1998) focused their review on behavioral violations of rational choice theory; Pidgeon and Gregory (2004) summarized contributions of the decision sciences to public policy applications, including a review of the important role played by heuristics as a means for cognitively simplifying otherwise dauntingly complex choices faced by individuals. In this section we provide an overview of some of the key concepts from the decision sciences related to individual preferences and preference elicitation. After that we turn to a number of topics that could help comprise future research priorities related to these issues.
We noted above that people often do not hold well-defined values for
complex and unfamiliar environmental goods. Hence, understanding preferences should stress the important role of helping people construct their preferences rather than simply revealing them through actions or questions (Slovic, 1995). Tools concerned with preference construction and elicitation include formal methods to elicit preferences, based on precepts of measurement theory and decision theory, along with wisdom gleaned from applied experience. Perhaps the most well known of these formal approaches, multiattribute trade-off (or MAUT) analysis typically involves an interview between an analyst and a decision maker to construct a utility or value function for the decision maker. Keeney (1992) provides many examples of these kinds of questions, and the functional forms and the assumptions involved in such efforts, as well as references to many examples in which these methods have been employed. The advantages of a MAUT approach are that the judgments involved are made explicit, the value information can be used in many ways to help clarify the decision process, and the decision maker typically learns a great deal through these joint efforts to construct their views on preferences. The disadvantages can also be substantial: the questions involved may be difficult to answer and require decision makers to make their inchoate feelings explicit, how the results will be used may not be transparent, and the process can be cognitively and analytically demanding.
In part as a response to these drawbacks, and in part reflecting the richness of decision research, several other approaches to preference elicitation have been developed and tested by researchers. For example, the analytic hierarchy process, developed by Saaty (1991) is a widely applied approach to eliciting preferences in decisions with multiple objectives. Proponents maintain that it involves more transparent methods and questions than those required for MAUT analysis; critics question whether results of an analytic hierarchy process might violate normative principles of decision making. Other, more intuitive approaches to judging preferences also are found in the behavioral decision-making literature. These include making decisions using only one of the objectives, even though several are important (lexicographic ordering), or selecting the first alternative that provides acceptable performance across all the objectives “satisficing” (Payne et al., 1992). Of course, these behaviorally straightforward methods also may encourage some of the heuristics, biases, and shortcuts that can undermine the quality of decision making in ways discussed above.
A trend to simpler and less demanding methods, which make use of differences among the alternatives to help clarify preferences, is evident in recent studies on decision making. Hammond et al. (1999), for example, discuss an approach to constructing and clarifying preferences called “even swaps.” This approach does not involve developing a utility or value function, but instead develops an objectives-by-alternatives, or “consequences,”
table based on the judgments of the decision maker to clarify the relative importance of differences in how well the different alternatives achieve the objectives for a decision. The approach then uses that insight as a basis for eliminating dominated alternatives and for expressing different objectives in common units, which in turn greatly facilitates judgments of which alternative is preferred.
Another useful analytical approach that has been developed in applied decision analysis practice is termed “value-focused thinking” (Keeney, 1992) The deceptively simple but fundamental notion is that attention to values can serve as the basis for several key steps in designing decision processes. For example, clarity about what matters and how it is to be measured is important in several ways: defining the information needed to characterize the consequences of alternatives; designing better, more widely supported alternatives; and indicating creative opportunities to improve the range of choices available (Keeney, 1992).
Identifying Subjective Judgments and Values
Scientific investigations often are viewed as “objective” in contrast to the “subjective” perspective of those concerned with social impacts, such as fear or worry, or process considerations, such as trust or fairness. Yet even the most highly complex scientific choices rest on subjective decisions that reflect what data to include, what people to ask questions of, and what methods should be used. Thus, both perspectives share a similar qualitative foundation. Choices about what people are included, what views are seen as data, and what criteria to use in analysis always are the results of judgments. Extensive research has been conducted on this topic (e.g., Jasanoff, 2002), but the audience of academics and resource managers remains largely uninformed, in part due to poor communication but also because much of the research fails to address relevant decision contexts.
Hence, research that more squarely addresses the role of subjective judgments in environmental decision making, both for issue-based topics and general policy analyses, could potentially make substantial contributions. Relatively little attention seems to have been paid to topics of problem choice or to the relationships among broad strategic objectives (such as long-term environmental, social, and economic sustainability) and near-term, more prosaic decisions like the choice of transportation modes or infrastructure options. Issues regarding the right level at which to conduct research and policy analysis should matter in shaping the kinds of analysis that are done and, ultimately, the policy decisions that are made. Other kinds of key judgments, beyond those listed above, include issues such as how we conduct analyses for linked decisions, in which current choices provide opportunities for learning over time (Keeney and McDaniels, 2002),
or situations in which regulatory issues are linked across multiple scales and levels of decision making (McDaniels and Gregory, 2004). Other kinds of subjective judgments that merit research could include generalizations of the conditions on which cooperation is likely to occur among parties in commons contexts, particularly in situations where multiple levels of cooperation may be needed.
Clarifying Performance Measures
Despite the obvious need to specify project or action consequences, insufficient attention has been given to the design of performance measures (Keeney, 1992). In part, this is because of the emphasis of many policy analysts on economic methods and the model of cost-benefit analysis, which uses dollars as a common metric for evaluating impacts. Yet measures such as dollars represent only one of three primary types of indicators or attributes: natural, proxy, and constructed.
Natural attributes are in general use and have a common interpretation. The management objective to “maximize profits” is naturally measured in dollars; similarly, if one management objective is to minimize the loss of wildlife habitat, then a natural indicator might be “acres of lost habitat.” Cost (also measured in dollars) and worker injuries (measured in numbers) are other examples of natural indicators.
Proxy attributes also are in general use and are well understood. However, they are less informative than natural attributes because they only indirectly indicate the achievement of an objective. An example is the use of a measure such as “returned items per $10,000 sales” as a proxy for product quality. Another common example in environmental contexts is the use of an easily measured indicator (such as air emissions in parts per million) as a proxy for impacts of concern that are harder to measure (such as adverse health or visibility impacts due to air quality impairment).
Constructed attributes are used when no suitable natural attributes exist. An example is a scale to measure community support for forest management practices. Because no natural scale exists to measure public support, an index (e.g., 1-5 or 1-10) needs to be created, with each rating denoting a different level of support. Many such constructed scales are in widespread use: the gross national product is a constructed measure, as is the Dow Jones stock average in the United States or the Apgar score given to track the health of a newborn. When thoughtfully designed, constructed indices can greatly facilitate management by defining precisely the focus of attention and by permitting trade-offs across different levels of the concern and other attributes (e.g., is it worth postponing harvest of an area for x years in order to increase support from level 2 to level 4?).
Attributes are made operational through the development of scales.
These scales serve two major purposes. First, they provide a means for distinguishing among different levels of impact with respect to the attribute. Second, the scales provide a clear means for distinguishing the endpoints of the range of anticipated impacts. As an example, consider a scale denoting the expected cost of a range of management options. If the lowest reasonable cost for the options under consideration is $20,000 and the highest reasonable cost is $70,000, then the scale should reflect this range and be measured in thousands of dollars (20-70). Using a measure such as “hundreds of dollars” is not appropriate because it conveys an unnecessary (and probably illusory) sense of precision. Similarly, converting a natural scale of this type to an index (e.g., whereby a “1” = $20,000-30,000, “2” = $30,000-40,000, etc.) is not a good idea because information is lost (i.e., is a “2 “ at the high or low end of the range?). On the other hand, a scale for measuring community support for environmental plans would be a constructed scale and it might be measured in terms of several related attributes (e.g., turnout at meetings, support conveyed through a survey, etc.) so that a “1” might denote a low measure of support (low turnout, low percentage of support in a survey) and a “5” denote a high measure of support. In general, scoring methods used to select scales should be accurate, understandable, and at the appropriate level of discrimination.
In our opinion, far more attention needs to be given to the design of clear performance measures and to their incorporation as part of environmental evaluations. This increased use of performance measures would yield three primary benefits, all relating to the provision of information that will aid stakeholders and decision makers. The first is to focus attention on those aspects of the problem that are considered to be important and to set up measurable criteria by which progress on these considerations can be assessed. For example, one objective of sustainable forest management may be to maintain overall forest health. To be useful for management purposes, forest health will need to be disaggregated into components that can be identified and measured. Resilience might be one such component, and productivity might be another; if so, then measures for resilience and for productivity will need to be developed along with some criteria for weighing their relative contributions (alongside other components) to overall forest health. A second benefit would be to discriminate more clearly among competing hypotheses, therefore contributing to the conduct of scientific investigation. A third benefit would be to stimulate the identification and creation of a range of management alternatives and to serve as decision aids for the identification of a preferred management plan (or set of plans). The best management actions are those that best achieve the objectives of the environmental management problem; without clear measures, there cannot be clear communication about the ability of actions to satisfy the identified objectives.
There are three main reasons why trade-offs are addressed poorly as part of many environmental management plans (Gregory, 2002). First, addressing trade-offs requires techniques that help the individual to explicitly address multiple dimensions of value. This task is cognitively difficult and in many cases requires the adoption of decision-aiding methods for weighing the importance of different components of the problem that are unfamiliar to many analysts and decision makers, let alone many members of the lay public. Second, trade-offs can be emotionally difficult: they raise moral and ethical dilemmas and can require individuals to address explicit choices about topics that they (and other people, including their elected leaders) may find uncomfortable (Baron and Spranca, 1997; Fiske and Tetlock, 1997). As a result, in many cases these decisions are either treated informally (to decrease perceived responsibility costs) or left to others. Third, it is often assumed that addressing trade-offs in a rigorous and defensible manner will prove cumbersome and expensive. This assumption is not necessarily true: relatively simple and straightforward techniques exist for helping people to address trade-offs in ways that can substantially assist many individual decisions and provide essential insights for negotiations between individuals whose values may differ (Hammond et al., 1999). Furthermore, even extremely difficult or complex problems often can benefit substantially from the insight provided simply by clarifying the nature of the trade-offs and how they influence choices across management options.
Using Expert Judgment Processes to Understand Uncertainty
Input from technical experts is required to help anticipate how actions might affect the natural and the human environment and to develop approaches for mitigating potentially adverse impacts. Both the identification of impact categories and the assignment of probabilities are judgmental actions, often requiring skills of inference and prediction for which most scientists (as well as most laypersons) have received little or no formal training. Current research on probabilistic analysis is highlighting ways in which the basis for technical judgments can be clarified and likelihood estimates of impact magnitude and severity can be refined and communicated (Cullen and Small, 2004).
New tools and approaches for clarifying uncertainty are intended to improve the quality of environmental management decisions over time and to increase the understanding of how a variety of techniques (such as adaptive management trials) might be used by managers as a possible response to uncertainty. At present, uncertainty often is handled in a more casual
manner, which can result in suboptimal decisions and fewer opportunities for learning (because feedback in terms of management responses is more difficult to incorporate). At minimum, environmental managers should consider the different sources of uncertainty—including uncertainty about impact severity, habitat responsiveness, the influence of slow variables such as climate change, the effectiveness of mitigation measures, monitoring results, and the compliance of proponents—and decide (and document) how each source is best handled.
A variety of techniques are available to incorporate judgments of uncertainty. As one example, expert judgment techniques (Keeney and von Winterfeldt, 1991) can be employed in cases in which higher or more consequential risks are anticipated and uncertainty is high. These techniques seek to first decompose a more complex problem into its parts (which allows for simpler judgments) and then to improve the uncertainty assessments of managers for those aspects of the problem where they are either least confident (i.e., about their assessments of expected impact distributions) or for which disagreements exist among managers or across stakeholders (Morgan and Henrion, 1990). Different approaches can be employed, depending on the needs of the problem domain; as one example, it may be more appropriate to elicit degrees of belief in specified endpoints (using questions along the lines of “Given mitigation plan X, assign 100 points among the three future habitat states of A, B, and C”) rather than probability distributions. Yet although formal expert judgment techniques have been used to enhance understanding of a variety of resource management problems, many natural scientists are hesitant to make use of the technique due to concerns that (a) it will highlight disagreements between individuals (whereas in fact explicit expert judgments typically lead to more agreement because the reasons for differences among views are clarified), (b) it will substitute elicitations for field trials (whereas in fact the two approaches are highly complementary), or (c) it will undermine confidence in the assessments of experts (whereas in fact the explicit documentation of uncertainty often eases worries and increases the credibility of scientific assessments) (Gregory and Failing, 2002).
APPROACHES TO AID GROUP DELIBERATIVE PROCESSES ON ENVIRONMENTAL POLICY CHOICES
The science underlying our knowledge of approaches to aid group deliberative processes for environmental choices has benefited from recent reviews. A useful starting point is again Kleindorfer et al. (1993) who provide an overview of small group decision-making processes. Renn et al. (1995) evaluate the ability of decision science methods to encourage fairness and competence in public participation. Beierle and Cayford (2002)
review the content and outcomes of a large set of applied cases, providing conclusions on the benefits and drawbacks of such processes.
Criteria for the Conduct of Deliberative Processes
Deliberative processes, involving either small groups of experts (science and lay representatives) or larger panels (advisory councils or specialty forums), are used increasingly to open up the environmental decision-making process and to ensure that a wide range of views is heard. Yet, in most cases, the weighing or balancing of conflicting objectives, which is the essence of clarifying trade-offs, is either ignored or only partially addressed. Instead, there is usually some attempt for public values to be expressed (e.g., in the form of goals or concerns), but rarely is attention given to carefully structuring the underlying choices and explicitly addressing the key trade-offs. As an example, Duram and Brown (1999:462) reported after extensive interviews with stakeholders active in U.S. watershed planning initiatives that “fewer than 50% noted that participation was useful in clarifying the issues,” although this would seem to be a minimum requirement of any policy-based deliberative process. The underlying behavioral questions—how well the deliberative process is understood, how thoughtfully the input is provided, and how meaningfully outputs are considered (in terms of process and outcome linkages)—remain areas of frustration to theorists and practitioners alike.
One obvious goal is to obtain judgments that are better matched to the decision at hand and more cognitively manageable by the participants. An example is the use of voting as a basis for preference elicitation in specific policy decisions. McDaniels and Thomas (1999) discuss an approach in which voters are asked to choose among a set of possible policy alternatives to address a given environmental question. This approach relies on problem structuring tools from decision analysis to develop a well-structured decision with explicit policy objectives and alternatives. Then risk communication methods are used to help voters understand the consequences of the alternatives. Voters then must make holistic judgments that integrate across the various objectives to select the preferred alternative(s).
Beierle and Cayford (2002) identify generic performance criteria for deliberative processes, emphasizing procedural matters relating to how the process is conducted. Writings by Renn et al. (1995) also emphasize issues regarding how such processes are conducted to build trust, avoid power imbalances, and foster agreement. Still other criteria, which deserve more attention, address how a given approach handles the complexity that is a crucial part of all important environmental decisions, not simply in terms of information sharing, but also how the problem structuring concepts, formal analysis of judgments, and analytical tools of decision sciences are
used. Other criteria would need to take account of the role that emotion and affective considerations, as well as deliberation, play in the conduct and quality of discourse pursued by a group. Key research needs, therefore, include methods that could help to ensure that participants understand a problem (e.g., recognize that its representation is both complete and comprehensible) and that they are able to make sense of their assigned evaluative task (e.g., by using valuation methods that are cognitively appropriate and provide for the expression of affective and emotional concerns).
Linking Local Knowledge and Scientific Expertise
Many environmental policy initiatives fall short of expectations: deliberations are not perceived as open, the scientific basis for decisions is questioned, and managers are unprepared for (the inevitable) ecological and political surprises. A particular problem for environmental managers is that, in many cases, lay and community participants feel disenfranchised and believe that key elements are missing from recommended management alternatives because their concerns and values have not been heard. As a result, policy recommendations fail to reflect the full set of relevant objectives and, not surprisingly, community support is withdrawn.
One explanation for this failing (and, consequently, for the dissatisfaction of many community residents) is the apparent choice, by the initiating agency or facilitators and analysts, to emphasize science literacy and thereby place primary emphasis on the opinions of scientists and other technically trained participants (Norton and Steinemann, 2001). Better science, it is concluded, will lead to better deliberative processes. The U.S. Environmental Protection Agency’s Science Advisory Board, for example, recently concluded that “stakeholder decision processes … frequently do not do an adequate job of addressing and dealing with relevant science” (Environmental Protection Agency, 2001). The rationale is that these are complicated technical issues and laypersons simply are not sufficiently well informed to make rational decisions. Examples include the hesitancy of many scientists to provide accurate information to public stakeholders about the uncertainties associated with risk management options due to concerns that disclosure might “cause panic and confusion regarding the extent and impact of a particular hazard” (Frewer et al., 2003). Greater reliance on science and on the judgments of scientists, it is believed, will make for better choices.
Although we agree that sound science is necessary for environmental decisions, it is often not sufficient: in particular, we believe that increased attention should be given to the significant body of knowledge held by local and community participants that is not grounded in conventional scientific methods but is nonetheless empirically derived. Some of these knowledge
holders are long-time community residents; some are aboriginal populations with special interests and cultural uses of environmental resources; some are resource users with specialized knowledge such as fishers or trappers. This alternative knowledge base may be based on different techniques, and reflect differently constructed forms of knowledge, than those of Western scientific methodologies, yet we believe it often represents a useful—in many cases essential—complement to science-based knowledge. For example, in the restoration of severely disrupted ecosystems (perhaps due to construction of a hydroelectric dam), traditional and local knowledge often provides the only record of ecological processes prior to disruption, and thus provides a template for the end goals of a postrestoration landscape. In other cases, local and traditional knowledge raises concerns (e.g., about protection of a plant or animal species) that are missing from scientific analyses or highlight considerations that have not been fully examined. Alternative knowledge sources can also provide an important test of convergent validity, as for example when anthropologists and archeologists check oral history against artifacts to understand more about how people first came to populate the Americas (Glavin, 2000).
Distinguishing Process and Outcomes
The quality of environmentally significant decisions cannot appropriately be judged solely by their outcomes because those outcomes are highly dependent on factors that are uncertain and often unknown when the decision is made. One can make a decision that is well informed and well considered given what is known, but be unlucky regarding the outcome of some key uncertainties, so that the results are not as hoped. Thus, it is necessary to have explicit criteria for judging the quality of decisions. Across many decisions, higher-quality decisions (those based on the best available information, careful identification of objectives and measures, and so forth) are more likely than lower-quality decisions to lead to desired outcomes (von Winterfeldt and Edwards, 1986). It is also reasonable to expect that higher-quality decisions will be more defensible, which is of concern to many decision makers facing scrutiny from the public, other constituents, or the courts.
The problem of defining decision quality for practical environmental decisions has not received the level of research attention it deserves. Both the normative and the empirical traditions in decision science have difficulty with this problem. Normative decision theory yields prescriptions that are difficult, if not impossible, to implement in practice because of the complexity of the value judgments involved and because of the requirement to assess every consequence of every possible alternative and to evaluate them all against the values of each decision participant. Decision makers do
not seek the normatively best practice, but the best practice under constraints of real people’s cognitive capabilities, legal requirements, limited time and resources, social conflict, and so forth. Researchers in the empirical tradition can be uncomfortable defining decision quality because they see a conflict between normative and positive science or because they question whether any single standard can hold up in a diverse society.
The kinds of research that could be conducted might address issues as diverse as the following: How do we judge the notion of decision quality in environmental contexts? How do we provide ongoing heuristics and routines that could make the elements of good decision process more accessible and readily applied? How can decision aids (such as, for example, a CO2 emissions calculator) be developed to help provide a structure and key information for everyday environmental choices? In addition, little is known about whether stakeholders might respond more favorably to management initiatives if different decision processes were followed; for example, could an improved process help to reduce criticism and increase acceptance in the aftermath of a low-probability but high-consequence accident such as a spill or collision? Research also has shown that reliance on, and citations of, broad stakeholder-based input can make the results of environmental decision processes more acceptable to others who are less familiar with the issues (Arvai, 2003).
APPROACHES TO AID ORGANIZATIONAL DECISIONS ON ENVIRONMENTAL POLICY CHOICES
Basic elements of research and practice regarding organizational decision making are surveyed in the writing of March (1997). Other writers, such as Argote (1999), have emphasized issues of organizational learning. There is an active field of research on how organizations respond to environmental regulations and how they structure and conduct their compliance efforts (e.g., Jennings, Zandbergen, and Martins, 2001). Policy analysis as an approach to environmental decision making in government organizations is extensively discussed in a number of textbooks, such as Weimer and Vining (1999).
The Choice of a Policy Evaluation Approach
The dominant method for the analysis of environmental options is to employ welfare economics-based cost-benefit techniques. Although a cost-benefit approach can encompass many facets of a natural resource initiative, it ultimately collapses these into a measure denoted in dollars. This has the advantage of providing a single measure with which decision makers can compare options but it also masks the contribution of individual fac-
tors, thereby making it difficult to track information about different effects (environmental, economic, social) and alienating individuals whose concerns (such as health and safety, biodiversity, or community image) might be difficult to translate into dollar measures. In addition, other approaches that make use of multiple metrics to assess the different dimensions of the problem and explicitly examine trade-offs, such as those based in decision analysis and multicriteria methods or other more qualitative approaches, tend to receive less attention.
Research questions concern the further refinement of these alternative approaches as well as the introduction of more case findings and comparative studies that can help to illustrate similarities and differences, both among techniques and in terms of how they are perceived by different groups within society. Early work by decision scientists (e.g., MacGregor and Slovic, 1986) offered interesting observations about the acceptability of different policy evaluation frameworks, but little further research has been conducted. An interesting new perspective is provided by the interest in narrative as an alternative to more analytic evaluation approaches; Satterfield, Slovic, and Gregory (2000), for example, compared the responses of participants with narrative and cost-benefit presentations of a complex environmental policy decision that required trade-offs across hydroelectric power production and fisheries health. Their results demonstrated advantages of narrative approaches for engaging participants and helping them to assess the relevance of technical information.
The Role of Learning
A crucial step in fostering good organizational response to complexity in environmental decisions is the notion of learning over time. Parson and Clark (1995) reviewed the literature on social learning (in this case, for sustainability issues), which is a solid starting point for understanding learning in organizations. This broader topic of organizational learning was surveyed by Argyris and Schon (1978) and, more recently, by Argote (1999). Policy analysts such as Lee (1993) and ecologists such as Gunderson et al. (1995) have stressed the organizational implications of adaptive management options, which involve viewing policies as experiments (discussed further in the next subsection). These concepts, which turn one-time decisions into repeated decisions with opportunities to learn and adapt, offer a wealth of new ideas for improved organizational practice in the context of complex environmental decisions.
Organizations do best in situations in which they can rely on standard procedures to address ongoing management issues. Hence, finding ways to turn complex choices into learning opportunities, and building ways in which this learning can serve as the basis for improved decision processes
over time, provide important research opportunities. McDaniels and Gregory (2004) discuss the benefits to organizations of treating learning as a specific objective in decision processes with stakeholders, including the potential for developing institutional processes that foster and measure organizational learning opportunities.
Implementation of Adaptive Management Methods
Adaptive management methods are designed to help reduce uncertainty through the conduct and comparison of selected experimental trials. Although scientific support for adaptive trials is high, few examples of successful implementation can be found. The reasons have to do with the design of the trials, the evaluation of their benefits and costs, and the institutional framework within which they have been proposed. New ideas are being proposed in each of these areas; together, they can help to facilitate learning, give meaning to guiding concepts (such as sustainability and the precautionary principle) whose vagueness has led to confused implementation, and encourage the selection of improved alternatives (Gunderson and Holling, 2002).
The concept of adaptive management was born out of the need to address the objective of learning about managed environmental systems over time. Learning is most important when uncertainties are high and when management actions are (at least in part) unfamiliar and of high consequence. Prescriptively, an adaptive management approach requires four primary elements (Walters, 1986):
bounding of the management problem in terms of objectives
characterizing existing technical knowledge about the system
designing management treatments (involving either passive or active management)
incorporating measures for reducing catastrophic risk and improving long-term outcomes
Traditional monitoring of environmental management initiatives tends to be relatively cheap; adaptive management tends to be relatively expensive. But in either case, the analysis of the pros and cons leads to subsequent questions: Why do we want to monitor? What do we hope to learn? How will we know when we have learned it enough to do something different? How do we know if this is the best way to learn? At the broadest level the question is: Why do we want to do adaptive management trials? Once these questions are answered, we must decide whether these are passive or active adaptive management efforts, over what time periods these issues are to be explored, and how those time periods correspond to the time periods of
change in other variables (for example, those with slower rates of change). Finally, we must consider how to structure the underlying expert judgment tasks. These are tough tasks, but solvable, in terms of being amenable to analytical techniques.
The toughest question of all, in many cases, is getting approval for adaptive management (or even for comprehensive monitoring). The only reason to do either is to know more later than is known now, in which case institutions need to be sufficiently flexible to acknowledge this learning and to do something different (and presumably better) in the future. Building in this sensible institutional response to adaptive management is not easy (for evidence, look to management of the Columbia River system over the past decades).
Reevaluating the Role of Time
All environmental decisions involve the element of time, yet it is rarely taken into account in directly meaningful ways. Intertemporal aspects of decisions clearly have to do with the occurrence of effects or consequences. Yet they also have to do with how impacts will be perceived (in terms of the values in place at the time) and how these perceptions will be coded (in terms of issues such as adaptation and vulnerability). Environmental activities such as the restoration of damaged riverine ecosystems may build on an extensive base of natural science research that takes place over time periods of 20 years or more; and climate change initiatives easily may span several generations, yet little attention typically is given to understanding how people’s values and perceptions (i.e., their anticipated as compared with experienced utility) may change over this same period. Further work on these topics is urgently needed to understand appropriate societal responses to some of the most serious (and more controversial) environmental policy debates such as global climate change or species vulnerability.
Even when time is considered explicitly, such as with discounting of streams of costs and benefits to determine a present economic value, the typical practice is to employ a single, constant rate of time preference. Yet, as shown by recent descriptive studies, serious questions exist about the applicability of a single discount rate to near term and more distant times (Benzion, Rapaport, and Yagil, 1989), to multiple types of effects (financial, environmental, health) (Chapman and Elstein, 1995), and to benefits (i.e., gains) as well as costs (i.e., losses) (Loewenstein and Prelec, 1992). Other factors, such as the embedding of outcomes in sequences (Loewenstein and Prelec, 1993) and accounting for the uncertainty associated with either the anticipated effects or the changes in future discount rates (Newell and Pizer, 2003), also can influence the choice of appropriate intertemporal environmental policies. Although questions relating to the
evaluation of time affect many environmental initiatives, the research conducted to date rarely has emphasized either prescriptive or normative implications; one result is that its influence on the practice or thinking of resource managers has been limited.
These three broad areas within environmental decision making all contain high-priority topics for research. In some cases, the further investigation of ideas and techniques would be quite straightforward; research already is well under way, and progress would involve application of known ideas to new areas or the formation of new bridges across disciplinary lines. In other cases there exist major barriers to the conduct of research or to the implementation of new ideas; these include institutional constraints, high levels of uncertainty, or fundamental conflicts among opposing views. The discussion in this appendix highlights these opportunities and barriers.
The potential users of the information that could be produced by a greater emphasis on research along the lines outlined here include all those individuals, small groups, and organizations that are faced with tough environmental choices. We started by emphasizing that decisions at all three levels of social decision making share the difficulty of addressing complexity in environmental decisions in responsible ways. All three of these levels interact, and opportunities and methods of improved decision making at one level can help inform and shape decision making at other levels.
It is hoped that outcomes such as lower environmental compliance costs, improved environmental protection, and increased community acceptance would accompany improved environmental decision making and provide visible evidence of substantive results. Indirect benefits could include better understanding of and preparation for low-probability high-consequence events, less time energy and money spent in the courts contesting regulatory decisions, and an enhanced belief and trust in the wisdom and common sense of decision makers.
Overall, we believe that environmental problems provide an excellent example of how society is attempting to deal with conflicts that involve multiple interests and complex technical content. When environmental decision-making processes succeed, a mechanism is provided for the orderly sharing of views and for the bridging of disciplinary gaps. When they fail, the door is opened to litigation, economic hardship, and the imposition of political solutions that often need to be revisited in a surprisingly short time. It is therefore important that the promise of sound environmental decisions be supported as fully as possible; in this appendix we have tried to outline some of the research and methods that might best achieve that goal.
The authors appreciate the guidance and encouragement of Paul Stern and two anonymous referees in completing this paper.
Argote, L. 1999 Organizational Learning: Creating, Retaining, and Transferring Knowledge. Boston, MA: Kluwer.
Argyris, C., and D. Schon 1978 Organizational Learning: A Theory of Action Perspective. Reading, MA: Addison-Wesley.
Arvai, J.L. 2003 Communicating the results of participatory decisionmaking: Effects on the perceived acceptability of risk-policy decisions. Risk Analysis 23:281-291.
Baron, J., and M. Spranca 1997 Protected values. Organizational Behavior and Human Decision Processes 70:1-16.
Beierle, T., and J. Cayford 2002 Democracy in Practice: Public Participation in Environmental Decisions. Washington, DC: Resources for the Future.
Bell, D., H. Raiffa, and A. Tversky 1988 Descriptive, normative, and prescriptive interactions in decision making. Pp. 9-30 in Decision Making: Descriptive, Normative, and Prescriptive Interactions, D. Bell, H. Raiffa, and A. Tversky, eds. New York: Cambridge University Press.
Benzion, R., A. Rapaport, and J. Yagil 1989 Discount rates inferred from decisions: An experimental study. Management Science 35:270-284.
Canadian Standards Association 1997 Risk Management Guidelines for Decision Makers. Ottawa, Ontario: Canadian Standards Association.
Cash, D.W., and S.C. Moser 2000 Linking global and local scales: Designing dynamic assessment and management processes. Global Environmental Change 10:109-120.
Chapman, G., and E. Elstein 1995 Temporal discounting of health and money. Medical Decision Making 15:373-386.
Chess, C., and K. Purcell 1999 Public participation and the environment: Do we know what works? Environmental Science & Technology 33:2685-2691.
Cullen, A., and M. Small 2004 Uncertain risk: The role and limits of quantitative assessment. Pp. 163-212 in Risk Analysis and Society, T. McDaniels and M. Small, eds. New York: Cambridge University Press.
Damasio, A. 1994 Descartes’ Error: Emotion, Reason, and the Human Brain. New York: Avon.
Duram, L., and K. Brown 1999 Assessing public participation in U.S. watershed planning initiatives. Society and Natural Resources 12:455-467.
Environmental Protection Agency 2001 Improved Science-Based Environmental Stakeholder Processes. (EPA-SAB-EC-COM-01-006.) Washington, DC: Environmental Protection Agency.
Fiske, A., and P. Tetlock 1997 Taboo trade-offs: Reactions to transactions that transgress spheres of justice. Political Psychology 18:255-297.
Frewer, L., S. Hunt, M. Brennan, S. Kuznesof, M. Ness, and C. Ritson 2003 The views of scientific experts on how the public conceptualize uncertainty. Journal of Risk Research 6:75-85.
Glavin, T. 2000 The Last Great Sea. Vancouver, British Columbia: Greystone Books.
Gregory, R. 2002 Incorporating value tradeoffs into community-based environmental risk decisions. Environmental Values 11:461-488.
Gregory, R., and L. Failing 2002 Using decision analysis to encourage sound deliberation: Water use planning in British Columbia, Canada. Journal of Policy Analysis and Management 21: 492-499.
Gregory, R., B. Fischhoff, and T. McDaniels 2004 Acceptable Input: When Are Deliberative Processes Good Enough? Unpublished manuscript.
Gunderson, L., and C.S. Holling, eds. 2002 Panarchy. Washington, DC: Island Press.
Gunderson, L., C.S. Holling, and S. Light, eds. 1995 Barriers and Bridges to the Renewal of Ecosystems and Institutions. New York: Columbia University Press.
Hammond, J., R.L. Keeney, and H. Raiffa 1999 Smart Choices: A Practical Guide to Making Better Decisions. Cambridge, MA: Harvard Business School Press.
Holling, C.S., ed. 1978 Adaptive Environmental Assessment and Management. New York: John Wiley & Sons.
Jasanoff, S.S. 2002 Citizens at risk: Cultures of modernity in the U.S. and E.U. Science as Culture 11:363-380.
Jennings, P.D., P.A. Zandbergen, and M. Martins 2001 Complications in compliance: Variations in enforcement in British Columbia’s Lower Fraser Basin, 1985-1996. In Organizations, Policy, and the Natural Environment: Institutional and Strategic Perspectives, A. Hoffman and M. Vantresca, eds. Palo Alto, CA: Stanford University Press.
Kahneman, D., P. Slovic, and A. Tversky, eds. 1982 Judgment Under Uncertainty: Heuristics and Biases. New York: Cambridge University Press.
Keeney, R., and T. McDaniels 2002 A framework to guide thinking and analysis about climate change policies. Risk Analysis 21(6):989-1000.
Keeney, R., and H. Raiffa 1993 Decisions with Multiple Objectives. New York: Cambridge University Press.
Keeney, R., and D. von Winterfeldt 1991 Eliciting probabilities from experts in complex technical problems. IEEE Transactions on Engineering Management 33:83-86.
Keeney, R.L. 1992 Value-Focused Thinking: A Path to Creative Decision Making. Cambridge, MA: Harvard University Press.
Kleindorfer, P., H. Kunreuther, and P. Schoemaker 1993 Decision Sciences. New York: Cambridge University Press.
Lee, K.N. 1993 Compass and Gyroscope: Integrating Science and Politics for the Environment. Washington, DC: Island Press.
Levitt. B., and J. March 1988 Organizational learning. Annual Review of Sociology 14:319-340.
Loewenstein, G. 1996 Out of control: Visceral influences on behavior. Organizational Behavior and Human Decision Processes 65:272-292.
Loewenstein, G., and D. Prelec 1992 Anomalies in intertemporal choice: Evidence and an interpretation. Quarterly Journal of Economics 107:573-597.
1993 Preferences for sequences of outcomes. Psychology Review 100:91-108.
MacGregor, D., and P. Slovic 1986 Perceived acceptability of risk analysis as a decision-making approach. Risk Analysis 6:245-256.
March, J. 1997 A Primer on Decision Making: How Decisions Happen. New York: Basic Books.
McDaniels, T., and R. Gregory 2004 Learning as an objective within a structured risk management decision process. Environmental Science & Technology 38(7):1921-1926.
McDaniels, T., and K. Thomas 1999 Eliciting public preferences for local land use alternatives: A structured value referendum with approval voting. Journal of Policy Analysis and Management 18(2):264-280.
Mellers, B., V.M. Richards, and M. Birnbaum 1992 Distributional theories of impression formation. Organizational Behavior and Human Decision Processes 51:313-343.
Mellers, B., A. Schwartz, and A. Cooke 1998 Judgment and decision making. Annual Review of Psychology 49:447-477.
Morgan, G., and M. Henrion 1990 Uncertainty. New York: Cambridge University Press.
National Research Council 1996 Understanding Risk: Informing Decisions in a Democratic Society. P.C. Stern and H.V. Fineberg, eds. Committee on Risk Characterization. Commission on Behavioral and Social Sciences and Education. Washington, DC: National Academy Press.
Newell, R., and W. Pizer 2003 Discounting the distant future: How much do uncertain rates increase valuations? Journal of Environmental Economics and Management 46:52-71.
Norton, B., and A. Steinemann 2001 Environmental values and adaptive management. Environmental Values 10:473-506.
Parson, E., and W. Clark 1995 Sustainable development as social learning: Theoretical perspectives and practical challenges for the design of a research program. In Barriers and Bridges to the Renewal of Ecosystems and Institutions, L. Gunderson, C.S. Holling, and S. Light, eds. New York: Columbia University Press.
Payne, J., J. Bettman, and E. Johnson 1992 Behavioral decision research: A constructive processing perspective. Annual Review of Psychology 43:87-131.
1993 The Adaptive Decision Maker. New York: Cambridge University Press.
Pidgeon, N., and R. Gregory 2004 Judgment, decision making, and public policy. Pp. 604-623 in Handbook of Judgment and Decision Making, D. Koehler and N. Harvey, eds. Oxford, England: Blackwell.
Presidential-Congressional Commission on Risk 1997 Risk Management. Washington, DC: The White House.
Renn, O., T. Webler, and P. Wiedemann 1995 Fairness and Competence in Citizen Participation. Dordrecht, The Netherlands: Kluwer.
Royal Commission on Environmental Protection 1998 Setting Environmental Standards. London, England: Royal Commission on Environmental Protection.
Saaty, T.L. 1991 Multicriteria Decision Making: The Analytical Hierarchy Process, extended ed. Berlin: RWS Publishers.
Satterfield, T., P. Slovic, and R. Gregory 2000 Narrative valuation in a policy judgment context. Ecological Economics 34: 315-331.
Slovic, P. 1995 The construction of preference. American Psychologist 50:364-371.
Slovic, P., B. Fischhoff, and S. Lichtenstein 1976 Cognitive processes and societal risk taking. In Cognition and Social Behavior, J.S. Carroll and J.W. Payne, eds. Potomac, MD: Erlbaum.
Slovic, P., M. Finucane, E. Peters, and D. MacGregor 2002 The affect heuristic. In Heuristics and Biases: The Psychology of Intuitive Judgment, T. Gilovich, D. Griffin, and D. Kahneman, eds. New York: Cambridge University Press.
Tversky, A., and D. Kahneman 1981 The framing of decisions and the psychology of choice. Science 211:453-458.
von Winterfeldt, D., and W. Edwards 1986 Decision Analysis and Behavioral Research. Cambridge, England: Cambridge University Press.
Walters, C.J. 1986 Adaptive Management of Renewable Resources. Caldwell, NJ: Blackburn Press.
Weimer, D., and A. Vining 1999 Policy Analysis: Concepts and Practice, Third Edition. Englewood Cliffs, NJ: Prentice Hall.