Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
3 The Effects of Public Participation A s Chapter 2 shows, there are many claims about the positive and ad- verse effects of public participation in environmental assessment and decision making. Our goal in this chapter is to assess the evidence regarding the degree to which public participation achieves what its propo- nents hope it will achieve and the degree to which it yields the problematic results its critics expect. We address two specific questions: â¢ Do processes that are more participatory yield better results in terms of criteria of quality, legitimacy, and capacity than processes that are less participatory? â¢ Are there trade-offs among results, such that success in terms of one of these criteria compromises success on another? There is the possibility of a pro-participation bias in the literature because researchers and practitioners predisposed in favor of public participation may be more likely to do research on participation and because reviewers with similar biases may be more critical of studies with negative findings. We have tried to take this possibility into account in reviewing the evidence on the overall effects of public participation. It is important to note, how- ever, that findings on fine-grained issues, such as about which processes produce which kinds of desired results, are much less vulnerable to such biases. The literature shows considerable variation in the degree of success, depending on the context and the process used, so whatever publication biases may exist have not precluded reporting of less than ideal results. The literature on meta-analysis provides guidance on detecting and correcting 75
76 PUBLIC PARTICIPATION for publication bias, and these methods can be used in the study of public participation. DOES PUBLIC PARTICIPATION IMPROVE RESULTS? We have already noted that the term âpublic participationâ connotes a highly diverse set of activities. As discussed in Chapter 1, processes can be seen as more or less participatory along several dimensions, notably breadth (who is involved), timing (how early and at how many points in the overall decision-making process they are involved), intensity (e.g., the amount of time and effort participants spend and the degree of effort made by conveners to keep them involved), and influence. Although processes can be considered more participatory to the extent that they score more highly on these dimensions, the available research does not always make explicit distinctions among the dimensions. In reviewing the evidence, we comment on the effects of particular dimensions of participation when the evidence allows. We return in Chapter 5 to the issue of whether increases in participation along particular dimensions are associated with better outcomes. As we detail in Chapter 2, public participation processes also vary in their objectives (e.g., to make assessments or inform decisions, to reach consensus, or only to identify options and issues) and in the kinds of decisions they address. In Chapter 7, we examine how these differences may affect results. This section shows that, on average, public participation is associated with better results, in terms of criteria of quality, legitimacy, and capacity. However, participatory processes can sometimes lead to undesired results that may be worse than what would have resulted from less participatory processes. The considerable variation in results is due largely to variation in the processes used to conduct public participation activities and in the extent to which these processes address the challenges posed by specific aspects of the context of participation. This evidence comes from a conver- gence of results from several sources. Experimental and Quasi-Experimental Studies An important source of evidence comes from experimental and quasi- experimental studies. Only a few experimental studies of participation processes have been conducted using control groups and random assign- ment to provide internal validity. Even fewer of these address environmental decision processes. Moreover, their practical value is unclear because it is hard to get an adequate sample of processes for statistical comparison, even if only one variable is manipulated. Quasi-experimental studies are a more common source of evidence on environmental decisions. These
THE EFFECTS OF PUBLIC PARTICIPATION 77 studies compare more and less participatory processes that occur naturally for similar assessments or decisions. Although such studies cannot ensure the level of internal validity provided by randomized control groups, they permit comparative observation in real-world settings. An experimental study by Arvai (2003) shows that when people believe that a decision resulted from a public participation process, they are more likely to accept the decision, an indication of legitimacy. Arvai surveyed 378 individuals about a decision by the National Aeronautical and Space Administration to deploy a nuclear generator in space exploration. All individuals received the same information about the risks and benefits in- volved in using the nuclear generator. However, some were told that mission planning, including the decision to use the generator, was based on expert knowledge and experience, while others were told that decisions about mis- sion planning, objectives, design, and the use of the generator were based equally on active public participation and on expert knowledge and experi- ence. The individuals who were told that the decision incorporated public participation were significantly more supportive of the decision itself, as well as the process by which the decision was reached. They also expressed greater support for similar future missions, even though the two groups ranked risks from nuclear generator use similarly. A number of studies by Fishkin and collaborators (e.g., Fishkin, 1997; Farrar et al., 2003, 2006; Fishkin and Luskin, 2005; List et al., 2006) used random samples of individuals in carefully planned participatory events, called deliberative polls, addressing a number of public policy is- sues. Participants were provided with balanced briefings on a policy issue, engaged in informal discussions in their everyday milieus, and participated in professionally facilitated small-group deliberations with opportunities to question experts. They were interviewed before and after the process. In some of these studies, control groups of individuals were interviewed with- out receiving briefings or participating in organized deliberations. These studies found that participation changed peopleâs opinions on the issues and that people who engaged in deliberative polls were more likely to vote afterward, which we interpret as a positive outcome. Participants gained factual information about the issue at hand, as well as more general politi- cal knowledge. People who learned the most changed their opinions the most, and changes in opinion were unrelated to social status. Furthermore, participants were more consistent and predictable in their opinions after their participation. Although the deliberative groups did not come to con- sensus (after deliberation, they were as likely to become more polarized as less), there was an increase in the extent to which the order of preferred options became more consistent across participants.1 These findings suggest that participatory processes increase participantsâ capacity through learn-
78 PUBLIC PARTICIPATION ing and increased motivation to participate, as well as developing greater consensus on at least some aspects of preference ordering. A few studies have considered the effects of a deliberative process on responses to a standard willingness-to-pay survey. For example, Dietz, Stern, and Dan (in press) randomly assigned survey participants to express willingness to pay to plant trees to offset carbon emissions, either in a sur- vey-only mode or after a small-group deliberation structured by the nomi- nal group technique. Deliberation increased the number of issues considered in answering the question, had some effect on attention to social costs and benefits, and reduced the effect of personal predispositions on stated will- ingness to pay, but it did not change mean or median willingness to pay. There was no evidence of within-group convergence indicative of âgroup think.â Overall, the results seem to indicate that even rather minimal deliberation enhanced the quality of decision making: more factors were considered, and there was less influence of personal predispositions. A form of quasi-experimental evidence specific to environmental deci- sion making is provided by a few studies that compared sets of decision processes in the same organizational context that used systematically differ- ent formats, one more participatory than another. Such comparisons lack experimental control in that decisions were not randomly assigned to more or less participatory formats. In one such study, Langbein (2005) compared six traditional and eight (presumably) similar negotiated rule-makings con- ducted by the U.S. Environmental Protection Agency (EPA). Langbein inter- viewed and collected survey responses from 152 representative participants in these rule-makings and found consistently better outcomes (see below for criteria) for the negotiated rule-makings, which are more participatory than conventional rule-makings on the dimensions of intensity and influence. Across cases, negotiated rule-makings received more positive ratings from participants on 13 of the 15 dimensions studied, including quality of the scientific analysis, cost-effectiveness, ability of the rule to survive a legal challenge, and overall assessment of the rule-making process. They received worse ratings only on the cost of the rule for the respondentâs organiza- tion. The negotiated and conventional processes were judged equal on the ability of the EPA to implement the rule. Participants were found to prefer negotiated âto conventional rulemaking mostly because they believe they get a better rule out of the process, and partly because some aspects of the process, but not all, work wellâ (Langbein, 2005:20). Negotiated rule-making resulted in participantsâ judgments that more issues were settled and that the issues were clearer. Perceived complexity of the issues, however, was greater in the negotiated rule-makings, and complexity was associated with significantly more negative evaluations, although not enough to overwhelm the positive overall effects of the ne- gotiated process. Although participants believed more issues were settled
THE EFFECTS OF PUBLIC PARTICIPATION 79 in negotiated rule-makings, there was no difference in the frequency of subsequent litigation. On the basis of participantsâ subjective judgments, their participation appears to have resulted in equivalent decision quality and legitimacy, possibly increased capacity, and overall greater satisfaction with the process. Much of the empirical literature on public participation is based on surveys of participants. Although such research is valuable, results should be interpreted cautiously. One reason is that the statistical comparisons are made across individuals rather than across rule-makings. Analyzing individualsâ responses increases the ability to find statistically reliable re- sults, but the responses from participants in the same rule-making are not statistically independent (this lack of independence can be overcome in some analyses, such as Langbeinâs use of robust standard errors). Responses are also subject to âhalo effects,â in which a participantâs judgment on one outcome variable colors judgments on others (see, e.g., Coglianese, 2003a,b,c). If many of the participants in the same process form a com- mon judgmentâor have a common halo effectâthese judgments can be multiplied to generate a spurious conclusion. And as already noted, the attribution of causation with quasi-experimental designs is never as certain as it is with pure experiments. A series of studies compared the results of watershed management planning in 20 estuaries operating under the National Estuary Plan (NEP), which sets guidelines and provides technical support for a participatory planning process, with results in 10 non-NEP estuaries (Schneider et al., 2003; Lubell, 2004a,b; Lubell and Leach, 2005). On average, stakeholders in NEP watersheds rated their estuary policies as more effective than did stakeholders in non-NEP watersheds (Lubell, 2004a); reported stronger perceptions of trust, fairness, and conflict resolution (Schneider et al., 2003); and showed a higher level of consensus (Lubell, 2004a). Lubell and Leach (2005) reported that the intensity of the participatory processes, as measured by an indicator of stakeholder teamwork, was positively related to all measures of success used in the study. Although NEP estuaries did better on a variety of indicators of suc- cess based on participantsâ reports, they did more poorly on a behavioral indicator of cooperation among the participants, apparently an indicator of support in the implementation phase (Lubell and Leach, 2005). Possible explanations of this finding include that it is harder to change behavior than attitudes, that the management problems were more difficult in watersheds that entered the NEP (i.e., a selection effect), and that the NEP process yielded only symbolic progress. As with the Langbein review, the results should be interpreted with caution because the data compared individuals rather than cases of participation.
80 PUBLIC PARTICIPATION Multicase Studies Many hundreds of case reports of environmental public participation can be found in the peer-reviewed literature and in reports that have not been subject to peer review. Because case studies are idiosyncratic in how they assess degrees of public participation and how they define successful outcomes, we do not consider single case reports by themselves as strong evidence in developing an overall evaluation of public participation. More useful are studies that examine multiple cases, some more participatory than others, using internally consistent definitions of the key variables. In addition to the studies already discussed, useful evidence comes from stud- ies that examine the outcomes of multiple public participation processes that are not systematically different because of a specified difference in format, but that vary in degree along dimensions of participation and are coded consistently on such dimensions and on indicators of results. The most extensive such study was by Beierle and Cayford (2002), who coded 239 cases into five categories from least to most intensively participa- tory, according to the mechanism used: from public meetings and hearings at the low-intensity end of the spectrum, through advisory committees not seeking consensus to advisory committees seeking consensus, and finally to negotiations and mediations. More intense mechanisms were strongly associated with high ratings on an aggregate success measure: less than one-quarter of the processes featuring public meetings and hearings were rated highly successful, compared with over 90 percent of the negotiations and mediations. Beierle and Cayford (2002:48) noted, however, that the more intensive mechanisms sometimes achieve consensus by âleaving out participants or ignoring issuesââthey look more successful from inside the process but may not yield better results when the participation moves out to the broader society. Several other attributes of the participatory process had stronger influences than the mechanism used. These results are discussed in Chapter 5. A series of studies of watershed partnerships, with participatory pro- cesses aimed at collaborative decision making and implementation (Leach and Pelkey, 2001; Lubell and Leach, 2005; Sabatier et al., 2005), supports the conclusion that when the participation was broader and more intensive, indicators of success in watershed management improved. For example, the studies found that participantsâ perceptions of human and social capi- tal (an indicator of capacity) were more positive when there were more participants, when all critical parties were present, and when deliberation was more intense (in hours per month), although the first two effects were labeled as only âmarginally significant.â In separate analyses at the level of partnerships rather than of participants, the strength of agreement (an in- dicator of legitimacy) and the breadth of project implementation were both
THE EFFECTS OF PUBLIC PARTICIPATION 81 positively related to the duration of the process (other intensity variables were omitted from these analyses). Leach (2006) reviewed the results of 25 empirical studies based on one or more cases that drew conclusions about the factors that were keys to success in public participation in the U.S.D.A. Forest Service activities. He coded each stated conclusion in these studies as relating to one of 21 âthemesâ that represented potentially important attributes of the process, the participants, or the context, and noted the number of studies sup- porting or detracting from the conclusion that each was a key to success. Three of the keys to success are reasonably interpreted as reflecting the intensity, breadth, or duration of participation. Having a âcomprehensive and sustained processâ was noted as a key to success in 12 studies, and none concluded that it detracted from success. Having âbroad or inclusive participationâ was a key to success in 10 studies, though 6 studies detracted from success. Typical of the negative results was the study by Floyd et al. (1996), which reported a negative correlation between number of parties and positive outcomes. Finally, seven studies cited continuity of participa- tion as a key to success, and none saw it as a negative factor. Most of the studies highlighted the need for continued participation of the same Forest Service officials. Bradbury (2005) reviewed 6 years of observations and surveys of eight sites at which the U.S. Department of Energy (DOE) had organized public participation processes to advise on the cleanup of hazardous materials, mostly associated with nuclear weapons production. She also reviewed a study of four similar sites managed by the Department of Defense (DOD). The review of all 12 studies showed that intensity of the public participa- tion process was strongly associated with success, as shown by subjective indicators of quality, legitimacy, and capacity gathered both from observa- tion and from participant surveys. Specifically, the two DOE sites where advisory boards met for 2 days bimonthly and had both standing executive committees and technical committees were rated highly on all indicators in both 1997 (survey data) and 2002 (observational data). The four sites that had the same committee structure but met for only 3-4 hours per month received ratings slightly above âmediumâ on all indicators in both time pe- riods, and the two sites that lacked both the intensive committee structure and the intensive meeting times were rated at or near the lowest level on all indicators at both time periods. Outcomes were generally less satisfactory at DOD sites, where participation was less intensive than at DOE sites. (At two DOD sites, participation was little more than provision of information to the public.) Representation of the parties had a more complex relationship to out- comes. The two DOE sites that were noteworthy for the diversity of view- points among the participants consistently scored at the top of the group on
82 PUBLIC PARTICIPATION subjective measures of success, but the three sites where important parties remained uninvolved (activists in two cases, business and local government in the third) had average outcomes slightly better than the three sites with intermediate breadth of representation. Bradbury (2005) provides a more detailed discussion of the complexities of the representation issue in these cases. Mitchell et al. (2006) summarized the results of a rich set of studies of âglobal environmental assessmentsââefforts to evaluate the state of knowl- edge about particular global environmental phenomena, such as climate change and ozone depletion, the likelihood of various scenarios of future change, and the likely benefits and costs of alternative policies. The usual practice for conducting such assessments has not been participatory: many assessments strive to ensure the credibility of the science by involving only scientists and insulating them from interested publics and political actors. However, some global environmental assessments have varied considerably from the experts-only norm, and this variation was a source of some of the major insights of the review. The extent of stakeholder participation was strongly and positively associated with the perceived impacts of the assessments. It is worth emphasizing that studies of global environmental assess- ments come from a research tradition quite separate from public participa- tion research: they rarely cite or draw on major works on environmental public participation. Mitchell and colleagues developed their conceptual framework inductively from the insights of assessment practitioners, their own initial studies of a few cases, other case-comparison studies of global assessments that appeared during the life of the project (Andresen et al., 2000; Young, 2002), and previously published analyses from their research group (Jasanoff and Martello, 2004; Farrell and JÃ¤ger, 2006). They defined their topic in terms of âthe influence of scientific information on policyâ (Clark, Mitchell, and Cash, 2006:6); consequently, the main outcome of interest was the degree to which the assessments influenced the think- ing of and particularly the actions of various policy-making audiencesâ an implementation-dependent rather than an immediate outcome, in the terms of this study. These studies did not set out to be studies of public participation. The results, however, are strikingly consistent with those from research on environmental public participation. Mitchell and colleagues (2006:326) concluded that âparticipation explains much of the variation in the influ- ence of our assessments.â Influence depended not only on the scientific credibility of assessments, which was usually the primary stated objective of the assessments, but on their legitimacy with audiences and on their âsalience,â that is, on whether they provided results that audiences saw as relevant to their decision making. There were sometimes trade-offs among
THE EFFECTS OF PUBLIC PARTICIPATION 83 credibility, legitimacy, and salience, such as when excluding nonscientists sacrificed legitimacy and salience for scientific credibility or when assess- ments tried to achieve salience by going beyond what the science could support and thus sacrificed scientific credibility. However, most importantly for the present study, the research found that there are ways to avoid such trade-offs: these ways depend on broadening participation to provide ac- cess to local knowledge that is essential for a credible assessment and to improve stakeholdersâ understanding of the foundations of the assessment findings. Mitchell et al. (2006:324) concluded that âthe effectiveness of assessment processes depends on a process of coproduction of knowledge between assessment producers and potential assessment user groupsâ and that âstakeholder participation fosters salience, . . . credibility, . . . [and] legitimacyâ (325). Finally, the study emphasized the importance of capacity building, both among scientists and nonscientists, as a way to foster the needed coproduction of knowledge and to increase the influence of envi- ronmental assessment processes. A recent report from the National Research Council (2007a) examined a set of âglobal change assessmentsâ that overlap with those examined by Mitchell and colleagues and offered several recommendations for action based on judgments about the benefits and costs of public participation in these assessments. The report recommended that âappropriate stake- holdersâ be identified and engaged âin the assessment designâ and noted âthe advantages of broad participation,â but noted the costs in terms of efficiency and the need to build capacity for âdiverse stakeholders and as- sessment participantsâ (National Research Council, 2007a:S-7âS-8). The results of the global assessment studies can thus be fairly summa- rized in the terms of the present effort as follows: when public participation involves the producers and users of environmental assessments in the copro- duction of decision-relevant knowledge, it simultaneously improves quality, legitimacy, and capacity outcomes as indicated by participantsâ judgments.2 It may be expected that by so doing, participation increases the likelihood that the assessments will lead to implementation. Capacity building is both an outcome of such participatory processes and a contributor to their suc- cess in the policy arena. Moser (2005), who examined the case of the First National Assessment of the Potential Consequences of Climate Variability and Change (a set of processes organized separately for different geographic regions and economic sectors) using the conceptual categories developed for the present study, similarly found that assessments characterized by more intensive involvement of stakeholders were of higher quality, legitimacy, and capacity as judged by participants in the assessment process. These assessment studies thus point to both the breadth and intensity of public participation as important influences on desired outcomes. Tuler (2003) reviewed the findings of 11 studies (not discussed else-
84 PUBLIC PARTICIPATION where in this section), each of which examined multiple cases (from 7 to 118) in a specific environmental policy area. He classified the factors associ- ated with success in terms of competence, legitimacy, and capacity. Broad representation of interested and affected parties appeared as a contributing factor to each of these outcomes in the multicase reviews. It was found to contribute to the authorsâ aggregate measure of success in a study of 30 cases of forestry planning (Selin, Schuett, and Carr, 2000). Broad represen- tation was also related to the indicators of decision quality and legitimacy used in Aronoff and Gunterâs (1994) review of 7 cases of public participa- tion in managing technological hazards and in the review by Henry S. Cole Associates (1996) of 11 cases of public participation in cleanup of hazard- ous waste sites. These studies also suggest, however, as did Leachâs (2005) study in the forestry context, that different ways of implementing broad participation can have very different results (see Tuler, 2003, for a more detailed discussion). Finally, Williams and Ellefson (1996), in a study of 40 natural resource partnerships, found that representation of all stakeholders led to decreased resistance to the efforts of the partnership, a measure of support for implementation. Practitionersâ Experiences Practitionersâ judgments about the overall effects of participation are based on their assessments of the results of processes that they consider to be either more or less participatory. These judgments are reflected in numerous handbooks for practice (e.g., Carpenter and Kennedy, 1988; Pritzker and Dalton, 1990; Creighton, 1992, 1999, 2005; Canadian Round Tables, 1993; Doyle and Straus, 1993; World Bank, 1996; Presidential/Congressional Commission on Risk Analysis and Risk Management, 1997a,b; SPIDR, 1997; Policy Consensus Initiative, 1999; Susskind et al., 1999; Bleiker and Bleiker, 2000; U.S. Environmental Protection Agency, 2000a,b; Institute for Environmental Negotiation, 2001; Adler and Birkhoff, 2002; Bingham, 2003; Chambers, 2003; International Association for Public Participation, 2003; McKeown, Hopkins, and Chrystalbridge, 2003; Gastil and Levine, 2005; Leighninger, 2006; Pomeroy and Rivera-Guieb, 2006). Although their definitions of degree of participation are qualitative and not always explicit, and they are therefore almost certain to vary across practitioners, every one of these practitioner handbooks strongly supports two fundamental con- clusions about environmental public participation from the practitionerâs point of view: that making environmental decisions more participatory can yield improved results, and that such results are contingent on a variety of process and contextual factors. We discuss these factors more thoroughly in the following chapters.
THE EFFECTS OF PUBLIC PARTICIPATION 85 Summary These lines of evidence demonstrate that there is great variation in the immediate outcome of public participation. They show that under many conditions, including some that are likely to apply often in environmental contexts, processes that are more participatory along the dimensions of breadth, timing, intensity, and influence lead to improved overall outcomes. The evidence also strongly suggests that public participation processes can lead to undesired results that may be worse than what would have resulted from less participatory processes. The evidence on both counts consists largely of associations between aspects of process and aspects of results, as discussed in Chapters 4-6. The strength of this evidence could be bolstered by complementing the existing literature with more experimental studies and carefully structured quasi-experiments that provide strong internal validity regarding cause- and-effect relationships, even if at some cost in external validity. Never- theless, the available evidence converges on fairly consistent results across several methods of measuring participation and outcomes and across a wide variety of environmental assessment and decision contexts. Evidence from such varied sources and from varied families of cases that are quite similar on variables other than participation provide confidence that the observed associations are due in substantial degree to causal links between participation and outcomes. However, the very limited evidence going beyond immediate effects is mixed. On one hand, regulatory negotiations lead to greater participant satisfaction, but no fewer cases of litigation, than conventional regulations. On the other hand, studies of groups of environ- mental assessments seem particularly consistent in indicating that broad participation increases not only the scientific credibility of assessments, but also their effects on policy. Of course, there are many cases in which participation processes have not managed to enhance quality, legitimacy, and capacity (see Coglianese, 1997; Carr, Selin, and Schuett, 1998; Imperial, 1998, 2005; Weber, 2003, among many others). Indeed, one of the bases of our analyses in the rest of the report is studies that show variation in the outcomes of participa- tion processes and identify the reasons for that variation. So while our overall conclusion is that participation enhances environmental assessment and decision making, it remains critical to understand the causes of the considerable observed variation among outcomes. The issue for practice is to specify the conditions that favor successful outcomes and to seek them out or create them. In Chapters 4 through 8, we review the evidence on how conditions in the practice and context of public participation affect its outcomes.
86 PUBLIC PARTICIPATION ASSOCIATIONS AMONG RESULTS: CAN YOU HAVE IT ALL? The evidence discussed in this section shows that the desired immedi- ate results of public participation are positively correlated: one generally finds similar levels of success in terms of quality, legitimacy, and capacity. Available evidence supports with high confidence a conclusion that trade- offs among these types of results are not inevitable. Across a wide variety of environmental assessment and decision contexts, there are practices that can simultaneously promote all three positive results. Government officials and critics of public participation sometimes ex- press the concern that although intense public involvement may increase the legitimacy of decisions, it is likely to reduce their quality (see ChapterÂ 2). The available evidence does not support either the hypothesis that such trade-offs are inevitable or the hypothesis that participatory processes that promote the legitimacy of an assessment or decision necessarily detract from its quality. Rather, the data strongly indicate that there are positive relationships among the various desired results, such that processes that perform well on one outcome measure are likely to perform well on other measures also. Case-Based Evidence The broadest relevant database comes from the work of Beierle and Cayford (2002) in their ratings of 239 public participation cases covering a great diversity of environmental decision contexts and processes on five outcome variables, which they called social goals: improving the substan- tive quality of decisions, incorporating public values into decisions, resolv- ing conflict among competing interests, building trust in institutions, and educating and informing the public. In our terms, the first is a measure of quality, the second may relate to both quality and legitimacy, the third and fourth are legitimacy indicators, and the last is a measure of capacity. The study found that each type of result was positively correlated with each of the others at a statistically significant level, with a median correlation coefficient of 0.46 (range = 0.16â0.57). Beierle and Cayford inferred that âaggregate successâ was a meaningful concept; their aggregate measure had correlations with the individual outcome measures that ranged from 0.56 to 0.73, although these part-whole correlations are somewhat inflated because the whole was created by averaging the parts. A reanalysis of that dataset for this study (Dietz and Stern, 2005) indicated that the five outcome vari- ables could be represented as measures of a single underlying success factor based on standard factor analysis and scaling criteria. The evidence from the case-comparison studies conducted for this study is consistent with the conclusion that different types of desirable results are
THE EFFECTS OF PUBLIC PARTICIPATION 87 positively correlated. These studies compared various outcome indicators across multiple public participation processes dealing with the same type of environmental problem. One example is the work, already mentioned, of Lubell and Leach (2005), who explored the effectiveness of environmental decisions in collaborative watershed partnerships involving 20 estuaries under the National Estuary Program (and 10 nonpartnership estuaries) and over 70 partnerships under the Watershed Partnership Project (WPP), combining and reanalyzing data from previous studies. In the NEP cases, in- dicators of legitimacy, such as perceived fairness, were among the strongest predictors of indicators of benefits to the watershed, project implementa- tion, and degree of consensus and policy agreement, with various contex- tual factors held constant statistically. For the WPP cases, similar indicators were among the strongest predictors of level of agreement (an indicator of support for implementation) and of benefits to human and social capital (an element of capacity). Moreover, agreement was in turn a strong predic- tor of perceived effect of the program on the watershed. Bradburyâs (2005) review of environmental restoration programs at eight DOE and four DOD facilities found similarly that sites that were rated positively or negatively on one type of result, whether by participants or researchers, tended to be rated similarly on others. The review by Tuler (2003) conducted for the panel covered 17 case- comparison studies, most of them not summarized above and almost all of them essentially qualitative in the methods used. He found that several specific attributes of the participatory process that were conducive to one type of outcome, such as legitimacy, were also found to be conducive to other types of outcome, such as improved quality and expanded capacity, using the indicators employed in the studies. For example, broad repre- sentation of stakeholders was found to contribute to indicators of both legitimacy and capacity in participation about decisions regarding techno- logical hazards (Aronoff and Gunter, 1994), Superfund sites (Henry S. Cole Associates, 1996), and forest planning (e.g., Selin, Schuett, and Carr, 2000). High-quality communication and quality of decision-relevant information were also associated with increased legitimacy, capacity, and decision qual- ity in some of the families of cases. There was no case family in which a process attribute was positively related to one of the major types of results and negatively related to another. Tuler (2003) did conclude, however, that the intensity of the mechanism used may be negatively associated with legitimacy if intensity is achieved by methods that limit representation. Intensity was positively associated with competence, a measure of quality in our framework.3 Thus, there may be conditions under which promoting success on one dimension can interfere with another dimension of success; however, these may result from specific process choices rather than an in- herent incompatibility of outcomes.
88 PUBLIC PARTICIPATION In sum, the best available case-based evidence indicates that positive results in terms of any one process outcome of environmental public par- ticipation are usually associated with positive results on others and that negative associations are rare. The evidence strongly suggests that there are ways of implementing public participation that can promote multiple desired results together and, moreover, that when practitioners are success- ful, they are frequently successful on multiple dimensions at once. These correlations do not necessarily mean that the different types of results are causally interrelated so that efforts focused on producing one of them will necessarily produce the others. However, it has been suggested that some positive causal relationships exist. For example, Lubell and Leach (2005) provide evidence consistent with one causal chain: capacity promotes legitimacy, which in turn con- tributes to decision quality. But their data do not provide strong evidence that building consensus causes better decisions for two reasons. Decision quality was measured by participantsâ judgments, which could have been influenced by feelings about the process as well as by characteristics of the decision themselves (the unit of analysis in the relevant regression analyses was the participant, not the case). Also, the analyses were based on causal modeling of data at one time rather than repeated observations over time. Thus, the data are consistent with Lubell and Leachâs causal model but, as in other areas of research on public participation over time, experimental and quasi-experimental evidence would enhance understanding of causal processes. Practitionersâ Experience Unlike researchers who collect data only at the end of a process, prac- titioners of environmental public participation observe these processes over time and often have observed dozens of such processes. Their observations provide a particularly useful window on the associations and possible causal connections among quality, legitimacy, and capacity. There are three somewhat distinct traditions of practice in environmen- tal public participation. One has roots in the practices of conflict resolu- tion, negotiation, mediation, and game theory (e.g., Bacow and Wheeler, 1984; Goldberg, Green, and Sander, 1985; Bingham, 1986; Moore, 1986; Carpenter and Kennedy, 1988; Crowfoot and Wondolleck, 1990; Fisher, Ury, and Patton, 1991; Raiffa, 1994; Dukes, 1996; Saunders, 1999; Daniels and Walker, 2001). A second emerges from planning and organizational development (e.g., Arnstein, 1969; Avery et al., 1981; Mansbridge, 1983; Susskind and Cruikshank, 1987; Doyle and Straus, 1993; Gastil, 1993; Chrislip, 1994; Schwartz, 2002; Creighton, 2005; Gastil and Levine, 2005; Schuman, 2005; Kaner, 2007). The third has its origins in risk assessment,
THE EFFECTS OF PUBLIC PARTICIPATION 89 ecological assessment, impact assessment, risk communication, and other kinds of environmental analysis (National Research Council, 1983, 1989, 1994, 1996, 2007b; Dietz, 1987; Fiorino, 1990; Chess, Tamuz, and Green- berg, 1995; Presidential/Congressional Commission on Risk Assessment and Risk Management, 1997a,b; Wondolleck and Yaffee, 2000). Public participation practitioners in the dispute resolution and planning traditions, in which consensus is often the most salient objective, devote much attention in their handbooks to broader involvement and improved communication. They emphasize understanding and communicating the interests that underlie diverse views, sharing information and developing creative options, using objective criteria, building trust, including the pub- lic early in the process, keeping the process open and flexible, maintaining transparency, and monitoring outcomes. This emphasis suggests that practitioners in this tradition believe that processes focused closely on improving legitimacy yield benefits in terms of the full range of desired outcomes, including decision quality and capac- ity. Experienced practitioners of environmental public participation know that legitimacy does not guarantee qualityâthat it is a mistake to ignore aspects of the process that directly promote the quality of assessments and decisions. In other words, the popularity of or support for a particular assessment or decision does not ensure the technical quality of the as- sessment or decision. Practitionersâ advice for consensus building in the context of contested science recognizes the importance of making assump- tions transparent, coproduction of information and analyses, getting the right information, clarifying the relevance of the information to decisions, and addressing problems that participants may have in understanding and communicating about scientific information (e.g., Ozawa, 1991; Adler and Birkhoff, 2002; Bingham, 2003). It is worth noting that at least some participants in environmental deci- sion processes are also well aware of the dangers of poorly informed deci- sions. Consequently, they are unlikely to accept as legitimate any process that does not pay specific attention to elements of decision quality, such as gathering all the relevant information, subjecting it to criticism from a variety of sources, and ensuring that decisions are consistent with the best available information. Thus, it seems likely that at least certain practices that are conducive to quality are also conducive to legitimacy. Practitioners in the environmental assessment tradition tend to focus primarily on decision quality: ensuring that public policy decisions are based on accurate scientific analysis. In early writings in this tradition, the values and judgments of nonscientistsâespecially public officials, but also other partiesâwere to be insulated from the scientific analysis (National Research Council, 1983, is frequently referenced in this regard). For some writers in this tradition, too much concern with legitimacy in the eyes of
90 PUBLIC PARTICIPATION nonscientists was considered a potential threat to good science (e.g., Starr, 1969). Over time, practitioners in this tradition have been coming to many of the same conclusions as practitioners in the dispute resolution and planning traditions. Writings about public participation in the environmental assess- ment tradition have been relatively scarce until recently, in part because this tradition is primarily concerned with the practices of analysts. An important statement about public participation practice in this tradition was the 1996 National Research Council (NRC) report, Understanding Risk, which drew heavily on practical experience in characterizing risks for nonscientists. The recommendations offered for practice in that report were justified primarily in terms of their contribution to the quality of environmental decisions, al- though legitimacy was also considered. The report specified four guidelines for deliberative practice (National Research Council, 1996:4-5): â¢ seek broad participation, especially in the early âproblem formula- tionâ phase of the process, and with respect to interested and affected par- ties that âare particularly at risk and may have critical information about the risk situationâ and lack the capacity to participate effectively, âit is worthwhile for responsible organizations to arrange for technical assistance to be provided to them from sources that they trust.â â¢ âclearly and explicitly inform the participants at the outset about . . . external constraints likely to affect the extent of deliberation possible or how the input from deliberation will be used.â â¢ âstrive for fairness in selecting participants and in providing, as ap- propriate, access to expertise, information, and other resources for parties that normally lack these resources.â â¢ build flexibility into deliberative processes. This set of guidelines is clearly aimed at achieving a full and informed deliberation that can provide the best available information relevant to the decision at hand. Nevertheless, it includes elements that are familiar from the writings of practitioners in the dispute resolution tradition, whose initial driving concerns were presumably legitimacy and agreement. Like the NRCâs Understanding Risk, the more recent work of the Global Envi- ronmental Assessment Project (Mitchell et al., 2006) and the National Re- search Council (2007a) review of global change assessments also begin with a primary interest in decision quality and end with practical recommenda- tions that strongly emphasize broad public participation and considerations of legitimacy as well. Similarly, the Presidential/Congressional Commis- sion on Risk Assessment and Risk Management (1997b:41), drawing its conclusions from practical experience, recommended that âpublic stake- holdersâ be placed in âprominent rolesâ in risk assessment and manage-
THE EFFECTS OF PUBLIC PARTICIPATION 91 ment processes and concluded (1997b:48) that public participation in the comparative risk process improves understanding of competing priorities, provides an appreciation of the complexity of decision-making, and can stimulate new insights into solutions. As a result of increased communica- tion among institutions and interest groups, new avenues of cooperation might be established. Adversarial relationships among interest groups and jurisdictional conflicts among agencies might not disappear, and could even be intensified, but [public participation in] comparative risk projects have revealed unexpected agreement among parties and enhanced understanding of differences in perspectives and values in some cases. Most important, experience has shown that the process itself can help to build coalitions that favor priority setting and shifting resources to the identified priorities. Broader public support for a common agenda might allow agencies, state legislatures, and Congress to move money and staff into priority problems with less litigation and less controversy. We see this convergence as an important trend: Practitioners of public participation from these three, somewhat distinct traditions of dispute reso- lution, planning, and environmental assessment have increasingly converged on similar ideas of best practice. Over time and with increased experience regarding difficult environmental decisions, each group of practitioners has come to recognize the critical importance of the central concerns of the other, with writings on risk assessment increasingly recognizing the importance of broad representation and of deliberation about judgments and values, and writings on environmental dispute resolution increasingly emphasizing the need to incorporate relevant scientific knowledge. This observation suggests that in most areas of environmental assessment and decision making, it is both imperative and possible to develop practices that promote decision quality and legitimacy at the same time. It also sug- gests that practices recommended in the various practitioner traditions, such as representation of the full range of interested and affected parties, early involvement of those parties, and flexible decision processes capable of taking new information into account, in fact contribute to both quality and legitimacy. Technical assistance to important parties that are not able to participate fully may contribute to all three objectives: capacity, legiti- macy, and decision quality. In this light, the empirical finding should not be surprising that participatory processes that are successful on one of these dimensions are also likely to be successful on others. CONCLUSION The evidence from the best available empirical studies of environmental public participation processes and from the experiences of practitioners in the dispute resolution, planning, and environmental assessment traditions
92 PUBLIC PARTICIPATION converges on the conclusion that best practices in public participation can advance decision quality, legitimacy, and capacity simultaneously. This does not imply that a standard set of guidelines for practice will work equally well and achieve all three objectives in all situations. In some situations, particular issues require special attention. For example, when the inter- ested and affected parties to a decision seriously mistrust each other or the responsible public authority, special attention to building legitimacy may be necessary. When the relevant science is known to be in dispute, special attention to issues of scientific quality may be necessary. When certain critical parties lack sufficient scientific understanding to participate effec- tively, technical assistance to these parties may be essential to any desirable outcome. It is also likely that certain attributes of public participation practice or its context strongly affect particular desired results but are unrelated to oth- ers. For example, some public participation practices may be particularly valuable for enhancing legitimacy, while others may be particularly valuable for educating the public. We consider such possibilities in the following chapters, in discussing how various practice and context factors may affect particular kinds of outcomes. It is also possible that in certain situations that are not well represented in the available data, there may be trade-offs among desirable outcomes. Notes 1In technical terms, participants approached greater âordinal single- peakednessâ in their opinions. That is, following deliberation of an issue with more than two possible decision options, more people ordered their preferred options in a way that was logically consistent with othersâ order- ing of preferred options. Whether or not they agreed on which specific options they preferred, they did agree more on the relationships among the options. Ordinal single-peakedness is a desired characteristic of preferences in theoretical work on social choice paradoxes (Condorcet, 1785; Black, 1948; Arrow, 1951; Niemi, 1969; Miller, 1992; Knight and Johnson, 1994; List, 2001; Dryzek and List, 2003; Gehrlein, 2004). In theory, it increases the ability to avoid, with democratic majorities, the instability and manipu- lability of âcycling,â which can occur when equal proportions of a popula- tion prefer each of three or more options, but voting takes place between only two options at a time. Single-peakedness helps to meaningfully aggre- gate individual preferences into social choices by arraying options along a continuum so that the median preference can prevail. 2Certain negotiated rule-making efforts, such as EPAâs federal advisory committee that produced the Disinfectants and Disinfection By-Products
THE EFFECTS OF PUBLIC PARTICIPATION 93 Rule, also have attributes of coproduction of information (National Re- search Council, 1996). 3Practitioners also express concerns that unexamined assumptions about the relative priorities among desired characteristics of public partici- pation may affect process choices implicitly or explicitly, creating trade-offs, for example, between the desire to make decision making more efficient and the time and attention needed to achieve joint gains or improved quality in the agreements reached.