Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
4 Public Participation Practice: Management Practices T he way a public participation process is conducted can have more influence on overall success than the type of issue, the level of government involved, or even the quality of preexisting relation- ships among the parties. Thus, those variables over which the convening agency has the greatest control turn out to be key to achieving the desired results. In this chapter and the two that follow we review the evidence with regard to the practice of participation. We find that good outcomes can be obtained even in difficult circumstances. The evidence discussed in this chapter supports generic principles of program management that apply to a broad range of programs managed by government agencies and other organizations and specifically to public participation programs. Many of the principles stated in these three chapters strongly echo those offered in other studies (e.g., Presidential/Congressional Commission on Risk Assessment and Risk Management, 1997a; Office of Management and Budget and Presidentâs Council on Environmental Quality, 2005) and in multiple works by practitioners and scholars of public participation practice (e.g., Dukes, 1996; Daniels and Walker, 1997; Susskind, Thomas- Larmer, and Levy, 1999; Wondolleck and Yaffie, 2000; Dukes and Firehock, 2001; Institute for Environmental Negotiation, 2001; U.S. Environmental Protection Agency, 2001; Creighton, 2005). We reiterate such previously stated principles for two reasons. First, it is important to recognize prin- ciples that are supported by a convergence of evidenceânot only by prac- titionersâ experience, but also by the kinds of careful case-study research, case-comparison studies, and basic social science knowledge examined in 95
96 PUBLIC PARTICIPATION these chapters. Second, some often-stated principles bear repeating because they are so often violated in practice. The principles presented in these three chapters overlap to some degree, as is often the case in describing effective practices. We begin by summariz- ing the main finding from our review of the evidence on management and then discuss more specific points along with the supporting evidence. Basic principles of program management apply to environmental public participation. When government agencies engage in public participation processes with clarity of purpose, commitment, adequate resources, appro- priate timing, an implementation focus, and a commitment to learning, they increase the likelihood of good results. When they fail to do these things or lack adequate organizational capacity, the results are likely to fall short of the potential of public participation. Public participation activities share a number of features with other programs that government agencies and other organizations run: they re- quire planning, resources, coordination, implementation, and the like. It should therefore not be surprising to see that much of the advice on how to run these programs echoes basic principles of program management such as can be found in the research literature on organizational management and on management of relationships between organizations and outside constituencies (Blundel, 2004). This section discusses aspects of the practice of public participation that are matters of basic program management and reviews what is known about good management practice, drawing from both the general management literature and from experience with environ- mental public participation. CLARITY OF PURPOSE When responsible agency develops a clear set of objectives for a partici- patory process, integrated with a plan for how the outcomes of the process will be used and with serious efforts to share that understanding with the participants, it increases the likelihood of acceptance of agency decisions and of public willingness to engage in future participation efforts. By do- ing these things, government agencies fulfill widespread expectations that they will play a leading role in setting the agenda for policy discussions and making public purposes clear (Hibbing and Theiss-Moore, 2001). Public participation processes tend to yield better results when the clear purpose reflects an agreement about goals among the convening organization and the participants and when it takes account of the objectives of all parties involved, the scope of legally possible actions, and the constraints on the process. Several lines of evidence support the proposition that clarity of purpose is conducive to success in public participation. This proposition is, first
PUBLIC PARTICIPATION PRACTICE 97 of all, supported by the general literature on organizational management (Blundel, 2004). Clear purposes reduce certain kinds of uncertainty for par- ticipants and thus reduce an impediment to high-quality thinking (e.g., Janis and Mann, 1977; Klein, 1996; Covello et al., 2001; Van den Bos and Lind, 2002). This research also shows, however, that uncertainty makes people more vigilant about evaluating the credibility of information sources (Half- acre, Matherey, and Rosenbaum, 2000; Brashers, 2001; Van den Bos, 2001) and leads their judgments about a process to be influenced more strongly by procedural fairness (Van den Bos, 2001; Van den Bos and Lind, 2002). In our judgment, the uncertainty of information for environmental decisions is almost always sufficient to trigger such effects, even when purposes are clearly stated. A process with clear purposes and procedural fairness is consistent with reducing these uncertainty-related cognitive effects. In the context of environmental public participation, Wondolleck and Ryan (1999) have argued that agencies can engage in public participation processes as leader, partner, or stakeholder, and that when it is not clear which role an agency is playing, the process can suffer. This argument is borne out by empirical work on participation processes. For example, Bradbury (2005) found, in an examination of public participation in the cleanup of multiple Superfund sites, that clarity about agenda setting and prioritization of issues are important factors influencing the perceived com- petence, legitimacy, and capacity of public participation processes. Leachâs (2005) review of 25 empirical studies of public participation in U.S.D.A. Forest Service decisions beginning in 1960 found strong evidence supporting the importance of focused scope and realistic objectives. Some of these studies highlighted the importance of clear purpose, goals, and objectives (Schuett, Selin, and Carr, 2001), along with measurable, quanti- fiable, or tangible goals (Doppelt, Shinn, and John, 2002). Others focused on the importance of defining results in terms of action rather than talk (U.S.D.A. Forest Service, 2000) and of focusing on attainable goals to build momentum, confidence, and reputation (Wondolleck and Yaffee, 1997). Findings highlighted the importance of addressing a manageable number of projects with a reasonable level of complexity (Daniels and Walker, 1997) and the importance of recognizing milestones throughout the process by setting and acknowledging short-term and long-term goals (U.S.D.A. For- est Service, 2000). Clarity in objectives of the process has also been found helpful for keeping decision processes focused on negotiable disputes rather than on discussion of values (Walters et al., 2003). The National Research Council (2007a) analysis of global change assessments, which examined a very different environmental context, concluded that a clear audience for an assessment product is essential to success, which also implies the impor- tance of the processes having clear goals. Conflicts about the scope of public participation efforts have often
98 PUBLIC PARTICIPATION derailed them. For example, the failure of community members and ex officio members to reach agreement about the goals of the Site Specific Advisory Boards of the U.S. Department of Energy (DOE) was cited as one of the factors that led to the inability of four boards to deal with substan- tive issues (Branch and Bradbury, 2007). In the advisory boards of the U.S. Department of Defense (DOD), when community interests did not reach the table, in part due to failure to reach an understanding on the scope of the discussion, this lack of clarity about purpose led many discussions to devolve to procedural issues (Branch and Bradbury, 2006). The ways in which agency decision makers intend to use the output of a process may or may not be clear to participants at the outset. In most cases, public participation is an informal element of the decision-making process, and thus the agency needs to clarify how the results of the participatory process will be incorporated into the decision process. When the participa- tory process is a formal part of the decision process, agency rules sometimes clearly specify the role of the public process in informing or making the decision. However, the role and influence of public participation may not be clearly specified in advance, as when a participation process is initiated at one level of an agency while the ultimate decision is made at another, higher level. Participants who take their charge seriously and who devote considerable effort to reaching a consensus may grow to assume that what the process recommends will be implemented. Explicit, honest agency statements about what it wants from the pro- cess and how the results will be used ensure realistic assessment by the other parties of the reasons for them to participate and reasonable expectations about results. A strong commitment to act on the results of a participa- tory process obviously increases incentives for parties to participate. Lack of clarity about how decision makers intend to use the results encourages skepticism. Because agencies are not monolithic, it is not always easy for participants to gain such clarity. While one unit may state its interest in in- corporating suggestions, another may be less invested in doing so (OâLeary and Summers, 2001; Bradbury, 2005). Public participation processes can be caught in the middle, reducing the usefulness of and trust in them. It is common at the outset of a participatory process for assumptions regarding the nature of the environmental problem being addressed and the possible paths to a solutionâthe âframesâ for the issueâto differ across participants (Snow et al., 1986; Bradbury, 1989; U.S. Environmental Protection Agency, 1992; Kroll-Smith and Couch, 1993; Thompson and Gonzalez, 1997; Pellow, 1999). Yet few external participants will under- stand in any detail the concerns of and constraints on the agency. Thus, developing clarity of purpose involves the emergence of mutual understand- ing of the alternative frames. Differing frames can contribute to undesired results. When individu-
PUBLIC PARTICIPATION PRACTICE 99 als believe that decision processes have not adequately taken into account important values, the results can include a loss of trust, exacerbated con- flict, and prolonged negative affective reactions (e.g., Fisher, 1991; Rich et al., 1995; Shah, Domke, and Wackman, 1996; Susskind and Field, 1996; Baron and Spranca, 1997; Thompson and Gonzalez, 1997). As we discuss in Chapter 7, discordant framing is also a source of conflict. When agencies have sufficient flexibility to allow problems to be reframed through delib- eration so as to incorporate participantsâ definitions, and they allow this to happen, these results may be avoided (National Research Council, 1996; Renn, 2004; Lemos and Morehouse, 2005). Clarity may include explicit recognition that the goals of the process may evolve as it is codesigned with citizens who may have somewhat different goals and expectations from the agencyâs initial ones. A convergence of purposes has the potential to reduce conflict and enable cooperation. However, agency constraints sometimes limit flexibility in this regard. In our judgment, clarity about such real constraints is preferable in the long run to a lack of clarity that allows participants to become engaged in a process they may later conclude was organized under false pretenses. AGENCY COMMITMENT Public participation processes are more likely to be successful when the agency responsible for the relevant environmental decisions is committed to supporting the process and taking seriously the results. This is in part be- cause the more committed a decision-making agency is to act on the results of a public participation process the more likely the parties are to engage seriously. Commitment involves support of both agency leadership and staff at all levels for the objectives of the process, stated at the outset and updated periodically as the participation process and the context evolve. It implies clarifying how and by whom the outputs will be used and a com- mitment to open-minded consideration of those outputs. Basic understanding of group processes and decision making suggests the importance of clear agency commitment. Ambiguity about how infor- mation will be used increases uncertainty, which, as already noted, makes high-quality thinking less likely. The research literature suggests, how- ever, that if the convening agency is committed to a high-quality process, rather than to a particular kind of decision outcome, participants are more likely to engage in evenhanded and effortful consideration of the available options, rather than defensive justification of their preferred alternative (Simonson and Staw, 1992) and arrive at higher quality judgments (Siegel- Jacobs and Yates, 1996; for more detailed reviews, see Lerner and Tetlock, 1999, 2003). This implies that a public participation process is likely to go better if the responsible agency can honestly signal to the participants that it
100 PUBLIC PARTICIPATION has not made a decision and does not have a strong predisposition for one course of action over another but is sincerely looking for input. Studies of environmental public participation reinforce the importance of agency commitment (e.g., Bingham, 1986; Wondolleck and Yaffie, 2000; Schuett, Selin, and Carr, 2001). Leach (2005:8), reviewing several studies of public participation in the U.S.D.A. Forest Service, stated that âsupport from line officers and agency-wide Forest Service Policy is the dominant contextual factor in the reviewed studies.â But such support must be stable. Lubellâs (2004b) analysis of estuary partnerships demonstrated that changes in agency plans at high levels degraded the quality of the participatory planning processes. Changes may be interpreted as signals from the agency that the process is not important in its decision making: this may lead par- ticipants to opt out and seek alternative mechanisms for being heard in the political process, which results in less careful and thorough consideration of the issues. A series of research studies suggests that the DOD advisory boards were less successful than similar ones convened by DOE because DOD failed to convey the importance of public participation (Branch and Bradbury, 2006). In contrast, the relative success of the DOE advisory boards in the late 1990s was attributed to âclarity and commitment to public participation in both policy and implementationâ (Branch and Bradbury, 2006:746). Similarly, a study of alternative dispute resolution by the U.S. Envi- ronmental Protection Agency (EPA) in the 1990s (OâLeary and Summers, 2001) found that despite a stated institutional commitment nationally, implementation varied regionally and with the enthusiasm and skill of staff, rather than on the basis of any consistent method of institutional assess- ment of when alternative dispute resolution would be most valuable. The resistance of mid-level managers was seen as particularly problematic. In addition, EPA often signaled a lack of commitment to the process by send- ing representatives who lacked the authority to make decisions. We have emphasized how too little agency commitment can hamper a public participation process, as when agency officials are not available to provide information about the issues or the decision context or to build relationships with the participants. Too much engagement can also be a problem, as when agency officials so completely dominate the process that participants cannot take an active role in shaping the questions for discus- sion or deciding how the process is organized (e.g., Delli Priscoli, 1983; Stewart, Dennis, and Ely, 1984; Plumlee, Starling, and Kramer, 1985). Whatever an agencyâs level of commitment, it is best for the agency to make clear to the participants how it intends to use the results of the participa- tory process.
PUBLIC PARTICIPATION PRACTICE 101 ADEQUATE CAPACITY AND RESOURCES Public participation processes are more likely to be successful when agencies have adequate capacity and resources and deploy them appro- priately according to the scale, complexity, and difficulty of the issues in- volved. The commitment of resources is both a practical matter and a signal from the agency that the participatory process is important. The difficulty of conducting a process without adequate resources is obvious, and perhaps for this reason it has not been the subject of much empirical research. However, the need for adequate funding and other resources for achieving goals of a participatory process is among the most frequently mentioned lessons from practitionersâ experience (e.g., Creighton, 1999; Leach and Pelkey, 2001; U.S. Environmental Protection Agency, 2001). Lack of resources is always a challenge. Budgetary constraints on agen- cies or decisions within an agency can cause problems in the process, such as âstop and goâ funding or uneven funding among different entities in the process (Moser, 2005). In the Forest Service context, one study (Wondolleck and Yaffee, 1994) pointed to the need for startup costs to be considered; especially when skilled facilitators are needed or public outreach needs to be undertaken. Frentz et al. (2000) recommended that convening organiza- tions consider setting aside dedicated funding so that staff can consistently be present during and participate in the processes. A U.S.D.A. Forest Service (2000) internal study suggested that for longer processes that involve mul- tiple parties, participants should contribute staff and financial resources. Bradbury (2005) found that the problems of two DOE advisory boards were reduced by an infusion of funds to facilitate access to information and provide for neutral facilitation, technical assistance, and support for a sufficient number of meetings. Organizational resources include more than money. Several studies have found that organizational capacity in the form of skilled and enthusi- astic staff is vital to program success (e.g., Henry S. Cole Associates, 1996; OâLeary and Summers, 2001), providing an invaluable reservoir of experi- ence (Henry S. Cole Associates, 1996). An assessment of EPAâs alternative dispute resolution program stated bluntly that the future of the program depended on the agencyâs ability to find trained mediators (OâLeary and Summers, 2001). Other studies have found an association between staff expertise and the extent to which communication activities were two-way and âsymmetricalâ (see Grunig and Grunig, 1992). Organizations that employed âtechniciansâ to develop communication materials were more likely to engage in one-way processes of message transmission in which the organization attempted to control the process. When senior communica- tions managers were part of the organizationâs decision-making structure,
102 PUBLIC PARTICIPATION it was more likely to engage in two-way communication processes that featured listening to outsiders. Continuity of agency personnel has also been found to benefit par- ticipatory processes. The set of empirical studies regarding Forest Service public participation points to the importance of staff continuity (Leach, 2005). A policy in the Forest Service to prevent conflicts of interest among agency personnel by rotating them between forests had the unexpected consequence of placing stress on long-range participation processes (Clarke and McCool, 1985). New personnel faced the challenges of assuring par- ticipants of the agencyâs continued commitment to the process (Wondolleck and Yaffee, 1997; Tuler and Webler, 1999). Similar findings have been reported with public participation at hazardous waste sites (Henry S. Cole Associates, 1996). Limited resources may not be only an external constraint. They may also reflect a lack of agency commitment, as reported in the examination of DOE and DOD public advisory boards by Bradbury (2005). Creating expectations that cannot be met can be a bigger problem than lack of resources per se. To a large extent, public participation processes can be scaled to the resources available. To do so, however, requires careful plan- ning. Some practical planning guides have been developed by environmental agencies that address resource issues in planning and developing participa- tory processes (e.g., U.S. Environmental Protection Agency, 2001). Practical experience suggests that diagnosis and process design efforts can determine the amount of time and resources available for the process and that it is important for the convening organization to make resource constraints clear to the participants, so that a realistic set of objectives for the public involvement process can be set. Being clear about resource constraints can also help an agency allocate resources so as to invest in meeting the most important challenges or obstacles that have been identified. It is often useful to be creative in looking for additional resources, including from participants, the public at large, and the nonprofit sector (U.S.D.A. Forest Service, 2000; Delli Carpini, Cook, and Jacobs, 2004:316). The practitioner literature reinforces this point: agencies are increasingly emphasizing partnerships, with stakeholders and with one another, through which resources as well as perspectives are brought together to accomplish environmental and other public policy objectives in a participatory or col- laborative manner. For example, a watershed management effort in New Jersey was able to include extensive public participation in part due to in-kind contributions and financial resources provided by a variety of or- ganizations, including the local water purveryor, nonprofit watershed orga- nizations, and municipalities (http://www.raritanbasin.org/). A substantial grant from EPA for the larger watershed management effort and limited funding awarded to municipalities by the state department of environmen-
PUBLIC PARTICIPATION PRACTICE 103 tal protection were also critical. (Numerous other case examples can be found at http://cooperativeconservation.gov/stories/index.html.) TIMELINESS IN RELATION TO DECISIONS Public participation processes are more likely to have good results when planned so that they can be informed by emerging analysis and so that their outputs are timely with regard to the decision process. That is, participatory processes need to be designed so that closure is achievable and outcomes are available to decision makers in a timely manner. Timing presents a âGoldilocks problemâ with regard to both scientific analysis and decision processes (National Research Council, 2007a:3-11). If a public participation process is started too soon, key information may not yet be available. If the process is started too late, there may not be adequate time to develop trust and understanding and to process scientific and techni- cal information. Furthermore, if the outputs from the participatory process come too late to influence decisions, it becomes impossible for an agency to fulfill promises to take the process seriously. And yet, if the process does not have sufficient duration, it may not be possible to develop the mutual understanding that underpins a successful participatory process. Most of the literature on environmental public participation emphasizes the importance of starting the process early enough. Including the public as early as possible is one of the most frequently mentioned lessons learned by public participation practitioners in the dispute resolution tradition (e.g., National Park Service Division of Park Planning and Special Studies, 1997; Cestero, 1999; Bleiker and Bleiker, 2000; McKeown, Hopkins, and Chrystalbridge, 2002). Similarly, the Consensus Building Institute (1999:14) stated that âmediation should be used when it is started at an early stage of conflict, before going to public hearings.â This lesson is also often cited by expert groups in the risk analysis tradition (e.g., National Research Council, 1996; Presidential/Congressional Commission on Risk Assessment and Risk Management, 1997a,b), suggesting that in practitionersâ judg- ment, early involvement contributes to both the legitimacy and the quality of decisions. The U.S. Environmental Protection Agency (1992) concluded that many weaknesses in ecological risk assessments emerge at the problem formulation stage, and thus public participation can be especially helpful at exactly the point at which such assessments go awry. Germain, Floyd, and Stehman (2001) found that participantsâ satisfaction was higher when they were involved early in scoping activities. Bradbury, Branch, and Malone (2003) found that the involvement of stakeholders in âscoping and framing of issues during the initial stages of a decision-making processâ was associ- ated with higher quality decisions, and Duram and Brown (1998) found that public participation was perceived to be most helpful in the planning
104 PUBLIC PARTICIPATION stages involving outreach and identifying and prioritizing issues. Peelle et al. (1996) identified early involvement in a long list of factors that influence success. Early involvement may not preclude and can enhance access to and generation of high-quality scientific information and analyses, particularly when time and resources are available for technical representatives of stakeholder groups to be involved in collaborative information generation processes (Susskind, Thomas-Larmer, and Levy, 1999). While early involvement may aid success in many contexts, it appears to be neither necessary nor sufficient. Ashford and Rest (1999), who stud- ied a series of relatively successful public participation processes, reported that this success occurred even though the public became involved fairly late in the process. Mitchell, Clark, and Cash (2006:314) concluded that it is also possible to be too early: âInformation must be . . . timely, coming beforeâbut not too long beforeârelevant decisions get made.â Multicase comparative studies of DOE and DOD advisory boards (Bradbury, 2005) and of the National Assessment of Climate Change (Moser, 2005) exam- ined cases in which the parties appear to have been involved very early. Nevertheless, the DOD and DOE projects had mixed success, and the climate change assessments received mixed evaluations from participants, suggesting that early involvement is not sufficient for success. We suggest that some, though not all, of the benefits that typically ac- crue from early involvement may be a result of having enough duration in the participation process to overcome obstacles raised by the context. For example, in analyzing watershed partnership programs, Lubell and Leach (2005) found that having a long enough duration for a planning process had a significant positive effect on both the scope of the policy agreements achieved and on project implementation and ultimately on watershed con- ditions. Sufficient duration allows mutual understanding and a degree of trust to develop. And without sufficient duration, it is difficult for the participants to develop familiarity with the science or to have input into the scientific analysis (issues to which we return in Chapters 5 and 6). Nor is it easy to allow participants to influence the design of the participatory process if the process starts relatively late and is of short duration. Yet, as discussed below, such collaborative design is important to successful participation. In sum, timing presents significant challenges for the conveners of public participation. Sometimes the best way to meet these challenges is to adjust the intensity of participation and the scope of issues to be covered so that the timetable is realistic. However, rushing the process and compress- ing its scope have downsides: a hurried timetable may not allow enough time for a process appropriate to the challenges of the context, and a re- duced scope will sometimes be seen as having the effect of putting aspects of the decision out of bounds for discussion.
PUBLIC PARTICIPATION PRACTICE 105 A FOCUS ON IMPLEMENTATION Participation processes tend to be more successful when designed to relate in clear ways to policy decision making and implementation. The responsible agencies need to be clear from the outset about what they can and cannot implement, a point already raised. Especially for public participation processes intended to inform decisions, it is important for implementation to be part of the purview. Many case studies have noted the importance of considering implemen- tation issues in defining the scope of a participatory process. For example, Bradbury (2005:13) noted in one decision-making context that being âable to identify and prioritize the issues on which to focus and to prevent issues that were not considered part of their designated scopeâ from being on the agenda was one of the biggest challenges faced by the advisory boards she studied. Mitchell, Clark, and Cash (2006), in their analysis of global assess- ments, concluded that the salience of an analysis to potential users is a key factor in determining how much impact the assessments have. Experienced practitioners often advise that it is useful to identify in advance roles and responsibilities of various groups following the for- mal public involvement process and to be sure to involve those who are needed for the implementation of decisions that result from the participa- tion process. Implementation considerations include possible partnerships for implementation, monitoring and oversight mechanisms, and incentives and disincentives to implementation. Many practictioners believe that an- ticipating difficulties in implementation from all perspectives and discussing contingencies makes public participation processes better informed and increases the chance that they will produce results that participants consider useful. Implementation raises the issue of limitations on an agencyâs scope of authority. Researchers and practitioners often advise that goals match what can be implemented and that the scope of a public participation pro- cess be defined accordingly. For example, Wilbanks (2006), considering the experience of local âsmart growthâ decision processes, advised convening organizations to âdeliver on promises. . . . It is better to indicate a positive intent but to limit oneâs promises than to take a chance that resource limi- tations or political complexities will lead to disillusionment.â But limiting scope can be frustrating to members of the public who have broader inter- ests in the issue. Thus, an early understanding of what an agency can and cannot do will enhance the chances of an effective participatory process. This understanding does not, of course, preclude participants from raising issues outside the agencyâs purview or pursuing those issues in contexts beyond the participatory process.
106 PUBLIC PARTICIPATION COMMITMENT TO LEARNING Public participation processes, as well as the larger assessment and deci- sion processes in which they are embedded, benefit from engaging in self-as- sessment and design correction as they proceed. The design of participatory processes can benefit from opportunities for participants and sponsors to assess the process both as it is under way and at the end. Designs that allow for midcourse adjustments and that are evaluated to generate lessons for future public participation efforts are most conducive to learning. Learning can be greatly advanced by independent evaluations of pub- lic participation efforts. Evaluation studies repeatedly demonstrate that careful research can reveal knowledge that does not emerge from intuitive judgments of what works and what does not (Rowe, Marsh, and Freaer, 2004; Blackstock, Kelly, and Horsey, 2007). Although some systematic ret- rospective studies of public participation now exist, the state of knowledge would be much advanced if organizers of participation supported careful evaluation studies, particularly including prospective studies comparing dif- ferent modes of participation, which can provide evidence about the causes of participation outcomes. Systematic evaluation is the most trustworthy way to gain understanding of the effects of participation practices and thus to ensure institutional learning and improvement in practice. Even when resources are limited, expenditures on systematic evaluation can add a great deal of value (Rohrmann, 1992). However, the scope and resources needed for the evaluation need to be appropriate to the public participation effort. A small, short-term public participation effort may not need as detailed evaluation as a more extensive effort, or one that may become a model for future participatory processes. Evaluation is not merely a report card that agencies get (or give them- selves) at the end of a project. So-called formative evaluation is aimed at improving programs in progress and provides managers with feedback dur- ing program development and implementation (Posavac, 1991). Multiyear assessments of DOEâs Site Specific Advisory Boards (Bradbury, 2005) are an example of an effort to improve the participation process over time and use evaluation data as a basis for making programmatic decisions. Because the boards were a major new agency initiative, it invested in multiyear, qualita- tive, and quantitative evaluation. A more limited effort would merit a more limited formative evaluation, perhaps brief surveys after each meeting or routine debriefings with participants. Other agencies have included formative evaluation as a critical com- ponent of their public participation efforts. For example, before the New Jersey Department of Environmental Protection conducted a trial release of a genetically modified rabies vaccine, it interviewed key opinion leaders to develop participatory processes that met local needs, and it modified its
PUBLIC PARTICIPATION PRACTICE 107 plan to release the vaccine on the basis of this feedback. According to staff, both this participatory process and the formative evaluation that kept it on track accounted for the marked difference between the programmatic suc- cess of the effort and the failure of other states to test rabies vaccines due to public opposition (Chess, 2001). The New Jersey agencyâs formative evalu- ations of the rabies effort were initiated by its public participation staff and involved program staff in implementation. Retrospective studies, even though they cannot employ contemporaneous measures to track change, can also serve to improve future agency programs (Rosener, 1981). Evaluations conducted by professional evaluators external to the con- vening agency can bring objectivity and insight that may not be available from internal evaluations. Several useful efforts have been made to develop evaluation measures for environmental public participation processes (e.g., Lauber and Knuth, 1999; Rowe and Frewer, 2000, 2004). The evaluation research community has also pioneered participatory evaluation, which involves stakeholders in designing the evaluation process (Fetterman, 1994, 1996). In this approach, participants clarify goals and expectations, as well as processes for ascertaining whether these goals are being met. Some evalu- ators feel that those who participate in evaluation design are more likely to use the results (e.g., Guba and Lincoln, 1989; Syme and Sadler, 1994). An example of participatory evaluation is the evaluation of DOEâs Site Specific Advisory Boards, which involved local and headquarters agency staff, as well as participants in the boards, in developing the goals, criteria, and instruments for evaluation (Bradbury and Branch, 1999). Evaluation is only one step in improving agency practice. Learning from the results and institutionalizing them are equally important. Theo- rists suggest that organizational learning goes beyond the learning of indi- viduals, so that agencies develop an institutional memory. According to one often-cited definition, organizational learning is âencoding inferences from history into routines that guide behaviorâ (Levitt and March, 1988:320). This definition implies that organizational learning is reflected in changes in policies, procedures, and systems. Without such institutionalization, learning about public participation may not extend beyond the personnel involved in public participation. This insight is consistent with a history of policy studies research that has long emphasized the need to treat policies as experiments and the value to policy effectiveness of instituting evaluation strategies that deploy both external reviewers and review by the partici- pants and the sponsors (Campbell, 1969). Learning also involves questioning assumptions and operating systems. This can be particularly important when agencies are in the midst of con- troversy. According to one widely accepted model (Argyris, 1982), agencies must learn how to learn. This goes beyond making strategic changes in specific programs, so-called single loop learning. âDouble loopâ learning
108 PUBLIC PARTICIPATION includes questioning the larger systems in which a process is embedded. It may involve changes in those systems, in the rules and methods for decid- ing, or in other organizational routines (Scott, 1992). Thus, an evaluation may yield feedback that can be used to change more than the specific par- ticipatory process being evaluated. Such feedback might lead to internal dialogue about the goals of the overall public participation program, the systems that support it, and the institutional memory needed for ongoing improvement (Chess and Johnson, 2006). In sum, accumulated experience and research on program management support the conclusion that successful practice is more likely to be found in agencies that develop a culture and set of procedures that allow them to learn not only from past experience as organized in the research literature and practitionersâ knowledge, but also from recent and ongoing experience in their own agencies and in other organizations convening public participa- tion in similar contexts. The notion of ongoing learning from participatory processes is congruent with the idea of integrated, repeated analysis and deliberation endorsed in Understanding Risk (National Research Council, 1996). Advice from public participation practitioners is also consistent with this view. Zarger (2003), reviewing a set of practitioner handbooks, listed âcommitment to iterative, resilient, responsive processes and moni- toringâ as one of the 10 most frequently mentioned lessons learned, citing Creighton (1999), Leach and Pelkey (2001), U.S. Environmental Protection Agency (2001), National Environmental Justice Advisory Council (1996), and McKeown, Hopkins, and Chrystalbridge (2002), and the work of Pierce Colfer (2005) as sources for the lesson. Another of the top 10 les- sons learned from watershed partnerships was openness and flexibility to respond and to change course if necessary to get to the end goal (Leach and Pelkey, 2001). Both of these lessons from practice speak to the importance to success of a commitment to learning in the responsible agencies. Learning is important not only in the agencies, but also among the parts of the public who have limited experience with participation and limited resources to devote to participation. As we have noted, many scholars have concluded (Bowles and Gintis, 1986; Delli Carpini, Cook, and Jacobs, 2004:322) that the apathy and alienation found in much of the public is substantially a consequence of limited opportunities for meaning- ful participation, so that, over the long term, increased public involvement depends on learning and enhanced capabilities among the public as well as in agencies. Leach (2005) found the Forest Serviceâs use of adaptive man- agement a sound example of an approach to policy that built capacity for participation.
PUBLIC PARTICIPATION PRACTICE 109 CONCLUSION The evidence indicates that public participation processes have better results when they follow basic principles of program management: clar- ity of purpose, commitment, adequate resources, appropriate timing, an implementation focus, and a commitment to learning. However, it is not always easy to follow these principles. Difficulties can arise from a variety of factors, including internal differences of purpose within the responsible agency, shortages of money or skilled personnel, timing of participation, and various other contestable factors outside the agency. When such dif- ficulties exist, success depends on how well the process is organized to avoid or overcome the problems they present. Chapters 7 and 8 discuss this issue.