Click for next page ( 238


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 237
Appendix F Methods to Support Decision Making We highlight here four approaches to formalizing decision making and the rationalization of decisions that have been useful in complex water resources planning. These are robust decision making, collaborative model- ing for decision support, decision scaling, and joint fact finding. We present these as examples of formal frameworks for tracking and understanding decisions in complex situations. We recommend that these, and others, be evaluated and some version (or perhaps a hybrid) be adopted for the delta. These approaches are frameworks that include a transparent procedure with a series of structured linkages and steps. Some of these steps include the use of statistical and numerical models, some of which already exist for the delta and others that would need to be developed should one of these approaches be adopted. ROBUST DECISION MAKING Robust decision making (RDM) (Lempert et al. 2003, 2006, Groves and Lempert 2007) is a quantitative, scenario-based method for identifying policies (or strategies) that are relatively insensitive to poorly understood uncertainty. Instead of developing a single and potentially contested proba- bilistic forecast and associated optimal solution, RDM evaluates candi- date solutions against large ensembles of possible outcomes to illuminate critical vulnerabilities and suggest approaches for increasing the strategies' robustness. RDM has been applied to problems related to climate change mitigation or adaptation in a variety of different contexts, including global sustainability (Lempert et al. 2004, 2006) and long-term water planning 237

OCR for page 237
238 SUSTAINABLE WATER MANAGEMENT IN THE DELTA (Groves and Lempert 2007, Groves et al. 2008a, 2008b, 2008c, 2008d). It has been a useful framework for developing robust climate adaptation strategies for water agencies. Key challenges to deploying RDM include retooling existing models to be evaluated many more times than is typical, deploying new and often unfamiliar statistical approaches for identifying vulnerabilities, and ensuring that decision makers and stakeholders under- stand the new approach. RDM proceeds through a series of steps that can be customized de- pending on the application. In the first step, analysts, often in conjunction with stakeholders and decision makers, specify the key uncertain exogenous factors (X) that are likely to be disputed by different parties to the decision, draw up a list of policy levers (L) that comprise strategies, identify measures (M) to consider when evaluating policy outcomes, and identify models and/ or relationships (R) that relate the uncertainties and strategies to outcomes. The resulting information, termed an "XLRM" chart, is used to assemble the quantitative models to be used to evaluate the performance of strategies under many alternative scenarios. The resulting analysis is not used to identify a single "optimal" strategy. Instead, one or a few strategies are identified for a structured evaluation of their performance against a wide array of plausible scenarios (steps 2 and 3). In the fourth step, statistical tools are used to identify the key vul- nerabilities, or sets of assumptions that lead the proposed strategy to fail. These vulnerabilities thus represent future conditions (or scenarios) that are critically important to the choice of strategies--they are the conditions that might lead the promising strategy to perform poorly. Under these conditions, alternative strategies would be preferred. The trade-offs among alternatives under these vulnerable conditions can be helpful in identifying new hedging options that can then be used to develop more robust strate- gies. These more robust strategies are then evaluated as before. Through iteration, RDM helps the analyst explore across a broad range of possible strategies without requiring the contentious specification of uncertain future parameters. The strategies identified become more robust, thus reducing the sensitivity of the strategy's performance to the key uncertainties. In contrast to probabilistic assessments, which typically provide rank- ings of strategies based on a set of underlying assumptions about climate change, RDM identifies the key uncertainties relevant to the choice of strat- egy and then provides trade-off curves that enable decision makers to assess the implications of different expectations of the key uncertainties to their choices. This information has been compelling to stakeholders and decision makers when evaluating climate change impacts on water-management systems (Groves et al. 2008c).

OCR for page 237
APPENDIX F 239 COLLABORATIVE MODELING FOR DECISION SUPPORT To evaluate alternative delta scenarios it would be helpful to have a multifaceted analysis that could address scarcity economics, water market prices, energy utilization, and alternatives for adaptive management. Collab- orative modeling for decision support (abbreviated CMDS or COMODES) is the "generic" (Cardwell 2011) name given to a suite of techniques that can be used to achieve consensus on complex, contentious issues. Indeed, Lorie (2010) defined CMDS as "integrating collaborative modeling with participatory processes to inform natural resource management decisions." CMDS is an approach to reach consensus and make decisions about com- plex systems that combines technical skills required to understand the systems scientifically and stakeholder involvement (Cockerill et al. 2006, Langsdale et al. 2011). With respect to stakeholder involvement, process skills such as an appreciation of institutional setting and ability to engage stakeholders and build their trust are essential (Langsdale et al. 2011). Various "brand names" of CDMS are Shared Vision Planning (SVP), the brand of CMDS used by the Institute for Water Resources (IWR) of the U.S. Army Corps of Engineers (Cardwell et al. 2008); Computer-Assisted Dispute Resolution or CADRe (Stephenson et al. 2007); or mediated mod- eling (van den Belt 2004). Although CMDS has been practiced in one form or anther since the late 1980s (Langsdale et al. 2011), only recently has there been specification of guiding principles and best practices. Langsdale et al. (2011) listed eight guiding principles: 1.Collaborative modeling is appropriate for complex, conflict-laden decision making processes where stakeholders are willing to work together. 2.All stakeholder representatives participate early and often to ensure that all their relevant interests are included. 3.Both the analysis and the process remain accessible and transparent to all participants. 4.Collaborative modeling builds trust and respect among parties. 5.The analysis supports the decision process by easily accommodating new information and quickly simulating alternatives. 6. The analysis addresses questions that are important to decision mak- ers and stakeholders. 7. Parties share interests and clarify the facts before negotiating alternatives. 8.Collaborative modeling requires both modeling and facilitation skills.

OCR for page 237
240 SUSTAINABLE WATER MANAGEMENT IN THE DELTA One aspect of CMDS that can perplex sophisticated modelers is the premise that stakeholders, many of whom have little or no experience with either the development or application of simulation models, will be active participants in the modeling process. For a system as complex as the delta, this may seem to be an impossible situation. Langsdale et al. (2011) offer some guidance on this. Often, system dynamics (SD) modeling techniques are applied to con- duct collaborative modeling studies because they allow participants to examine complex physical systems that involve social and economic factors involved (Cockerill et al. 2006). This section would be incomplete without addressing the prospects for consensus that collaborative modeling seeks to achieve. Madani and Lund (2012) have traced changes in the delta conflict in the context of game theory and suggest that the conflict has evolved with time from cooperation to "chicken." In the early 20th century, stakeholders agreed to cooperative solutions; later on, fights over water allocations led to stakeholders com- peting as opposed to cooperating (Madani and Lund 2012). They do state that a win-win resolution may be possible but that a cooperative solution is unlikely without external influence. Indeed, they conclude their paper with the following: Including the state of California (or federal government) did not funda- mentally alter the game. For the cases examined, the Chicken charac- teristics remained and cooperation was unlikely. Adding the state to the game suggested that California can be the victim and loser in the conflict, bearing much of the cost of a Delta failure, due to its past failure so far to develop reliable mechanisms to enforce cooperation. Whatever plan is adopted to fix the Delta in the coming decades, the Delta's sustainability is not guaranteed without powerful mechanisms which provide incentives for cooperation or penalties for deviation from cooperation. While recent efforts address symptoms of the problem, they have not yet solved a main cause - lack of effective and responsive govern- ing mechanisms. California must "govern" the Delta or pay for absence of effective governance. The prospect for achieving consensus, whether by collaborative model- ing or some other means, is a daunting task. Collaborative Modeling in the Delta Two episodes in the recent history of delta management illustrate the value of collaborative modeling. The first of these was the development of flow standards by the U.S. Environmental Protection Agency (EPA) in the period 1992-1994, which began with the 1992 EPA workshops (Schubel

OCR for page 237
APPENDIX F 241 et al. 1993, Kimmerer and Schubel 1994). The key step in translating the conclusions of this workshop into a workable standard for flow, in this case based on the position of X2, was modeling used to understand the water- supply recommendations of the standard. This was done through a collabo- ration between a regulatory agency (the EPA) and by engineers from the Contra Costa Water District acting on behalf of the California Urban Water Agencies (CUWA), an organization of stakeholders who would have been affected by the regulation (R. Denton, personal communication, 2012). In the end, the EPA X2 regulations as modified by CUWA were adopted as the 1994 Bay-Delta Accord, an agreement that helped lead to the establishment of CALFED (Rieke 1996, Hanemann and Dyckman 2009). The second episode was the gaming carried out to design the Envi- ronmental Water Account (EWA). In this case, a group of regulators, con- sultants, and representatives of water agencies and environmental groups explored the water-supply implications of different-size EWAs using the wa- ter resources system model CALSIM (Brown et al. 2004, Booher and Innes 2010). Using historical salvage data, this gaming was used to developing strategies for deploying EWA assets in order to have maximal effect (Brown et al. 2009). Importantly, as Connick and Innes (2003) write, it [the EWA] would not have been even imaginable without the trust and co-operation of the stakeholders. Moreover the details could not have been worked out without this social capital. Agency personnel and stakehold- ers from agricultural and urban water interests and environmental groups spent hundreds of hours working through various scenarios to test how the approach could be used before recommending that it be part of the CALFED programme. Thus, the aims of these collaborative efforts were in some ways modest; that is, the outcomes of the modeling they used were relatively straightfor- ward, being focused on water operations and their effect on the physical environment. Nonetheless, the committee views them as examples worth emulating in future efforts to manage the delta ecosystem. Collaborative Modeling in Everglades Restoration In 1993, the U.S. Army Corps of Engineers (USACE), in partnership with the South Florida Water Management District (SFWMD) and other stakeholders, initiated the Comprehensive Review Study of the Central and Southern Florida (C&SF) Project. This study, commonly called "the Restudy," was intended to integrate solutions which when implemented will enhance the ecological values of the Florida Everglades by increasing the total spatial extent of natural areas, improving the habitat and functional quality, plant and animal species abundance, and diversity. Another objec-

OCR for page 237
242 SUSTAINABLE WATER MANAGEMENT IN THE DELTA tive was the enhancement of economic values and social well-being through increase of the availability of freshwater for agricultural, municipal, and industrial users, reduction of flood damages, provision of recreational and navigational opportunities, and protection of cultural and archeological resources and values. The Restudy followed a transparent, multiagency, participatory, and highly iterative process with a strong collaborative modeling component for the development of the Comprehensive Everglades Restoration Plan (CERP). The core Restudy team of analysts consisted of multidisciplinary professionals from numerous federal, state, local, and tribal organiza- tions, and subteams for modeling, alternatives design, alternative analysis, and public involvement. The Restudy's success in meeting deadlines and consensus building required the use of a large team consisting of over 150 individuals from 30 different public entities representing over 20 different professional disciplines. The modeling team relied heavily on the use of several hydrologic, ecological, and water-quality simulation models and expert judgment. Plan formulation began by developing a list of many different ideas to achieve goals and objectives. The ideas, called "components," were the individual building blocks that were combined in various ways to form al- ternative plans that included both structural and nonstructural features. In each iteration, alternative plans were formulated by the Alternative Design Team (ADT) and modeled by the Modeling Team. The designs of the alter- native plans were built into the South Florida Water Management Model (SFWMM), a regional-scale hydrologic model, for performance evaluation and to provide input to other models in the toolbox. The modeling output was used to produce a large suite of performance measures that had been developed from conceptual models of the major landscapes and water- supply planning efforts. Each alternative plan was evaluated by another multiagency team called the Alternative Evaluation Team (AET), which incorporated comments from different agencies and the public, together with their own evaluation to make recommendations to the ADT for the next iteration. The AET was responsible for evaluating each plan's strengths and weaknesses, and describing plan shortfalls to the ADT. This repetitive formulation and evaluation process progressively refined and improved the performance of subsequent alternative plans. Because of the large and geographically dispersed number of people involved and interested in the Restudy, the Internet was used to communicate formulation and evaluation results. This allowed the Restudy team to solicit comments from a broad base of the public and permitted people to participate as team decisions were being made. The collaborative modeling effort continues today through a newly cre- ated Interagency Modeling Center (IMC) with key leadership of sponsoring

OCR for page 237
APPENDIX F 243 agencies and participation by others. It is a single point of service for the modeling needs of CERP projects and programs and provides coordination, review of other modeling efforts. Through interagency collaboration IMC acts as a clearinghouse for all project-specific modeling and conducts its own regional-scale analysis.1 DECISION SCALING Brown et al. (2011) have recently described an alternative approach to decision making under climate change that may be applicable to the delta. Rather than begin with climate change predictions and their asso- ciated uncertainty downscaled to the problem at hand, the concept is to turn traditional decision analysis around and start by identifying which uncertainties are important from the viewpoint of the decision maker. In the case of climate change, the framework facilitates the identification of climate information that is critical to the planning decision. As a result, decision analysis provides an analytic framework that can be exploited to link bottom-up climate vulnerability analysis with the generation of climate change projections. The process is entitled "decision scaling." The key tenet of the approach is that the appropriate orientation for adaptation planning is one of acceptance of large uncertainties and plan- ning for a wide variety of possible futures. This runs contrary to the general scientific orientation of focusing on the reduction of uncertainty and then planning for the accepted expert characterization of the future. Instead, the approach emphasizes robustness over a wide range of climate futures. It has been applied to the development of a regulation plan for the Upper Great Lakes (Brown et al. 2011). The regulation plan utilizes dynamic responses to evolving conditions and adaptive management of uncertainties and sur- prise. However, Brown et al. present a general process for water resources planning under climate change (or any other uncertainty for which a va- riety of predictions are possible) based on a decision-analytic approach to identifying and tailoring the necessary information. The framework links insight from bottom-up analysis, including performance metrics defined by stakeholders with the processing of, in the Great Lakes example, climate change projections to produce decision-critical information. A key aspect of decision scaling is that the specification of the climate states, that is, the specific climate information that causes a particular decision to be favored over another (or an impact to be large enough to warrant preventative actions, i.e., the identification of thresholds), may al- low the credibility of climate information derived from GCM projections (or other sources) to be improved. That is, with the information from the 1 See www.evergladesplan.org. Accessed July 17, 2012.

OCR for page 237
244 SUSTAINABLE WATER MANAGEMENT IN THE DELTA bottom-up, decision-analytic framework in hand, the generation of climate information may be tailored to best provide credible information through the selection of process models, temporal and spatial scales, and scaling techniques given the time. The approach begins with stakeholders rather than predictive system models. Planners ask stakeholders and resource experts what conditions they could cope with and which would require substantial policy or in- vestment shifts. This is then formalized within a framework that links the multiple models needed to relate changes in the physical climate conditions to performance metrics of interest to stakeholders. After these are estab- lished, hydrologists and climate scientists estimate the plausibility of the water conditions that exceed the coping thresholds, taking into account not only climate change but also natural climate variability and stochastic variability observed with a stationary climate assumption. While the exist- ing applications of decision scaling focus on uncertainties associated with climate change, the approach could be adjusted to consider other uncer- tainties that are key in the bay delta including consumption patterns and environmental factors. Joint Fact-Finding in Bay-Delta Science The products of the delta science process involve at least three science efforts: one carried out by wildlife agencies, one by water users, and a third effort by professional environmental organizations. They each involve many scientists, including agency staff, academics, consulting firms, and individual experts with national reputations, and are carried out largely separately. There are fundamental disagreements. Each attempts to present the objective truth on a variety of issues. While there are several forms for collaboration, there does not seem to be a format for resolving professional scientific differences of opinion. A process called "joint fact-finding" may be of value. Ehrmann and Stinson's seminal chapter in the Consensus Building Handbook: A Com- prehensive Guide to Reaching Agreement, describes the process as follows: "Joint fact-finding" offers an alternative to the process of "adversary sci- ence" [what has been, perhaps inappropriately, termed, "combat science" in this estuary] when important technical or science-intensive issues are at stake. Joint fact-finding is a central component of many consensus build- ing processes; it extends the interest-based, cooperative efforts of parties engaged in consensus building into the realm of information gathering and scientific analysis. In joint fact-finding, stakeholders with differing viewpoints and interests work together to develop data and information, analyze facts and forecasts, develop common assumptions and informed

OCR for page 237
APPENDIX F 245 opinion and, finally, use the information they have developed to reach decisions together. Several references describe the important features of joint fact-finding (see Ehrmann and Stinson 1999, Karl et al. 20072), which can be sum- marized as participation by all parties with interest and scientific contributions to make; use of a neutral, expert facilitator to manage the process; identification of key scientific questions to be addressed by the process; development of an agreed-on process for answering the questions; and carrying out that process and jointly evaluating the results. Although this process might rely in part on outside, independent ex- perts, it primarily involves the disputing experts, those with the most at stake, those whose ultimate buy-in is necessary to resolve or narrow scien- tific disputes. It certainly is true that without joint fact-finding, long-held positions can change. As Kuhn (1970/1996) observed, "Sometimes the convincing force is just time itself and the human toll it takes," or as Kuhn quoted Max Planck, "a new scientific truth does not triumph by convincing its opponents and making them see the light, but rather because its oppo- nents eventually die, and a new generation grows up that is familiar with it." One purpose of joint fact-finding is to speed this process and make its outcomes relevant to decisions that will be made soon. Where "joint fact-finding" can run awry is the premise sometimes put forward by the advocates of that concept that once the facts are on the table then the scientists can "resolve the issue." It is very important that the goal of the "jointly evaluating results" segment is clear. Clear "resolution" of a complex problem is rarely how science works (see Chapter 5). For example, ranking stressors certainly has many policy benefits, but it is a simplification that, if resolved by a joint fact-finding panel, would be turned over by the next panel, ad infinitum. The benefit of properly focused joint fact-finding is broad involvement of many parties in the scientific dialogue. Adversary science can be minimized for the purposes of dialogue, if the immediate discussion of a workshop, for example, is constrained to defining the state of the science, defining where disagreements exist and what they are, and 2 For additional information on joint fact-finding, see http://ocw.mit.edu/courses/urban- studies-and-planning/11-941-use-of-joint-fact-finding-in-science-intensive-policy-disputes- part-i-fall-2003/readings/.

OCR for page 237
246 SUSTAINABLE WATER MANAGEMENT IN THE DELTA deciding on the path forward and/or what the policy choices are. If the dialogue is allowed to turn into an argument about who is right and who is wrong, constructive progress is lost; this is where the court cases have taken California to today. A dialogue in which public events are focused on constructive progress, if given time and supported by the policy com- munity, (a) helps develop at least some commonalities in views of the state of the science among adversaries; (b) points out where new work is needed as agreed on by all parties; (c) can smooth the waters of conflict by provid- ing a nonadversarial forum in which people from different sides can find at least some subjects on which they can agree; and (d) improves public trust if the dialogue takes place in public forums. The Science Program of the Delta Stewardship Council has a history of attempting to build such a dia- logue. In a speech in 2002 Secretary of Resources Mary Nichols suggested this approach was gaining traction with policy makers. It appears that Madani and Lund's game of "chicken" reasserted itself after CALFED was declared a failure in 2004. But there is still an undercurrent of constructive scientific dialogue taking place, sponsored by the Science Program of the Delta Stewardship Council, from which there are opportunities to build if given support. A return to an enthusiastic joint, constructive scientific dialogue, per- haps mediated by independent experts, might possibly be an ingredient that could help bridge what is now an ever-widening gap between key interest groups. Seeking points of agreement among adversaries, even if only over the science, would be a step toward consensus about at least some aspects of important science-driven policy issues and their uncertainties. This small, easily implemented change could begin to improve public trust, placing decision making on firmer ground. It is a process that can provide a more timely result than that which might occur by waiting for the professional demise of leading proponents of the opposing viewpoints. It is a process whose application to the delta science process is long overdue. REFERENCES Booher, D. E., and J. E. Innes. 2010. Governance for resilience: CALFED as a complex adap- tive network for resource management. Ecology and Society 15(3):35. Available at http:// www.ecologyandsociety.org/vol15/iss3/art35/. Accessed June 4, 2012. Brown, C., W. Werick, W. Leger, and D. Fay. 2011. A decision-analytic approach to manag- ing climate risks: Application to the upper great lakes. Journal of the American Water Resources Association 47:524-534. Brown, L., W. Kimmerer, and R. Brown. 2009. Managing water to protect fish: A review of California's environmental water account, 2001-2005. Environmental Management 43:357-368, doi:10.1007/s00267-008-9213-4.

OCR for page 237
APPENDIX F 247 Brown, R., T. Cannon, D. Fullerton, and B. Herbold. 2004. California (CALFED) daily en- vironmental management modeling for fish protection and water supply evaluation. In Building Partnerships: Proceedings of Joint Conference on Water Resource Engineering and Water Resources Planning and Management 2000, July 30-August 2, 2000, Min- neapolis, MN, edited by R. H. Hotchkiss and M. Glade. ASCE Publications. Cardwell, H., 2011. Introduction. In Converging Waters: Integrating Collaborative Modeling with Participatory Processes to Make Water Resources Decisions, edited by L. Bourget. Alexandria, VA: Institute for Water Resources, U.S. Amry Corps of Engineers, 218 pp. Cardwell, H., S. Langsdale, and K. Stephenson. 2008. The Shared Vision Planning Primer: How to Incorporate Computer Aided Dispute Resolution in Water Resources Planning. Alexandria, VA: Institute for Water Resources (IWR) Report No. 08-R--2. www.iwr. usace.army.mil/docs/iwrreports/2008-R-02.pdf. Accessed July 20, 2011. Cockerill, K., H. Passell, and V. Tidwell. 2006. Cooperative modeling: Building bridges between science and the public. Journal of the American Water Resources Association 42:457-471. Connick, S., and J. E. Innes. 2003. Outcomes of collaborative water policy making: Applying complexity thinking to evaluation. Journal of Environmental Planning and Management 46(2):177-197. Ehrmann, J. R., and B. L. Stinson. 1999. Joint fact-finding and the use of technical experts. In The Consensus Building Handbook: A Comprehensive Guide to Reaching Agreement. Cambridge, MA: The Consensus Building Institute. Groves, D. G., M. Davis, R. Wilkinson, and R. Lempert. 2008a. Planning for climate change in the inland empire: Southern California. Water Resources IMPACT, July. Groves, D. G., D. Knopman, R. Lempert, S. Berry, and L. Wainfan. 2008b. Presenting Uncer- tainty About Climate Change to Water Resource Managers--Summary of Workshops with the Inland Empire Utilities Agency. TR-505. Santa Monica, CA: RAND. Groves, D.G., and R.J. Lempert. 2007. A new analytic method for finding policy-relevant scenarios. Global Environmental Change 17:73-85. Groves, D. G., R. Lempert, D. Knopman, and S. Berry. 2008c. Preparing for an Uncertain Future Climate in the Inland Empire--Identifying Robust Water Management Strategies. DB-550. Santa Monica, CA: RAND. Groves, D. G., D. Yates, and C. Tebaldi. 2008d. Developing and applying uncertain global climate change projections for regional water management planning. Water Resources Research 44:W12413, doi:10.1029/2008WR006964. Hanemann, M., and C. Dyckman. 2009. The San Francisco Bay-Delta: A failure of decision- making capacity. Environmental Science and Policy 12:710-725. Karl, H. A., L. E. Susskind, and K. H. Wallace. 2007. A dialogue not a diatribe. Effective integration of science and policy through joint fact finding. Environment 49(1):20-34. Kimmerer, W. J., and J. R. Schubel. 1994. Managing freshwater flows into San Francisco Bay using a salinity standard: Results of a workshop. Pp. 411-416 in Changes in Fluxes in Estuaries, edited by K. R. Dyer and R. J. Orth. Fredensborg, Denmark: Olsen and Olsen. Kuhn, T. (1970/1996). The Structure of Scientific Revolutions, 3rd Edition. Chicago: Univer- sity of Chicago Press. Langsdale, S., A. Beall, L. Bourget, E. Hagen, S. Kudlas, R. Palmer, D. Tate, and B. Werrick. 2011. Collaborative Modeling for Decision Support in Water Resources: Principles and Best Practices. Available at http://www.computeraideddisputeresolution.us/bestpractices. Accessed 21 July 2011. Lempert, R.J., D.G. Groves, S.W. Popper, and S.C. Bankes. 2006. A general, analytic method for generating robust strategies and narrative scenarios. Management Sciences 52(4): 514-528.

OCR for page 237
248 SUSTAINABLE WATER MANAGEMENT IN THE DELTA Lempert, R., N. Nakicenovic, D. Sarewitz, and M. Schlesinger. 2004. Characterizing climate- change uncertainties for decision-makers. Climatic Change 65:1-9. Lempert, R. J., S.W. Popper, and S. C. Bankes. 2003. Shaping the Next One Hundred Years: New Methods for Quantitative, Long-Term Policy Analysis. Santa Monica, CA: RAND. Lorie, M. 2010. Computer-Aided Dispute Resolution 2nd Workshop: Summary and Strategic Plan. Alexandria, VA: Institute for Water Resources (IWR) Report No. 2010-R-5. Madani, K., and J. R. Lund. 2012. California's Sacramento-San Joaquin Delta conflict: From cooperation to chicken. Journal of Water Resources Planning and Management 138 (2): 90-99, doi:10.1061/(ASCE)WR.1943-5452.0000164. Rieke, E. A. 1996. The Bay-Delta Accord: A stride toward sustainability. Colorado Law Review 67:341-69.S. Schubel, J. R., et al. 1993. Managing Freshwater Discharge to the San Francisco Bay/ SacramentoSan Joaquin Delta Estuary: The Scientific Basis for an Estuarine Standard. San Francisco Estuary Project, U.S. Environmental Protection Agency, San Francisco, CA, 109 pp. Stephenson, K., L. Shabman, S. Langsdale, and H. Cardwell. 2007. Computer Aided Dispute Resolution: Proceedings from the CADRe Workshop. Alexandria, VA: Institute for Water Resources, IWR Report No. 07-R6. Available at www.sharedvisionplanning.us/docs/SVP ---2007---R- --06.pdf. Accessed July 22, 2011. Van den Belt, M., 2004. Mediated Modeling: A System Dynamics Approach to Environmental Consensus Building. Covelo, CA: Island Press, 296 pp.