National Academies Press: OpenBook
« Previous: REFERENCES
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 24
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 25
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 26
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 27
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 28
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 29
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 30
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 31
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 32
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 33
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 34
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 35
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 36
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 37
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 38
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 39
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 40
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 41
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 42
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 43
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 44
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 45
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 46
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 47
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 48
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 49
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 50
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 51
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 52
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 53
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 54
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 55
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 56
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 57
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 58
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 59
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 60
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 61
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 62
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 63
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 64
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 65
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 66
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 67
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 68
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 69
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 70
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 71
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 72
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 73
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 74
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 75
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 76
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 77
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 78
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 79
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 80
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 81
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 82
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 83
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 84
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 85
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 86
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 87
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 88
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 89
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 90
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 91
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 92
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 93
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 94
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 95
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 96
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 97
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 98
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 99
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 100
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 101
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 102
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 103
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 104
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 105
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 106
Suggested Citation:"APPENDIX A: Participant Case Summaries." National Research Council. 2006. Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop. Washington, DC: The National Academies Press. doi: 10.17226/11652.
×
Page 107

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

APPENDIX A Workshop on Linking Knowledge with action for Sustainable Development Hosted by the U.S. National Academies Roundtable on Science and Technology for Sustainability Participant Case Summaries7 To begin understanding the diversity of cases that we would explore during the workshop, we requested that participants briefly answer the following questions about their cases. These case summaries were distributed to all participants prior to the workshop. Previous workshops held by the task force suggested that successful programs linking knowledge with action are agents of change and innovation. However, established interests and organizations generally seek to oppose or co-opt such programs. In fact, it is a wonder that any succeed at all. We list below some characteristics of successful programs that have emerged from our previous workshops, together with questions about these characteristics that we asked participants in this workshop to reflect upon in the context of their own program experience. Included below is the set of questions posed to workshop participants and the answers that we received. 0. Short descriptive title of program Question: What is a short, descriptive title for the program you are presenting? 1. Problem definition Tentative finding: Successful programs linking knowledge with action require dialogue and cooperation between the scientists who produce knowledge and the decision makers who use it. Especially important is that the problem to be solved be defined in a collaborative but ultimately user- driven manner. Question: What is the problem to be solved by your program? How—if at all—did the program provide for a user-driven dialogue between scientists and decision makers to shape problem 7 The case summaries submitted for the workshop are included as an appendix because they: provide valuable information about the programs represented at the workshop and how they contribute to sustainability; offer specific examples of and lessons from program managers’ efforts to link knowledge with action; and include resources for additional information, such as program URLs and program managers’ contact information. The case summaries may provide the reader with a more thorough and nuanced understanding of some of the key points made at the workshop. Note: Participants’ case summary responses are included in the appendix as submitted to the National Academies, without substantive editing. They represent the perspectives of the individual authors, and not necessarily those of the National Academies or the organizations that employ them. 24

APPENDIXES 25 definition? How—if at all—did the ultimate problem definition differ from initial formulation by scientists and decision makers, respectively? 2. Program management Tentative finding: Successful efforts to develop programs linking knowledge with action generally adopt a project orientation and organization, with dynamic leaders accountable for achieving use- driven goals and targets. They avoid the pitfall of letting study of the problem displace creation of solutions as the program goal. Question: Was your program developed in such a project mode? Did it have specific, measurable goals and targets? If so, what? To what extent and in what ways was goal and target definition driven by scientists or decision makers, or both? To what extent and in what ways were program leaders held accountable for achieving those goals and targets? 3. Program organization Tentative finding: Successful programs linking knowledge with action include boundary organizations committed to building bridges between the research community on the one hand, and the user community on the other. These boundary organizations often construct informal and sometimes even partially hidden spaces in which project managers can foster user-producer dialogues, joint product definition, and end-to-end system building free from distorting dominance by groups committed to the status quo. In order to maintain balance, most effective boundary organizations make themselves jointly accountable to both the science and user communities. Question: Did your program involve a boundary spanning function or organization? If not, how did you organize the dialogue between producers and users? If so, where and how was the boundary organization or function created? What did it do? To what extent was it accountable to both users and producers for achieving its goals? 4. The decision-support system Tentative finding: Successful programs linking knowledge with action create end-to-end, integrated systems that connect basic scientific predictions or observations to decision-relevant impacts and options. They avoid the pitfall of assuming that a single piece of the chain (e.g., a climate prediction) can be useful on its own, or will be taken care of by someone else. Question: To what extent is the decision-support system developed by your program an end-to-end system? What are its discrete elements (e.g., a climate forecast, an impact model converting climate forecasts into yield forecasts required by decision makers)? Which were the hardest elements to put in place? Why? What changes in research, decision making, or both have occurred as a result of the system?

26 LINKING KNOWLEDGE WITH ACTION 5. Learning orientation Tentative finding: Successful programs linking knowledge with action are designed as systems for learning rather than systems for knowing. Recognizing the difficulty of the task, such programs are frankly experimental, and expect and embrace failure in order to learn from it as quickly as possible. Success requires appropriate reward and incentive systems for risk-taking managers, funding mechanisms that enable such risk taking, and periodic external evaluation. Question: Did your program have an expressly experimental orientation? How did it identify which risks to take? How did it identify success and failure? How did it engage outside evaluators to help it reflect on its own experience? What are the most important lessons you have learned regarding pitfalls to be avoided, or approaches to be followed in the future? 6. Continuity and flexibility Tentative finding: Successful programs linking knowledge with action must develop strategies to maintain program continuity and flexibility in the face of budgetary and human resource challenges, such as the dual public-private character of knowledge-action systems; budgetary pressure to highlight short-term, measurable results; uncertainty regarding future budgetary priorities in a dynamic political environment; and shortages of people who can work effectively across disciplines, issue areas, and the knowledge-action interface. Question: How do budgetary requirements and/or human resource pressures influence your program? What, if any, collaborative funding mechanisms have you developed to ensure continuity and relevance to user needs? If applicable, how do you maintain public funding, or incorporate private funding, for the provision of a partially private good? What, if any, innovative approaches have you developed for enhancing human capacity in your program area (e.g., building curricula or providing incentives to reward interdisciplinary activities)? 7. Other insights? Question: What other insights or conclusions emerge from your experience about the factors responsible for success and failure in activities designed to link knowledge with action? 8. Other issues? Question: Are there any other issues that you would like to discuss during the workshop? 9. Contact information Question: Could you please list for the case presented the key contact person (presumably but not necessarily yourself), with title and contact information?

APPENDIXES 27 10. Representative publications/products Question: Could you please list a couple of key publications or products that would help us to understand the program you have described, including websites? (If possible, please append electronic copies or links). Participants’ Answers THEME I: AIR QUALITY AND CLIMATE International Research Institute for Climate Prediction (IRI) Jim Buizer Arizona State University 1. Problem Definition To provide usable seasonal-to-interannual climate forecast information to resource managers and policy-makers worldwide, particularly to those living in regions impacted by the El Nino- Southern Oscillation (ENSO) phenomenon. Throughout the entire design and implementation process, input from stakeholders was sought via user-producer workshops. Whereas initially the product was primarily the construct of the physical scientists at the IRI, it has evolved substantially from its original formulation, heavily influenced by ongoing stakeholder input. 2. Program Management The IRI was designed with the explicit goal of providing usable climate information to those making resource decisions. One of the biggest challenges was, and continues to be how to measure success. For example, is the program successful when “skillful” forecasts are produced? When produced and communicated? When produced, communicated and considered for use? When produced, communicated and actually used, with demonstration of benefit from use? Given that the IRI originated from the earth science research community, and that the IRI personnel are primarily from that community, there has been a tendency to define success closer to that which the physical sciences can measure. Also, there are many (social and economic) reasons why an individual user might not “take advantage” of new scientifically-based information, even if its use would result in greater benefit in the long run. This too contributes to program leaders’ tendency to define success using metrics of those factors more in their control (such as the quantitative measure of “skill” of a forecast.) 3. Program organization Whereas a great deal of the resources have been dedicated to improvement of the climate models, and development of more “user-friendly” climate information products, a significant “boundary spanning” function is central to the IRI’s mission. The IRI spans: a) across disciplines (by employing physical, natural and social sciences at it’s facility in New York), b) between producers and users (by convening and participating in “user workshops” and climate outlook

28 LINKING KNOWLEDGE WITH ACTION forums), and c) between the more developed to less developed nations (by focusing primarily on: providing climate information for the ENSO-impacted regions in the tropical, less-developed nations, providing training for individuals from those regions, and conducting research in those regions.) 4. The decision-support system The end-to-end characteristic of the IRI is perhaps one of its greatest assets and most complete characteristics. It was explicitly stated in 1992 when the original concept was formally articulated that the IRI would be an end-to-end system…from global ENSO forecasts, using state-of-the-art climate models, through higher resolution regional forecasts co-produced with local forecasting entities, to fairly localized, practical forecast products that incorporate input from potential users of the information. The most difficult to implement has been that closest to the final user for a number of reasons. First, the stated needs of the users easily go beyond the capacity of the science to deliver; second, the need for resources (financial, technical, and personnel)to dedicate to the problem increase dramatically as one moves closer to the application, and third, as one gets closer to the use end, potential sources of funds tend to dry up. A couple of reasons for this might be, that scientific agencies who might otherwise fund inquiry into the problem at the global scale are generally not prepared to dedicate resources at the local scale. Further, as one gets closer to production of information that might be useful for individual resource managers, competition between them leads to a disincentive to finance activities to produce a “common good.” 5. Learning orientation The IRI was expressly experimental from the outset, as evidenced by the inclusion of the word “Research” in the title. This allowed the IRI to produce “experimental forecasts”, and hence, a chance to co-exist with the National Meteorological Services who claimed the “climate forecasting” domain as their turf. However, a forecast product heavily couched in “experimental” and “probabilistic” terms is less likely to be readily assimilated into decision-making processes, especially by those who do not understand the nature of the climate system and the inherent uncertainties within (i.e., literally “betting the farm.”) With the constantly updated climate forecast on the web, “outside evaluators” of the IRI are everywhere, from the scientific community to the user community. Other reviewers are built into the management structure, with a IRI Board of Overseers evaluating overall policies and budget, and an International Scientific and Technical Advisory Committee established to advise on the programs. Further, the program is reviewed every 5 years by NOAA as it considers renewal of the grant to Columbia University. 6. Continuity and flexibility The IRI was purposefully established by NOAA as a 5-year grant to Columbia University so that the institution would have some budgetary stability. Outside funds have been sought, and Taiwan has contributed financial support. Also, IRI is the recipient of funds from the USAID and the Inter-American Development Bank for specific projects. Nevertheless, the majority of the funds come from the NOAA Office of Global Programs which suffers from constant budgetary attacks from individuals who would rather see the funds go into the NOAA Labs.

APPENDIXES 29 Training of individuals both from the U.S. and abroad is a big part of the IRI mission. Further, Columbia University has recently created a “Climate Affairs” Masters Degree program which the IRI administers. 7. Other insights Any attempt to create an institution that radically changes the way things are traditionally done will be met with unbelievable opposition by those who would rather preserve the status quo. Without the strong leadership of one individual, then Director of NOAA’s Office of Global Programs, J. Michael Hall, the IRI would not have been established. Contact information James L. Buizer Executive Director of Sustainability Initiatives and Special Advisor to the President Arizona State University Tempe, AZ 85287 Tel: (480) 965-6515 Fax: (480) 965-0865 Email: james.buizer@asu.edu On IPA from NOAA: (was) Director, Climate and Societal Interactions Division NOAA Office of Global Programs 1100 Wayne Avenue Rm 1225 Silver Spring, MD 20877 Publications Cash, D.W. and S.C. Moser (2000): Linking global and local scales: designing dynamic assessment and management processes. Global Env. Change, 10:2 pp. 109-120. International Research Institute for Climate Prediction (IRI), 2001: Coping with the Climate: A Way Forward: Summary and Proposals for Action, A multi-stakeholder review of Regional Climate Outlook Forums concluded at an international workshop October 16- 20, 2000, Pretoria, S. Africa, 28 pp. Moura, A.D., ed., et al. 1992: International Research Institute for Climate Prediction: A Proposal, IRICP Task Group, NOAA, 51 pp. National Oceanic and Atmospheric Administration (NOAA), 1994: A Proposal to Launch a Seasonal-to-Interannual Climate Prediction Program, Office of Global Programs, Silver Spring, MD, 19 pp. ________1996: International Forum on Forecasting El Niño: Launching an International Research Institute, Forum Proceedings, Office of Global Programs (6-8 November, Washington, D.C.), 277 pp.

30 LINKING KNOWLEDGE WITH ACTION The NOAA Research Applications Program: Bringing climate research to bear on practical challenges associated with natural resource management and hazard mitigation through applications development and capacity building. Lisa Vaughan NOAA 1. Problem definition Climate variability and the associated fluctuations in rainfall and temperature patterns can have a significant impact on developing countries, affecting critical sectors such as agriculture and food security; water resource availability and management; disaster preparedness and civil defense; and public health and well-being. For example, the 1982-1983 and 1997-1998 El Niño events were associated with severe droughts and floods which occurred throughout much of the world. The footprint of these extreme climatic fluctuations on developing countries is shaped not only by the severity of the physical impact of an event, but by the existing infrastructure, capacity and coping strategies. Climate science and services have the potential to help improve the resilience of socioeconomic systems in the face of a variable climate by providing understanding and information products (e.g., climate forecasts) to decision makers in climate sensitive sectors and regions. However, a multi-disciplinary research, assessment and applications effort is fundamental to creating an effective bridge between societal need and capacity, and scientific insights and products. NOAA’s Research Applications Program focuses on the applications component of this end-to-end system by fostering the understanding and the technical, scientific and institutional capacity necessary to forecast and adapt to climate variability. The effort takes a place-based approach to resolving interrelated problems associated with research, institutional development and capacity building with support provided through a variety of funding mechanisms and partnerships with international, regional, national and local organizations. The regional thrusts of the program are the following: Africa, Latin America and the Caribbean, the Pacific Islands and Southeast Asia. The role of stakeholders in the climate applications initiative evolved over time, through an adaptive learning and management process. Recognizing the potential applications of climate research during the late 1980s, NOAA initiated a series of workshops on behalf of the research community. Participation in this applications dialogue was initially focused on the physical scientists involved in understanding the dynamics of the climate system, and the creation of observations systems and models to support this work. During the early 1990s, NOAA began to realize that the internal focus of these discussions was not likely to lead to the realization of any socioeconomic applications and value, so the program management staff began to purposefully incorporate an increasingly wider range of participants in this dialogue. The expansion began with social scientists, to help articulate the impact of climate on society and to begin to understand the potential applications of climate information in decision making (including opportunities and barriers related to its use). While the inclusion of another type of academic was useful, we soon realized that actual decision makers needed to be at the table to help frame their challenges and information needs, and to participate in a “negotiation” with the scientific community about what was desirable and feasible. Finally, a fourth group was sought out for participation in the dialogue: the intermediary technical experts and facilitators (e.g., agricultural extension services).

APPENDIXES 31 The outcome of this multilateral problem definition and more participatory approach is a much richer perspective on climate research applications, and a research agenda that is more attuned to societal need. For example, we now have a better understanding of the importance of socioeconomic context, and the role of the associated vulnerability and resilience of a sector or region in the effective use of scientific information products (e.g., scientific information is of no value if the capacity to utilize it does not exist or is unrecognized). In addition, starting with an impacts approach (or problem definition) has led to the specific study of other climatic phenomenon that influence rainfall patterns in specific regions (e.g., the Atlantic and South America). 2. Program management The NOAA Research Applications Program was initially developed as a pilot effort, with objectives related to: raising awareness of climate impacts and research applications; increasing capacity related to the successful use of climate information; and identifying research needs, including process studies, modeling and observations networks. The initial pilot effort was focused on one type of climate variability (El Niño-Southern Oscillation), and one scientific product (seasonal to interannual forecasts). The overall vision driving the pilot effort was the creation of a network of applications activities (then referred to as “centers”) throughout the developing world, and the connection of these applications capacities to a central forecasting and research entity (the International Research Institute for Climate Prediction/IRI). Thus, the creation of this network (including the IRI) and the associated capacities established a framework for the evaluation of the activities of the NOAA effort. Beyond this larger objective, however, the program did not have articulated metrics for measuring success (e.g., capacity enhanced by X amount in Y country). There were, however, indirect measures of success, including the use of climate information in decision making by groups like the US Agency for International Development and the World Bank, and the existence of new institutional arrangements and coping strategies to deal specifically with climate information. The managers of the NOAA effort, which included physical scientists, social scientists and political scientists, worked with the broader community to create and sustain goals and objectives. Program leaders were accountable to the director of the office, but also to a broader community who helped fund some of the applications work, including the USAID Office of Foreign Disaster Assistance. 3. Program organization The NOAA Research Applications Program served as a boundary or bridging organization, as it helped create and enhance boundary functions and relationships in the field. The program management staff was composed of individuals with diverse backgrounds, ranging from the physical and social sciences to tropical agriculture and development. As such, the group working on this project could facilitate and create linkages in the space between science and society by understanding context and language on both sides of the bridge. In the regions where the bulk of the work was conducted, NOAA sought to create structured and informal dialogues between scientists and decision makers. Examples of the formal (and sometimes virtual) boundary entities include a) the Climate Outlook Forums (COF), which bring together research scientists, operational experts from the weather services, decision makers, and technical intermediaries on a

32 LINKING KNOWLEDGE WITH ACTION regular basis to generate and analyze pending climatic conditions; and b) standing regional committees dedicated to integrating climate information and key socioeconomic factors into decision making processes. Often catalyzed by a specific need (e.g., pending ENSO event, post- Hurricane Mitch reconstruction), the NOAA effort sought to develop relationships and boundary functions that would continue to grow and be nurtured during times of non-crisis. Our experience has demonstrated that the highest chances for successful applications of scientific information exist in a system with ongoing and regular communication between scientists and decision makers, where each set of actors have an understanding of and trust in the others. 4. The decision-support system NOAA’s Research Applications Program seeks to catalyze and support end-to-end decision support systems. The framework utilized for these systems (in general terms) includes the following elements which do not occur in a linear, independent manner: 1) creation of a climate outlook through a participatory process; 2) dissemination of climate outlook information; 3) application of climate information; 4) evaluation of information and application; 5) applications research and development; 6) training and education; 7) sustained stakeholder dialogue. The reality is that we encourage the development of these various components, but do not have the financial or human resources available to invest adequately in every area for every region. We try to compensate for this resource issue by developing partnerships with other funding agencies (e.g., USAID, Inter-American Development Bank, World Bank, World Meteorological Organization, National Science Foundation) with a stake in the existence of such an end-to-end system. Some regions have been more successful than others in creating and sustaining an end- to-end system, due to resource constraints and cultural emphasis. 5. Learning orientation In the early 1990s, the NOAA Research Applications Program sought to productively connect an emerging scientific capacity in the form of seasonal to interannual climate forecasting—still in the development and experimental stages itself—with a broad, and as of yet unarticulated, societal need. Beyond literature related to technology diffusion and experience in weather forecasting, there was no roadmap to guide the agency in this effort. By necessity, then, the research applications pilot program was considered an experiment. Nature provided the community working in this field with a “field experiment” in the form of the 1997-1998 ENSO event. This event tested and shaped new, emerging institutions (e.g., IRI) and gave rise to new virtual institutions that continue today (e.g., COFs). The research applications program is housed in an environment that has historically encouraged calculated and strategic risk taking among its program management staff. A careful risk analysis was conducted, often in a small group setting, which weighed the potential benefits to be realized against any negative consequences. Failure is indicated by harm done to people, economies or the program effort; however, the environment encouraged program leaders to embrace and learn from their mistakes. The program formally seeks advice from outside evaluators in the form of peer-review of proposals, and has consulted with the external NOAA Climate and Global Change Panel as appropriate. In addition, there is a community of individuals supported by NOAA’s Human Dimensions of Global Change Research Program (HDGCR) which studies the use of climate information. Projects supported by the Research Applications Program have been part of the context within which these projects take place (e.g., an HDGCR study which analyzes

APPENDIXES 33 climate information usage in Africa might consider the effectiveness of a specific COF and associated training activity supported by the Research Applications Program). This relationship between two of NOAA’s programs strengthens our ability to fully realize a socially-relevant return on the agency’s investment in climate science. Finally, the NOAA Research Applications effort is now conducted within the same programmatic framework as a project on Knowledge Systems for Sustainable Development (KSSD). The KSSD project seeks to identify and articulate the characteristics of effective decision support systems. The research applications program serves as a source of real time experiments in decision support for rigorous study by the KSSD group (which is also looking at other sectors and topics), and will also be a beneficiary of the findings of the KSSD project. Linking the study of decision support to actual applications efforts serves to improve the role of science and technology in societal decision making processes, even as real impacts are realized from current applications. 6. Continuity and flexibility NOAA funding dedicated to the Research Applications Program has traditionally been relatively small in relation to its other research and assessment programs, and has essentially remained level for almost a decade. There are multiple factors that influence this situation, for example, including the perception in the physical science community and some of its managers that money invested in research applications represents less support available for advancing forecasting skill levels. On a more positive note, one rationale for this level of funding was that organizations with a stake in climate and the use of climate information would be willing to support applications activities that benefited their respective agendas. In large part, this principle has proven true. The Research Applications Program has leveraged funding from other USG agencies (e.g., USAID, NSF), international organizations (e.g., World Bank, Inter-American Development Bank, World Meteorological Organization), regional science institutions (e.g., Inter-American Institute for Global Change Research), and a large number of national and state organizations around the world. Co-sponsorship of activities with scientific and decision making organizations help maintain a consistent and problem-oriented effort. In several cases, NOAA-initiated activities, including the COFs, are wholly supported by other organizations. We consider this a success. 7. Other insights? A quick summary of some insights: • Full involvement of stakeholders (scientists, operational entities, decision makers, and intermediaries) results in a sense of ownership of the endeavor, and a more socially- relevant effort. This dialogue can be enhanced by the facilitation by an individual or organization that is perceived as legitimate and “neutral” by the respective parties (e.g., no political or financial stake in the outcome of the dialogue).

34 LINKING KNOWLEDGE WITH ACTION • Climate is an issue that benefits from international (and regional) collaboration, in spite of the challenges associated with working across cultural, language, political and disciplinary boundaries. • Understanding and enhancing regional, national and local capacity is essential to efforts to apply science and technology for sustainable development (i.e., the scientific product alone does not affect behavior). • A separate but well entrenched focus on applications can provide the incremental resources necessary to “connect the dots” between science and decision making in specific contexts. The resources for such an effort should be “fenced off” from other research and assessment activities (for several reasons, including the nature of the scientific review process…workshops, capacity building and targeted applications activities do not review against longer term scientific studies), but tightly linked to these other efforts in terms. • Full investment in research applications requires the development of a group of individuals that can serve the boundary function of bringing people and ideas together to create something that is larger than the sum of its parts. Contact information Lisa Farrow Vaughan Program Director for Environment, Science and Development NOAA Office of Global Programs 1100 Wayne Avenue, Suite 1225 Silver Spring, MD 20910 Tel: 301-427-2089 ext. 132 Fax: 301-427-2082 Email: Lisa.Vaughan@noaa.gov Representative publications/products An Experiment in the Application of Climate Forecasts: NOAA/OGP activities during the 1997- 1998 ENSO event. Available at: [http://www.ogp.noaa.gov/library/index.htm] NOAA Environment, Science and Development (ESD) Program (including Research Applications). Available at: [http://www.ogp.noaa.gov/mpe/csi/esd/index.htm] Coping with Climate: A Way Forward. An analysis and recommendations for action for the Regional Climate Outlook Forum process conducted by an international workshop of experts in October 2000 (Pretoria; South Africa). Available at: [http://www.ogp.noaa.gov/mpe/csi/doc/] EPA Global Change Research Program’s Great Lakes Regional Assessment (GLRA) Activity Joel Scheraga EPA

APPENDIXES 35 1. Problem definition Program objective. The Great Lakes Regional Assessment (GLRA) program is a stakeholder- oriented assessment program with primary focus on understanding the potential consequences of global change, and to assess adaptation options to increase resilience and improve society’s ability to effectively respond to the risks and opportunities presented by global change. The ultimate goal of the research and assessments conducted in this program is to provide timely and useful information to decision makers, resource managers, and other stakeholders. We constantly strive to bridge knowledge producers and users. No single “problem” to solve. The GLRA program entails an ongoing process. There is not a single “problem” to be solved by the program. Rather, on an ongoing basis, we engage both researchers and end-users to analyze, evaluate and interpret information from multiple disciplines to draw conclusions that are both timely and useful for decision makers, resource managers, and other stakeholders in the Great Lakes Region. Shaping the problem definitions. On an ongoing basis (within limitations imposed by resource constraints), we strive to first identify users and then to understand their needs, the particular effects of concern to them (e.g., changes in water quality), and the questions they would like answered. Throughout this process, key research gaps are identified and prioritized in order to produce the information needed to better answer the questions being asked by users over time. On a periodic basis, assessment products are produced using the best-available scientific and socioeconomic information to inform a particular set of policy decisions. The time frame within which the assessment products must be completed is defined by the users. Focus of this case study. This case study is a “snapshot” of two specific and related assessment products that were produced as part of our ongoing GLRA process, specifically: Collaborative effort with the US/Canada International Joint Commission: An assessment of adaptation strategies to increase the resilience of water resources in the Great Lakes Region to climate change and to protect their “beneficial uses” as required under the 1978 US-Canada Great Lakes Water Quality Agreement. (Client: The Water Quality Board [WQB] of the US- Canada International Joint Commission [IJC]) Collaborative effort with mayors in the Great Lakes Region: A preliminary assessment of the potential effects of climate change on combined sewer overflow (CSO) events in the Great Lakes Region. (Clients: EPA Region 5 [Great Lakes Region]; mayors in the Great Lakes Region) Key insights for the NAS Task Force derived from EPA’s GLRA program. Key insight #1: For an assessment product to be informative, the assessors must know the particular issues and questions of interest to stakeholders – those parties with an interest in the consequences of a problem or its solution. Key insight #2: Stakeholders/users should be engaged throughout the assessment process; i.e., they should be involved from the outset of the assessment process and then involved in the analytic process on an ongoing basis.

36 LINKING KNOWLEDGE WITH ACTION Key insight #3: Openness and inclusiveness enables different participants to bring a diversity of views and information that may benefit the assessment process. Also, including all interested parties makes the assessment process more transparent and credible. Key insight #4: For an assessment to be timely, the assessors must understand how the information will be used by the relevant stakeholders and the time frame within which the information is needed. Key insight #5: Researchers/assessors and stakeholders are not necessarily distinct communities. In many cases, the stakeholder community can offer data, analytic capabilities, insights and understanding of relevant problems that can contribute to the assessment. 2. Program management Both of the assessment products were developed in a “project” or “product” mode, with specific, user-defined deliverables that had to be completed by a particular date. In both cases, the National Program Director for EPA’s Global Change Research Program (Dr. Joel Scheraga), as well as specific project managers (Mr. John Furlow, Dr. Jordan West), were held accountable for successful completion of the deliverables in a timely fashion. IJC Assessment. The IJC activity is an excellent example of the usefulness of a problem formulation exercise at the outset of an assessment process intended to link knowledge with action. More specifically, it is an example of how researchers inform a particular group of users about the best available science on a particular topic, the users then identify the specific issues and questions of concern to them, and an assessment plan is formulated. In 2002, the IJC Board of Commissioners charged the WQB with developing adaptation strategies to increase the resilience of water resources in the Great Lakes Region to climate change and to protect their “beneficial uses” as required under the 1978 US-Canada Great Lakes Water Quality Agreement. The WQB recognized that two Great Lakes Regional Assessments of the Potential Consequences of Climate Change had just been completed: one had been sponsored by Environment Canada and one by the U.S. EPA. The WQB decided to build off of these assessments in order to arrive at recommendations that it could use in developing an adaptation strategy. Following the IJC Board of Commissioners’ charge to the WQB, Dr. Linda Mortsch (Environment Canada) and Dr. Joel Scheraga were invited by the WQB and Board of Commissioners to brief them on the potential consequences of climate change for the Great Lakes Region in February 2002 and April 2002, respectively. Following successful presentations on climate science and potential impacts in the Great Lakes Region, Mortsch and Scheraga were commissioned by the WQB to co-author an assessment of possible IJC adaptation strategies (hereafter referred to as a “white paper”). This paper was successfully completed, peer reviewed, revised, and presented to the WQB in September 2003. Based on the conclusions of the white paper, the WQB and Board of Commissioners recommended a set of adaptive actions that the IJC could implement to help protect the beneficial uses derived from water resources in the Great Lakes Region from climate change.

APPENDIXES 37 Follow-up CSO assessment. Climate change will likely increase the frequency and intensity of rainstorms, potentially affecting the frequency of combined sewer overflow (CSO) events. During the IJC assessment activity (e.g., at a stakeholder and peer review workshop), the particular issue of the effects of climate change on combined sewer overflow (CSO) events was identified as an increasing issue of concern for particular stakeholder groups. EPA’s Region 5 Office (Great Lakes Region), requested that EPA’s GLRA Program complete a preliminary assessment of the potential effects of climate change on CSO events in the Great Lakes Region. Region 5 works closely with mayors in the Great Lakes Region and was responding to their expression of interest about the subject. The preliminary assessment showed that if combined sewer systems meet the EPA’s CSO Control Policy design standard of 4 events per year, then (1) climate change may result in failure to meet the standard; (2) there could be an average of 334 events per year above the control policy’s objectives across 220 communities in the Great Lakes Region; and (3) storage/treatment capacity would need to increase, thus increasing system costs. The success of this study, combined with other insights related to water resources gained in the EPA-sponsored Great Lakes Region Assessment (2000), led to several invited presentations of the assessment findings to stakeholders in the Great Lakes Region and the Northeast Region. One important presentation was made at the Great Lakes Cities Initiative conference hosted by Mayor Richard M. Daley (Chicago) in December 2003. The presentation, entitled “Preparing for a Changing Climate: Opportunities for Cities in the Great Lakes Region,” introduced mayors in the Great Lakes Region to the potential impacts of climate change on water resources, and potential adaptation strategies they could implement to increase the resilience of their cities to change. A second important presentation, entitled “Water Resources: An Emerging Challenge,” was given at a bilateral (US/Canada) symposium, “Climate Change in New England and Eastern Canada: Natural Resource Impacts and Adaptation Responses,” in March 2004. The symposium was held under the auspices and direction of the Conference of New England Governors and Eastern Canadian Premiers. These and other presentations of our assessment findings have led to an ongoing dialogue with communities and decision makers in the Great Lakes Region and the Northeast Region about specific scientific questions and water-related issues of concern to them, including the potential implications of climate change for combined sewer overflow (CSO) events. The issue of CSO events is of particular concern to decision makers because of the significant investments they are now contemplating to rebuild sewer systems in major urban areas. Key insights for the NAS Task Force derived from EPA’s GLRA program. Key insight #1: It is sometimes difficult to immediately identify all constituencies that might have an interest – a stake – in a particular environmental problem. One of the lessons of the GLRA activity has been that new stakeholders often are identified during the course of an assessment process. The process of identifying and involving stakeholders must be an ongoing process. Key insight #2: Even with stakeholder involvement, research scientists often are hesitant to make definitive statements that might be used by policy makers because scientific uncertainties still exist; the science is not yet “perfect.” Yet, policy makers often have to make decisions under

38 LINKING KNOWLEDGE WITH ACTION uncertainty, whether or not scientists are prepared to inform those decisions. GLRA assessors strive to answer decision makers’ questions to the extent possible given uncertain science, in the belief that informed decisions are better than uninformed decisions. They also characterize the uncertainties and explore their implications for different policy or resource management decisions. 3. Program organization Our GLRA program views the process of linking knowledge to users as consisting of four principal elements: problem formulation, analysis, characterization of consequences, and communication of results. We view the communication of results as a critical phase throughout the assessment process. If the purpose of the effort is to convey insights to decision makers, communication during the problem formulation stage is important to ensure that useful assessment endpoints are identified and pursued. Not only should information needs be identified, but analysts should understand how and when stakeholders will use assessment information. Will end users find and read a scientific journal article? Would they prefer a tool or a model to help them evaluate and employ assessment results? If the audience is the public, is it best served by a pamphlet that simply and accurately relates the findings? Understanding the audience’s ultimate needs shapes the communications strategy. Effective communication of assessment results helps analysts and stakeholders alike to identify additional research and assessment priorities. Effective communication also encourages stakeholders to conclude that their contributions are being utilized and their needs for information are being effectively met. We require that our academic partners – in this case, the University of Michigan (during the first phase of our GLRA effort) and Michigan State University (during the current phase of GLRA activities) – build a boundary spanning function into their assessment activities. Since funding awards to our partners are all made through competitive processes, we were able to make the inclusion and implementation of a boundary spanning function a requirement in the Requests for Assistance (RFAs) when the competitions occurred. Example of an important boundary spanning activity. When the first EPA-sponsored GLRA product was produced in 2000 (prior to the IJC activity), it was critical for our academic partners to follow through on the communication of assessment findings to various stakeholder groups. As part of their boundary spanning responsibilities, the GLRA team hosted five stakeholder- oriented workshops to inform users about the assessment conclusions and to elicit new information needs. (Participating users included those involved from the outset of the assessment activity, as well as new users who had been identified during the assessment process as having a potential interest in the results.) The five workshops included: • Great Lakes Water Levels (March 2001): Focus on shipping, recreational boating, safety, and infrastructure • Lake Ecology (June 2001): Focus on productivity and fishing • Agriculture (March 2002): Focus on farming, insurance, adaptation • Terrestrial Ecology (June 2002): Focus on forests, wildlife, and timber industry • Recreation (October 2002): Focus on winter recreation and economy

APPENDIXES 39 It was during the March 2001 Great Lakes Water Levels workshop that initial interest was expressed by the IJC in our GLRA activities. This highlights again that the GLRA program entails an ongoing process, with new user needs identified over time and specific assessment products delivered at different points in time. Key insights for the NAS Task Force derived from EPA’s GLRA program. Key insight #1: Boundary spanning activities are essential, but require a major, ongoing investment. They are resource intensive, requiring significant time, financial, and personnel resources. Key insight #2: Although establishment of a boundary spanning function is critical, one must be careful to delineate between the roles of researchers/assessors and the decision makers/users. The role of the GLRA program is to inform decisions makers and resource managers, not to make policy decisions. We view our responsibility as being to evaluate alternative response strategies, not to choose a “best” policy response. This is a policy decision that inherently depends upon social values and selection criteria that must be identified by decision makers. 4. The decision-support system The activities of the GLRA program do not support the notion that linking knowledge with action always requires end-to-end integrated systems. As was suggested earlier, we believe it is critically important that assessors listen to the decision makers they are trying to serve, and try to understand the types of information they need, the time frame in which the information is needed, and the ways in which the information will be used. Admittedly, in some cases, decision makers will demand information that requires the development of end-to-end integrated systems. The research efforts required to develop integrated systems tend to be data intensive, resource intensive, and difficult to complete. But for a wide range of decisions, integrated end-to-end systems are neither necessary nor in some cases appropriate. For example, a decision maker (e.g., an engineer in Chicago responsible for designing a new and expensive sewer system that will be in place for the next 50-100 years) may simply want to know whether or not climate change is an issue of concern and should be factored into a decision making process taking place today. The decision maker may recognize that once the investment is made and the new infrastructure (e.g., sewer system) is in place, some future opportunities to adapt to a changing climate may be foreclosed. In these cases, simple bounding analyses may suffice to provide the necessary information. In other cases, a stakeholder (e.g., an owner of a shipping company that transports freight across the Great Lakes) may wish to understand what opportunities may be presented if the climate changes in certain ways (e.g., when longer shipping seasons occur as ice cover on the Lakes lessens). The stakeholder may be interested in understanding relative changes in economic activity and business opportunities in particular sectors of the Great Lakes Region as climate change occurs. Key insights for the NAS Task Force derived from EPA’s GLRA program. Key insight #1: If the ultimate goal of decision support is to provide timely and useful information to decision makers, then the analytic approach that is taken should be driven by the users’ issues and questions of concern. Once the issues and questions of concern have been identified, an appropriate analytic technique for answering the questions can be identified.

40 LINKING KNOWLEDGE WITH ACTION Key insight #2: We view decision support as a process, as opposed to a product (particularly a product requiring a specific methodology, like integrated end-to-end modeling). In the course of the process, particular approaches and tools that are best suited for answering the questions being asked by decision makers are identified. Key insight #3: Our GLRA approach to decision support can be simply put as: “Right model/approach for the right question!” 5. Learning orientation The GLRA program is an applied, stakeholder-oriented assessment program. But the process- orientation of our GLRA program, as opposed to product-orientation, inherently lends itself to “learning by doing.” Although the GLRA activities are intended to provide timely and useful information to decision makers, it is recognized that GLRA studies will not likely be able to completely answer all of the questions posed by stakeholders. The GLRA program must entail an ongoing process to reflect new scientific information, elimination or creation of new uncertainties, or changes in scientific understanding or beliefs. In this sense, the GLRA program is experimental. Our experience in the GLRA program is that it is usually possible to conduct an analysis of the best-available scientific information at any point in time — despite the existence of uncertainties — in response to questions being posed by users. (This does not preclude the possibility that an analysis may conclude that insufficient scientific information exists to provide any useful insights to stakeholders in the time frame specified.) But assessment is an ongoing process. Scientific uncertainties will exist and unanswered user questions will remain. And the science may change and uncertainties reduced or increased, with resulting implications for policy and resource management decisions. New assessments must be conducted as new scientific information is produced and uncertainties reduced. Value of information. The GLRA program uses a “value of information” (VOI) approach to identifying what problems to study, research to invest in, and what risks to take. The VOI exercises are periodically conducted to identify key research gaps, new research questions for the intramural and extramural (grants) research programs, and new assessment questions relevant to the decision needs of stakeholders in the Great Lakes Region. The last step in any particular Great Lakes assessment is the identification and prioritization of “key” research gaps, i.e., those knowledge gaps that must be filled in order to answer stakeholder questions. Some of the stakeholder questions will be the same as those asked at the outset of the assessment process. But the stakeholders may have new questions they wish to pose, either because of the insights they have already gained from the assessment process or because of changes in other factors unrelated to the assessment process. Because the resources available for conducting research related to an assessment process are scarce, research needs must be prioritized. Research dollars that are used to support assessments need to be directed to their highest-valued uses, i.e., toward producing timely research products that fill key knowledge gaps that are needed to answer stakeholders’ questions. This requires that

APPENDIXES 41 VOI calculations be done (either explicitly or implicitly). Such calculations yield insights into the incremental value to stakeholders of information expected to be derived from an investment in a particular research activity. The results of these calculations depend on changing stakeholder needs and values, and the timeliness and relevance of information. “Success” and “failure”. We prefer to think of the results of GLRA activities as “useful” or “not useful,” where usefulness is a function of timeliness. Ultimately, the usefulness of GLRA studies is determined by the users, who have been engaged from the outset of any particular assessment process. Having said this, another (more bureaucratic) measure of success or failure of the GLRA program and the National Program Director is whether: (1) well-defined activities in the Research Strategy for EPA’s Global Change Research Program are completed in a timely fashion; and (2) whether specific well-defined goals and measures to which the GLRA program has committed as part of the Government Performance and Results Act (GPRA) have been met. EPA’s Global Change Research Program has developed a Research Strategy (consistent with the Strategic Plan for the U.S. Climate Change Science Program). The Research Strategy articulates a vision of the Program’s long-term goals for developing assessments of global change issues and the research to support such efforts. The Great Lakes Regional Assessment activity is one component of the larger EPA Global Change Research Program, and the strategic vision for the GLRA efforts are explicitly described in the Research Strategy. The Research Strategy describes the direction of the Program, not its implementation. As a result, it provides only the framework of the research and assessment process, not an itemization of specific projects. A companion document, the Multi-Year Plan (MYP), provides an implementation plan for accomplishing the work described in the Research Strategy, including the GLRA program. The Research Strategy and MYP are consistent with requirements of the Government Performance and Results Act (GPRA), which require agencies to provide the Congress with measurable “annual performance goals” and “performance measures.” The MYP establishes interim performance goals and measures for the next 10 years of Program activities. The MYP is revised annually based on congressional budget appropriations. The ability of EPA’s Global Program to achieve its long-term goals and to fulfill its role under the Global Change Research Act depends, in part, on adequate appropriations. External review. The EPA Global Change Research Program and the GLRA component are committed to the highest standards of scientific excellence. This includes extensive independent peer review of (1) the long-term Research Strategy for the program; (2) all research and assessment activities (including the GLRA activities); and (3) all research and assessment products. We also conduct periodic external reviews of the past performance of the program (i.e., retrospective reviews, as opposed to reviews of planned future work).

42 LINKING KNOWLEDGE WITH ACTION The Global Change Research Program’s Research Strategy was peer reviewed by an external panel convened in Washington, D.C. on February 15-16, 2001. More than 250 individual comments from that panel were addressed when the Research Strategy was revised and finalized. Key insights for the NAS Task Force derived from EPA’s GLRA program. Key insight #1: The GLRA program entails an ongoing process. This process facilitates incorporation of new scientific information, elimination or creation of new uncertainties, or changes in scientific understanding or beliefs. In this sense, the GLRA program is experimental. Key insight #2: Ultimately, the users determine whether particular GLRA activities are useful or not (i.e., successes or failures). Key insight #3: Value of information (VOI) are required to identify the highest-priority research and assessment activities within the GLRA. VOI exercises can be expensive to undertake, but need to be part of any assessment process. Key insight #4: It is essential to conduct regular external peer reviews of all components of a program like the GLRA that has as its goal to provide timely and useful decision support. 6. Continuity and flexibility As noted in our response to the previous question, a Multi-Year Plan (MYP) serves as the implementation plan for the program’s long-term Research Strategy. The MYP lays out “critical paths” for completing each research activity called for in the Research Strategy. The MYP is a “living document” that is revised annually to reflect changes in our understanding of the science, experiences with and lessons learned from our research and assessments, and annual congressional budget appropriations. The MYP is revised annually based on congressional budget appropriations. As noted earlier, the ability of EPA’s Global Program to achieve its long- term goals and to fulfill its role under the Global Change Research Act depends, in part, on adequate appropriations. For example, changes in the MYP may reflect the fact that funding has declined for the GLRA, so that fewer activities along the critical path can be completed in any particular year. Multidisciplinary nature of GLRA: The GLRA activities are multidisciplinary endeavors. Ideally, assessment teams are composed of researchers from a variety of disciplines working together to address complex research and assessment questions. Because of the complexity of the issues involved, user-relevant assessments require insights from multiple, diverse disciplines. But the different disciplines can’t work in isolation from one another. They must interact and work together on a regular basis. As an incentive to potential collaborators, our GLRA program requires in all competitions for funding (e.g., grants) that multidisciplinary teams be assembled. Also, teams must include some representation from user groups in the Great Lakes Region. Leveraging with other private funds: To ensure adequacy and continuity of funds for the GLRA (as well as other assessment activities of the EPA Global Change Research Program), we

APPENDIXES 43 encourage our collaborators to locate other private and public funding for various components of GLRA activities. Lists of funding partners available upon request. Contact information Dr. Joel D. Scheraga National Program Director Global Change Research Program U.S. Environmental Protection Agency Office of Research and Development Mailing address: Mail Code 8601-N 1200 Pennsylvania Avenue, NW Washington, DC 20460 Fed Ex address: 633 Third Street, NW Room 7101 Washington, DC 20005 Phone: 202-564-3385 Fax: 202-564-2018 Email: Scheraga.Joel@epa.gov Representative publications/products Website of the EPA Global Change Research Program: [www.epa.gov/globalresearchhttp://cfpub.epa.gov/gcrp/about.cfm] Papers about the program or descriptive of the approaches taken in the program Scheraga, J.D., and J. Furlow, “From Assessment to Policy: Lessons Learned from the U.S. National Assessment,” Human and Ecological Risk Assessment, Vol. 7, No. 5, 2001, 1227-1246. [http://cfpub.epa.gov/gcrp/recordisplay.cfm?deid=23887] Scheraga, J.D., J. Furlow, J. Gamble, A.E. Grambsch, S. Julius, and C.E. Rogers, “Assessing the Consequences of Global Change for the United States: An Overview of EPA’s Global Change Research Program,” in Proceedings of the 12th Symposium on Global Change and Climate Variations (81st Annual Meeting of the American Meteorological Society), Albuquerque, New Mexico, January 2001. [http://cfpub.epa.gov/gcrp/recordisplay.cfm?deid=18219] Scheraga, J. D., and A.E. Smith, 1990. Environmental policy assessment in the 1990s. Forum for Social Economics, Volume 20(1), 33-39. Sample key products International Joint Commission Water Quality Board, Climate Change and Water Quality in the Great Lakes Basin, Report of the Great Lakes Water Quality Board to the International Joint Commission, August 2003. [http://www.ijc.org/php/publications/html/climate/index.html]

44 LINKING KNOWLEDGE WITH ACTION Sousounis, P.J., and J.M. Bisanz, editors. Preparing for a Changing Climate: The Potential Consequences of Climate Variability and Change: Great Lakes, University of Michigan, 2000. [http://cfpub.epa.gov/gcrp/recordisplay.cfm?deid=18667] Regional Integrated Sciences and Assessment (RISA) Program Claudia Nierenberg NOAA The RISA program supports research that addresses complex climate sensitive issues of concern to decision-makers and policy planners at a regional level. The research team members are primarily based at universities though some of the team members are based at government research facilities. A few of the researchers are affiliated with non-profit organizations or private sector entities. Traditionally the research has focused on the fisheries, water, wildfire, and agriculture sectors. The program has begun to support research into climate sensitive public health issues. Recently, coastal restoration has also become an important research focus for some of the teams. 1. Problem definition The “problem” to be solved that led to the creation of the Program was that NOAA’s Climate and Global Change Program lacked integration and the ability to connect well (and by design) to issues faced by decision makers (whose “problem” was rarely framed as “climate”). The Program was launched in order to define problems or challenges for which climate information and data might be useful. Each set of investigators within a region was asked to design a research agenda in partnership with stakeholders in their particular region. Indeed, “region” would be refined through interaction with decision makers from a relatively broadly-defined area facing climate-sensitive challenges. Whereas the problem that the C&GC Program addressed was establishing and characterizing predictability of the climate system (with the exception of small investments in Human Dimensions and Applications), the RISA program established a way to legitimize the pursuit of problems defined differently. In the Southeast, the “problem” was defined in terms of the vulnerability of important crops; in the Pacific Northwest the problem was variations and changes in water supply, in California one prominent stakeholder-defined issue is the restoration of the San Francisco Bay Delta and the resolution of competing resource demands. In the Southwest, fire risk and the spread of disease have dominated parts of the research agenda. The user-driven dialogue in each case was designed and implemented by the individual teams and given a high priority in the context of program goals. 2. Program management During the first stage of the RISA program, simply legitimizing a process through which a rather traditional earth science program could make resources available for building interdisciplinary teams whose first task was to identify decision-makers confronting climate-relevant challenges, was a substantial goal in itself. We built the design around phases where the first phase was

APPENDIXES 45 devoted to discovery and team building. A parallel initial task, rooted in the identity of the C&GC program, was that of climate diagnostics in a regional context. In general, program leaders have been held accountable through review processes similar to those used in other research programs, albeit with variations designed to address an interdisciplinary, problem-driven agenda. They are also held accountable, in effect, through the reputations they build with the community of decision makers in their individual regions. One of the principal goals of the Program is to expand the options available to decision makers. A true understanding will require a more concentrated effort on evaluation (both internal and external). As the individual RISAs have matured, they have adopted a project mode and though defining measurable goals and targets has not been stressed enough, in retrospect, they have certainly achieved one of the overall goals which was to demonstrate, in practice, the potential utility of climate information in very specific contexts. And through this they have also demonstrated the value of an “impacts” focus in terms of revealing uncertainties in our understanding most critical to decision making. One of the latest innovations is the NOAA Climate Transition Program (NCTP) which is designed to encourage RISAs and others to focus attention on those research innovations ready for transfer to operational settings. The NCTP will help in focusing some of the goals and targets. 3. Program organization This would be best answered by the individual RISA managers, all of whom are currently accountable to both their regional user communities and to the science communities. They have proven to be particularly innovative at organizing the dialogue between scientists and practitioners. Indeed, a review of their techniques and experiences would be tremendously useful to this community. They have experimented with public forums, regular and sustained meetings, proactively seeking opportunities to participate in technical or professional meetings, disseminating material through web sites and targeted publications, identifying research partners who sit in resource management agencies, and a range of other techniques. Insisting that the research team be resident in their region of study, so that they were also stakeholders, contributes to their success in creating a boundary organization. And to some extent, the Climate and Societal Interactions group has characteristics of a boundary organization. An interagency decision support capacity that had characteristics of a boundary organization would be useful to building an integrated earth science research program able to better connect knowledge and practice. 4. The decision-support system Interestingly, and probably most appropriately, each RISA project has developed its own version of “end-to-end” integrated systems after a process of issue identification and team formation. The Climate Impacts Group, in the Pacific Northwest, for example, takes an integrated view of climate, natural resources, and socio-economic systems. Their sectors include water resources, salmon, forests, and coasts (with a desire to move into health and agriculture). They are

46 LINKING KNOWLEDGE WITH ACTION investigating the critical interactions among resources (and resource management) that will shape regional impacts of climate variability and change including climate/hydrology/water management and the consequent impacts on fish and forests, water availability, and water quality through loss of snowpack and the effects on ecosystems. Their research components include climate statistics and dynamics, hydrologic modeling, reservoir operations modeling, user interviews and historical studies, surveys, institutional mapping and policy analysis, and policy and economic evaluation. Other RISAs have developed differently; each according to its own particular circumstances and strengths. That in itself has been considered a success of the program. The Southwest, for example, stressed vulnerability analysis early on and developed sector-based integrated models designed to capture the complexity of an issue like fire risk with its physical, natural, and social aspects. And in the Southeast, the initial focus was deep within an individual sector with horizontal expansion coming after several years of experience working deeply across scales of decision making relevant to agriculture. Only now, a decade later, are we starting to see some network capabilities emerge that may provide for greater efficiencies across the suite of projects. Only the teams themselves could really explain which were the hardest elements to put in place, but from the program management perspective, the elements involving institutional decision making, barriers to the use of information, and capturing the multiple stress nature of the problems as they exist in practice, seemed to present the greatest challenges. While there is lots of evidence that decision makers within the RISA regions are responding to having been brought in as participants in this process, and RISA has produced a number of decision support tools at various stages of development and testing, we are lighter than we should be on effective evaluation methods for the program overall and its lasting impact on adaptive capacity. 5. Learning orientation The program had an expressly experimental orientation that was central to its design. It had to because it looked so different than the other programs within NOAA’s Climate and Global Change portfolio. We knew almost instinctively that we could not have gotten off the ground through traditional means of scientific advisory bodies and an open competition around interesting questions as defined by scientists. We worked hard to forge partnerships with the emerging research teams and create an environment of experimentation and learning. Their early experiences and lessons shaped the further development of the program. And, generally the philosophy of experimentation characterized the interaction between the researchers and the stakeholders. One of the biggest risks was that we wouldn’t find a ready (or ready enough) community of decision makers interested in climate information. Another was that investigators wouldn’t be able to stick with the long start-up time of a project like RISA and find it professionally rewarding enough to put in the time it took to work with stakeholders. With all of the attention in recent years on adaptation and assessment, finding outside evaluators from the scientific research community is no problem. Finding outside evaluators from the resource management and other relevant communities is more difficult, but is happening.

APPENDIXES 47 Although the experimental approach was key to the success of the program in many ways, it also resulted in a lack of specificity on our part (NOAA management) about specific project goals early in the process. We are working right now on a more well-defined long-term strategy. One of the successes, is that we knew to build this program as a deliberate learning experiment for the program overall. We have not pursued that nearly far enough, but it is there waiting. 6. Continuity and flexibility The RISA program, as a network of regional, integrated, sustained assessments research and development centers, was scoped at approximately $20M annually. It is a relatively small investment on the part of the federal government for a program intended to build regional capacity and research insights critical to adaptation to climate variability and change. With 8 centers established (and at varying degrees of maturity) the budget is only $4 M from NOAA. The level of funding has been a profound constraint leading to the loss of some key personnel as well as a reduced ability to demonstrate utility to the whole of the scientific research endeavor. The funding we have acquired, as well as future growth from NOAA, is firmly connected to the critical role the RISAs play in the overall NOAA Climate Services strategy Overall levels aside, we purposely initiate each project with relatively modest funding in order to focus on the proof of concept within any given region and encourage the establishment of key relationships with a small number of stakeholders. Some of the RISAs are currently receiving resources from state agencies with resource management challenges and from other federal mission agencies. There are one or two cases of funding or personnel transactions with the private sector, but they tend to be highly specific. One of the innovations that we imagined even aside from funding limitations was the emergence of certain efficiencies once a core number of centers were established. In other words, expertise in fire risk, or climate-hydrology interactions, or water banking analyses could be tapped into rather than having to develop it locally in every instance. The RISAs have also become fairly skilled at attracting federal funds outside of NOAA, and while this may represent success on their part, it implies a certain failure on the program management side that we have not been able to provide a coordinated federal announcement or announcements for this community. 7. Other insights? There still exists a mismatch, in part, between what is success or failure in linking knowledge to action and what is success or failure in managing federal research programs. In part, the problem is historic and cultural. Success from the perspective of a federal research program is a high quality peer-reviewed system that gets the funding out the door in a timely and effective manner and can demonstrate a long-list of peer-reviewed publications. I am overstating a little, because NOAA has responded to RISA output that shows an ability to characterize and present scientific insights in terms meaningful to real-world challenges. But there is not nearly enough pressure to evaluate—critically and consistently—the extent to which practice or action benefited from the incorporation of new insights. And there is not nearly enough emphasis on the importance of innovative personnel and management options. One of our greatest successes was bringing a “real live stakeholder” onto our staff for a limited time. The arrangements were not easy and it was not exactly an encouraged practice. Yet it resulted in the manual entitled, Connecting

48 LINKING KNOWLEDGE WITH ACTION Science and Decision Making (Available at: http://www.wcfia.harvard.edu/conferences/sustaindev/papers/CashRecommendsJacobs.pdf). NOAA is building a climate service, but what defines that service and how do we know for sure that the necessary connections are being made in terms of enhancing decision options and building capacity to adapt to or take advantage of climate variability and change? Will investments in a climate service ever be based on the extent to which action benefits from knowledge? It has often been suggested over the course of the RISA experience that we design a “federal RISA” that builds and sustains interactions with “stakeholders” as a basis for informed research investments. 8. Other issues? Are there better ways to consider the whole of the federal investment in ways that reveal more consistently or vividly our most pressing challenges that call for new knowledge? Is there a federal forum that involves partners from both the Executive and Legislative branches of government as well as the university, private, and non-governmental communities, to discuss such topics as the management of urban sprawl, changes in water delivery and supply, the role of climate in the emergence and spread of disease? Could such a forum or process inform the development of scientific research agendas? Contact information Harvey Hill Program Manager, Regional Integrated Sciences and Assessment NOAA Office of Global Programs 1100 Wayne Avenue, Suite 1225 Silver Spring, MD 20910 Tel (301) 427-2089 ext 197 Harvey.Hill@noaa.gov Representative publications/products Links to all of the individual RISA web-pages which includes publications are available at: [http://www.ogp.noaa.gov/mpe/csi/risa/index.htm] NASA Earth Science Results to Support Air Quality Planning and Forecasting Activities Lawrence Friedl NASA 1. Problem definition EPA, NOAA, and other Federal organizations have significant operational responsibilities relative to delivering air quality information to the public. The NASA Earth Science Applications program extends NASA’s Earth science research results to national-regional organizations that have air quality management responsibilities and mandates to support air

APPENDIXES 49 quality managers. This case study reflects NASA’s primary work with EPA and the increasing involvement of NOAA. EPA has several activities related to air quality. NASA collaboration with EPA has focused on two activities – AirNOW and the Community Multi-scale Air Quality model (CMAQ). CMAQ (CMAQ/Models-3) is a comprehensive air quality modeling system that simulates processes to describe the generation, fate, and transport of atmospheric pollutants and urban, regional, and national air quality over several time scales. EPA, states and Regional Planning Organizations (RPOs) use CMAQ to simulate effects of pollution control options, assess multi-pollutant impacts, track and predict changes in emissions mitigation strategies, develop implementation plans, and make regulatory decisions. EPA’s AIRNow system gathers information from numerous sources to develop air quality forecasts, and EPA AIRNow developed the AQI as a health-based index for reporting air quality. EPA, state and local agencies, and the media report current and forecast AQI and air quality conditions, especially for ozone and particle pollution (PM). In 2002, NASA and EPA decided to examine how NASA research results might serve CMAQ. In particular, the program supported researchers and systems engineers to examine if Total Ozone Mapping Spectrometer (TOMS) ozone data and Global-to-Regional models (GOCART, RAQMS) could provide regional boundary conditions to initialize CMAQ. In 2003, however, EPA leadership decided to pursue the development of a PM transport rule. An EPA liaison/researcher located at NASA-Langley expressed this priority activity to the program. Following an initial assessment that NASA data might be useful in PM transport, the program decided to focus activities on the PM issue and delay the CMAQ work. NASA & EPA researchers verified and validated NASA Earth observation satellite measurements (including MODIS aerosol optical depth-AOD and MODIS cloud optical thickness-COT), with EPA ground measurements (AirNOW) and found promising correlations. The partnership supported activities to extend the products to EPA AirNOW’s air quality forecasting activities. At EPA’s request, NASA provided a near-real-time data-fusion product for air quality that served as a prototype during the “pollution aerosol season” in September 2003. This prototype involved MODIS AOD & COT, NOAA wind speeds and air trajectory models and fire locations, and EPA ground data. The prototype served a subset of air quality forecasters, who used the 3-day visualizations of the data-fusion products to assess transport of aerosols into their region and develop the air quality forecasts they issued. The program supported activities to benchmark the use of research quality data streams by documenting the prototype and assessing lessons learned from the forecasters. The forecasters provided feedback on various ways they used the product. Some of their uses differed from the original, expected uses. Their feedback helped NASA and EPA make adjustments to the products for 2004. In addition, EPA provided funds in October 2003 to support the transition of the product to an operational environment, eventually allowing NASA researchers to focus again on improvements and future prototype products.

50 LINKING KNOWLEDGE WITH ACTION 2. Program management In FY03, the activity developed into, and adopted, a project mode. Initially, the examination of the NASA satellite-derived information products was exploratory with respect to EPA use in air quality products. Therefore, the progress had to be “staged”. For example, until there were indications that the correlations between MODIS AOD and EPA ground measurements showed promise, the discussion of a prototype was premature. Thus, the “measurable goal” was largely binary—is this something of value to explore/develop further or not? The researchers initially made this determination of value. Following presentations of the results (i.e., correlations, data- fusion techniques, and visualizations) in several forums, EPA expressed interest in a prototype. At that stage, the activity pursued a project mode to develop the prototype for September 2003— the goals were largely to produce the product as promised to the forecasting community and to prepare a report of the results of the prototype. The project leaders were inherently accountable through the products and results—if EPA had not been interested in the product or the results, then the project would have been re-vectored or terminated. If the targets had not been met on schedule, then the NASA and EPA program managers would have evaluated the circumstances and could have redistributed project resources to other more promising collaborative projects. In FY04, the activity to transition the products to an operational environment is pursuing a much more standard project mode. The project plan has clear goals and objectives, work breakdown structures, and budgets. In future years, as NASA’s involvement in this activity focuses again on evaluation and verification work (rather than transition), the project plans will revert to a more open style of goals and objectives—accountability will focus on whether determinations of value of Earth science product were made (rather than were the determinations positive). 3. Program organization The program involved several types of boundary organizations. First, the NASA Earth Science Applications Program is designed to support the transition of research to federal/national organizations and the user communities. Thus, the activities the program funds are directly focused on activities that will support the “bridges.” Second, the program benefited extensively from the presence of an EPA liaison who was permanently assigned to NASA and located at Langley Research Center. This EPA employee worked for EPA’s Office of Air Quality Planning and Standards and was a researcher/scientist. He served as a bridge between the technical issues associated with the air quality data and the policy issues and priorities within EPA. He could bring NASA results back to EPA, and he could report EPA’s priorities to NASA. In addition, he helped transcend the cultural differences between the organizations—operational and research—and provided context on the use of NASA data in EPA decision making that NASA was seeking. Third, the researchers worked extensively with the Cooperative Institute for Meteorological Satellite Studies (CIMSS), which is a joint NOAA-NASA institute located at the University of Wisconsin. CIMSS provided the ability to span the research and operational domains of NASA and NOAA in collecting and combining data. CIMSS served as an additional bridge between organizations.

APPENDIXES 51 4. The decision-support system As the NASA Earth Science Applications Program does not “develop” decision support systems, the Program partners with federal agencies and national organizations that own, operate, or develop decision support systems, seeking to extend Earth science results. The EPA AirNOW program was developed as an end-to-end system, based on information it gathered through the EPA ground networks. In this project, researchers and engineers worked to provide new sources of information that EPA (and eventually NOAA) could use in supporting their air quality forecasting activities. In other words, EPA & NASA extended the one “end” to an already existing end-to-end system. The discrete parts in providing the prototype product to the forecasters involved: satellite measurements (NASA and NOAA), ground measurements, air trajectory modeling, data fusion techniques, visualization techniques, algorithms, direct broadcast stations, human analysis, and websites. Regarding changes in decision-making as a result of the prototype project, forecasters reported improved ability to estimate PM2.5 transport into their forecast area; use of the tools to identify the frequency and extent of particle pollution events; improved use for more accurate emissions inventories and trending analyses; and improved abilities to fill previously unmet requirements for forecasting PM2.5. In addition, EPA integrated the data products into four regional U.S. EPA PM forecasting workshops, and EPA is supporting the transition to operational use and the production of the forecast tool products throughout the year. The forecast tool also supports other decisions within EPA’s mission. In particular, EPA used a series of archived data (“case studies”) to evaluate the 2003 Transport Rule. EPA funded a technical support document that qualitatively interpreted the archived data and related these results to other analyses (models, observations). NASA will continue to examine and explore other products that might support EPA-NOAA relative to air quality forecasting. For example, NASA may examine whether specialized products can be generated for urban areas or whether higher-resolution products can be generated from the MODIS data to support EPA-NOAA and the forecasters. 5. Learning orientation The initial milestone of the project was to assess if there was value in pursuing the MODIS data relative to EPA’s PM activities. Thus, this binary assessment (i.e., continue or not) provided a “stage-and-gate” aspect to the project. Following this assessment (positive/ promising in this case), the project entered a “verification/validation” phase in which the NASA, EPA, NOAA, CIMSS, and others worked out technical details and logistics to develop products and prepare the prototype. This stage involved significant technical iteration. Furthermore, the interest and response from EPA was a key factor in pursuit of this project. The support and commitment from EPA (i.e., the user organization) was critical to deciding to move forward.

52 LINKING KNOWLEDGE WITH ACTION During and after the prototype, the team developed a “benchmark” report to document the activity, to evaluate the value/benefit of the product and the Earth science data, and to provide information to support the transition to operational use. This benchmark report, while very valuable to document performance and reduce risks for the operational agency, can be very difficult to prepare. 6. Continuity and flexibility Human resource and supervisory issues within organizations can affect manager’s abilities to allow employees to work and remain at remote locations. This project faced the real possibility in this past year that the EPA employee located at NASA might be “recalled.” His ability to represent EPA’s interests was much stronger by the fact that he wore an EPA badge. Budgetary pressures largely affect 1) the number of projects a program can pursue and the 2) extent of the project through support contractors that can assist with the work. At some point though, projects experience diminishing returns and more funds are not always the solution. In the development/prototype phase, due to the inherent “learning curve” or a limited quantity of people with the required expertise, additional funds do not always allow a better or quicker product. However, additional funds do allow more room for broader evaluations or more extensive verification/validation of activities. As for building human capacity, the team has worked with EPA to extend the tools and products into forecasting workshops that EPA offers to the air quality forecasters. 7. Other insights? The presence of an EPA employee permanently located at NASA-Langley has been extremely valuable to this process of knowledge/technology transfer. The value largely focused on his ability to communicate EPA’s interests and priorities, to provide contacts within EPA, and to “short-circuit” bureaucracies. 8. Other issues? In the cooperation between research and operational agencies, there may be differences in the types of approaches to planning and changes. For example, NASA’s approach is to solicit a wide range of projects through six Earth science research focus areas (www.earth.nasa.gov) to continue to increase our understanding of Earth system science. Operational agencies, including EPA or NOAA, may adjust activities to immediate priorities, including the data and analysis they need. Partnerships between NASA and these agencies require flexibility in projects and in accountability. There is value in NASA coordinating with partner agencies on solicitations for Earth Science Applications in order to optimize the value in meeting national priorities. A challenge for activities focused on link knowledge to action is to establish a balance between longer-term projects (including innovative applications using evolving research capacity) and shorter-term projects (that target on the specific needs to serve a particular project). Program plans, project plans, and accountability measures can reflect the dual nature of these activities. Performance measures may need flexibility to adjust to immediate concerns while making progress toward longer-term goals.

APPENDIXES 53 9. Contact information Doreen Neil NASA-Langley 757-864-8171 Doreen.O.Neil@nasa.gov Jim Szykman EPA Office of Research & Development Located at NASA Langley 757-864-2709 j.j.szykman@larc.nasa.gov 10. Representative publications/ products www.epa.gov/asmdnerl/models3 Forecast Tool operational website: [http://idea.ssec.wisc.edu/] THEME II: TECHNOLOGY CO-DEVELOPMENT Earth Science Applications Program Ron Birk NASA 1. Problem definition The problem to be solved by the Earth Science Applications program is to systematically extend the results of research and development of aerospace science and technology to benefit society. The program uses a systems approach to address specific applications of national priority and partners with federal agencies and national organizations to collaborate on integrating observations and predictions into decision support tools. The program focuses on the nexus between national and international priorities for policy and management in 12 applications of national priority (on the demand side) and the research results of scientists working on 6 focus areas using the 2500 products enabled by 17 Earth observatories carrying over 80 sensors and the forecast and prediction capacity of 24 Earth system models in the Earth System Model Framework (on the supply side). The overarching national and international context for the program is based on the U.S. Administration’s and congressional emphasis on the value of using Earth observations and Earth science knowledge to enable and facilitate decision support systems in the public and private sector. Key domestic and international programs are focused on the application of Earth science and its attendant observations and predictions for weather, climate, natural hazards, and other Earth processes (see Table A-1). Representative priorities and their respective committees include:

54 LINKING KNOWLEDGE WITH ACTION • At the June 1-3, 2003 Summit in Evian, the G8 established the top three priorities for science and technology to be energy, agriculture and Earth observations. • On July 25, 2003, the Climate Change Science Program Office released the strategic plan [www.climatescience.gov] for U.S. climate change research focusing on key areas of scientific uncertainty and identifying priority areas for research and development. The plan promotes a vision focused on the effective use of scientific knowledge in policy and management decisions, and continual evaluation of management strategies and choices. This strategy is aligned with the National Academy of Sciences’ recommendations presented in the June 2001 Academy report, entitled Climate Change Science: An Analysis of Some Key Questions [http://newton.nap.edu/html/climatechange/]. An objective of this plan is to develop research and data products that will facilitate the use of scientific knowledge to support policy and management decisions. • On July 31, 2003 the U.S. hosted the Earth Observation Summit in Washington D.C [www.earthobservationsummit.gov] to establish a declaration for a 10-year plan for Earth Observations Systems to serve society. The 10-year implementation plan was chartered by the Group on Earth Observations (GEO) and coordinates international and national inputs for global observations. • The November 27-28, 2003 United Nations Framework Convention on Climate Change [www.unfccc.int] establishes the importance of Earth observations and predictions for addressing societal impacts of climate change. TABLE A-1 Domestic and international committees as related to the NASA Earth Science Enterprise. Domestic International Climate Change Science Program (CCSP) Climate Change Technology Intergovernmental Panel on Climate Change Program (CCTP) Climate Change (IPCC) U.S. Weather Research World Meteorological Weather Program (USWRP) Organization (WMO) Committee on Environment and Natural Resources (CENR) International Strategy for Natural Hazards Subcommittee on Natural Disaster Disaster Reduction (ISDR) Reduction (SDNR) World Summit on Sustainable Sustainability NAS Roundtable on Sustainability Development (WSSD) Earth CENR Interagency Working Group on Observation Group on Earth Observations Earth Observations (IWGEO) Systems

APPENDIXES 55 The NASA's Earth Science Applications theme is driven by a mission “to understand and protect our home planet” through the use of the results of NASA research and development of aerospace science and technology to serve the citizens of our society. The goal is to extend the societal and economic benefits of NASA research in Earth science, information, and technology. 2. Program management The program has been developed in project mode. It has specific, measurable goals and targets for each of the 12 applications of national priority. These performance goals are captured in program element plans, and in the agency’s annual performance goals that are reported through the Integrated Performance and Budget Document (accessible at www.earth.nasa.gov/eseapps). The Earth Science Applications theme of NASA, conducted within the Earth Science Enterprise (ESE), benchmarks practical uses of NASA-sponsored observations from Earth observation systems and predictions from Earth science models. NASA implements projects that carry forth this mission through partnerships with public, private, and academic organizations. These partnerships focus on innovative approaches for using Earth science information to provide decision support that can be adapted in applications worldwide. The ESE program focuses on applications of national priority to expand and accelerate the use of knowledge, science, and technologies resulting from the ESE goal of improving predictions in the areas of weather, climate, and natural hazards. The approach is to enable the assimilation of Earth science model and remote sensing mission outputs to serve as inputs to decision support tools in integrated system solutions. FIGURE A-1 NASA’s applications program approach to integrated systems solutions architecture

56 LINKING KNOWLEDGE WITH ACTION The outcomes are manifest in enhanced decision support and the impacts are projected to be manifest in significant socio-economic benefits for each of the national applications. NASA ESE has developed discrete configurations of integrated system solutions for each of the twelve (12) national applications with partner federal agencies and national organizations that can be served by the results of NASA aerospace research and development of science and technologies (see two examples in Figures A-3 and A-4). 3. Program organization The Earth Science Applications program provides a boundary spanning function. The organization provides a “bridge” between the research and development programs of the NASA Earth Science Enterprise and the decision support functions (and programs) of partnering federal agencies. In the process of benchmarking beneficial uses and applications for Earth science measurements and technology, the Earth Science Applications program is enabling significant scientific and technological returns on the federal investment. Activities are underway in each of the twelve applications of national priority. For instance, in the area of community preparedness for disaster management, NASA is working with NOAA to integrate innovative scientific knowledge and technologies to improve warnings and predictions of hurricanes, tornadoes, and other severe weather events. The resulting solutions enable more cost effective damage mitigation, emergency preparation, and contribute to emergency management functions provided by the Federal Emergency Management Agency (FEMA). For agricultural efficiency, NASA is working with the US Department of Agriculture to benchmark the use of predictions of El Nino and La Nina events for management of our nation's farmlands (see Figure A-4). Integrated system solutions used to monitor and assess the health and condition of crops and forests around the globe are being improved. In aviation, measurements and predictions from our weather and environmental satellites are being integrated with other traditional aviation weather information. These are just a few examples of how NASA works through partnerships to utilize science and technology to serve society. The set of applications, partners, and decision support systems includes:

APPENDIXES 57 TABLE A-2 NASA Earth Science Applications program’s applications, partners, and decision support tools. National Partner Agencies Decision Support Systems Application Agricultural CADRE - Crop Assessment Data Retrieval & USDA, NOAA Efficiency Evaluation (USDA) CMAQ - Community Multi-scale Air Quality Air Quality EPA, NOAA, USDA Modeling System AQI - Air Quality Index NAS_AWRP - National Air Space - Aviation Aviation DOT/FAA, NOAA Weather Research Program Carbon CQUEST-EA92-1605b - Energy Act of 1992, USDA, DOE, NOAA Management Section 1605b HAB - Harmful Algal Bloom Bulletin / Mapping Coastal NOAA, EPA, NRL System Management CREWS - Coral Reef Early Warning System Disaster DHS/FEMA, NOAA, HAZUS-MH - Hazards US - Multi Hazards Management USGS, USFS Ecological USAID, NOAA, SERVIR - Regional Visualization & Monitoring Forecasting NPS, CCAD, USGS System Energy DOE, UNEP, NOAA, RETScreen - Energy Diversification Research Management NRC Laboratory (CEDRL) Homeland DHS, USGS, NOAA, IOF - Integrated Operations Facility Security NIMA, DoD USGS, USDA, Invasive Species ISFS - Invasive Species Forecasting System NOAA PSS - Plague Surveillance System EPHTN - Environmental Public Health Tracking NIH, CDC, DoD, Public Health Network Program Research EPA MMS - Malaria Monitoring & Surveillance RSVP – Rapid Syndrome Validation Project RiverWARE - Bureau of Reclamation Decision Support Tool Water EPA, USDA, USGS, AWARDS - Agricultural Water Resources & Management BoR Decision Support Tool BASINS - Better Assessment Science Integrating Point & Non-point Source

58 LINKING KNOWLEDGE WITH ACTION 4. The decision-support system The systems approach used by the NASA Earth Science Applications program is based on an architecture (see Figure A-1) that includes the discrete systems components of observatories, Earth system models, and decision support systems. There are 80 discrete sensors on 17 discrete Earth observation satellites that provide 2500 discrete science data products and 24 discrete models that are available to be configured into integrated system solutions with a set of 18 discrete decision support systems. The challenge is to establish a common architecture and a common approach for the coordination of systematically integrating system solutions amongst a cadre of thousands of individuals at 10 federal agencies, 10 NASA centers, and hundreds of universities and other science and research organizations throughout the country. A few examples of successful implementation of the program have resulted in changes to the way that: • EPA conducts and delivers Air Quality Index and Air Quality Forecasts • USDA conducts and delivers Global Crop Production • CCAD (Central America) conducts and delivers ecosystem assessments • NOAA conducts and delivers hurricane forecasts • FAA conducts and delivers warnings to the aviation community regarding volcanic ash • the Navy conducts and delivers oceanic diver visibility observations 5. Learning orientation The program has a mandate to expand and accelerate the realization of societal and economic benefits from Earth science, information, and technology. The strategy for the program (accessible at www.earth.nasa.gov/visions) was developed in conjunction with the Office of Science and Technology and reviewed by the National Academy of Sciences. The initial risks were establishing: 1. Meaningful and documented partnerships 2. “Zeroth order” versions of integrated system solution configurations 3. Processes (guidelines, handbooks) for conducting systems engineering functions of evaluation, verification and validation, and benchmarking of the integrated system solutions Current risks include: 1. Partner requirements for Earth observations and predictions as inputs to their decision support systems 2. Systematic transition from research to operations 3. Continuity of observations 4. Stakeholder direction(s) 5. Assessing/accommodating uncertainties in observations and forecasts

APPENDIXES 59 6. Continuity and flexibility Budget stability is an important aspect of multi-year efforts to systematically evaluate, verify, validate, and benchmark integrated systems into solutions. The NASA Earth Science Applications program is designed to enable the assimilation of products resulting from the NASA Earth System Science theme (approximately $1.5B per year) into decision support tools funded by partnering agencies and organizations. An innovative approach to human capacity building includes the DEVELOP project (details are accessible at http://develop.larc.nasa.gov). 7. Other insights? It appears to be valuable to recognize the importance of the following considerations: 1. Focus on applications that can serve communities throughout the nation and the world 2. Focus on discrete solutions with specific purposes and constituencies 3. Employ a systems approach 4. Characterize impacts/limitations of uncertainties in the context of decision processes 8. Other issues? Explore the impacts of national and international interoperability and standards on information products, handling techniques, and protocols for assimilating observation and prediction products and processes into decision support tools. Contact information Ronald J. Birk Director, Earth Science Applications Program NASA Office of Earth Science 202.358.1701 rbirk@hq.nasa.gov Representative publications/products Website: [www.earth.nasa.gov/eseapps] Program Strategy: Earth Science Applications Strategy: 2002 - 2012 Program Plan: Earth Science Applications Plan Overview: NASA Earth Observations for Society These documents are available at: [www.earth.nasa.gov/visions] and [http:/webserv.gsfc.nasa.gov/images/aiwg.html] Houston Advanced Research Center (HARC): Development of clean air policy in Houston and Dallas Todd Mitchell

60 LINKING KNOWLEDGE WITH ACTION 1. Problem definition What is the problem to be solved? Public policy leaders in the Houston Galveston Area (HGA), particularly mayors and county judges, confront two problems related to air quality. In the short-term, they need to determine what strategies will allow the HGA to come into compliance with federal clean air standards under the State Implementation Plan (SIP) agreement between the state and the EPA. In the long- term, strategies to achieve myriad air quality goals, not all of which are related to the SIP, must be put into place. The problem confronting decision makers that the program described below addresses is the lack of science on which to base policy. Research is required to improve air quality models, model input parameters, and to understand ozone formation in a region of unique geography, climate, industry, and transportation. How did the program provide for a user-driven dialogue between scientists and decision makers to shape problem definition? A nonprofit organization, the Texas Environmental Research Consortium (TERC), was formed to provide the scientific and technical knowledge necessary to craft a viable SIP. TERC is led by a “decision maker” board of directors, where stakeholders of varied and sometimes opposing positions can address science and policy issues. These stakeholders include representatives from state, county and city governments and academic, business, environmental and health organizations. Research management functions were delegated to a non-profit research management organization, the Houston Advanced Research Center (HARC), and thus kept separate from policy decision making. HARC created a Science Advisory Committee consisting of members from the scientific community to guide the development of the Strategic Research Plan, assess the credentials of research subcontract teams, and guide development of the request for proposals (RFPs). It then circulated the Strategic Research Plan widely for stakeholder comment. HARC also convened the Consortium Advisory Council to advise the TERC board in business, policy, management and research priority issues. How did the ultimate problem definition differ from initial formulation by scientists and decision makers? For air scientists, the problem is defined as taking known inputs (mobile source emissions, point source emissions, biogenic emissions, climate, wind, etc.) and running airshed models that (a) accurately predict observed phenomena, and (b) predict the effects of possible control measures. Scientists view Houston as an intriguing brew of point and mobile source emissions mixing in a hot, humid climate to produce rapidly forming ozone, rivaling any region in the country in severity. As a focus of large scale data collection field programs, Houston is a data-rich region for research into the physical process of ozone formation. For decision makers, the problem is defined as developing strategies that, when implemented, satisfy the goals defined in the SIP agreement while causing the least offense to business, health, and environmental communities. These differing problem definitions converge in that both scientists and policy makers want the scientific models to be able to simulate reality so that they can be used to test regulatory decisions. The original focus on models changed when it became apparent that there were problems with the model inputs such as emission inventories. Once this greater problem was assessed, the program

APPENDIXES 61 was redefined, with equal weight given to improving models and model inputs. Additionally both scientists and decision makers are becoming aware of the need to address other air toxins and a multidisciplinary air quality study planned for this summer will also collect data on particulates and 1,3 butadiene. 2. Program management Was your program developed in a "project" mode? Using sound science to devise strategies to meet SIP requirements is a process that is dependent upon the successful completion of discrete research projects and administrative tasks. Did it have specific, measurable goals and targets? Although the need for good science to underpin policy decisions is ongoing, there are milestones to be achieved that function as discrete projects and have measurable goals and targets. Some of these were related to administrative tasks, such as disbursing funds, and others were related to research tasks. Initially HARC was charged with disbursing a defined quantity of money – approximately $4 million—and given a defined target date—the spring of 2004, when the state would modify its SIP in a Midcourse Review process with EPA. HARC had to achieve both financial and scientific goals. Financial goals included (a) spend all the available money in a two-year period with (b) a minimum of overhead cost. HARC’s scientific goal was to deliver sound science to decision makers before the Midcourse Review by providing a structure to manage the necessary research through the allocation of contracts. These measurable goals were met in the first phase of the project. Less specific at the program’s inception were (a) the topics of the scientific inquiry, and (b) the measurable impact of the science on policy decisions. The project team is now looking back and attempting to assess what portion of the scientific findings have actually resulted in policy decisions, or in improvements in the science community’s capacity to model the airshed. To what extent and in what ways was goal and target definition driven by scientists or decision makers, or both? The ultimate goal is driven by decision makers because research needs are identified and prioritized in relation to regulatory needs. There would be no program if the Houston region were already in compliance with EPA air quality standards. That said, decision makers and scientists also had independently derived goals. Decision makers were interested in designing effective and acceptable regulations and scientists were interested in filling knowledge gaps and improving the ability to model systems related to air quality. In two cases, decision makers requested that particular research be conducted, but generally speaking the scientists have shaped the definition of needed advancements. Despite decision makers’ initial orientation toward science for SIP compliance, their position has expanded somewhat through their communication with scientists. Many now appreciate the need for more information about air quality than only that which is required to meet near term air quality standards. In the matter of defining research priorities, To what extent and in what ways were program leaders held accountable for achieving those goals and targets? Program leaders have been held extremely accountable for achieving the measurable goals described above. HARC meets with the Consortium Advisory Council monthly and with the

62 LINKING KNOWLEDGE WITH ACTION TERC board of directors quarterly. Research progress reports and annual program reports are delivered systematically. HARC, as the research management organization, is evaluated on an annual basis through a survey instrument sent to a cross section of stakeholders, and continues to serve at the pleasure of the TERC board. Delivering research results on time and on budget is an important measurement of the HARC’s success. HARC is also held accountable for good research results. It is accountable to the scientific community in formulating research questions and avoiding bias and influence from stakeholders in the prioritization of projects, choice of research teams and allocation of funds. 3. Program organization Did your program involve a boundary spanning function or organization? If so, where and how was the boundary organization or function created? Yes. HARC, as research manager, was hired in this process to span the multiple boundaries between the science community and TERC, the scientific community (researchers engaged in TERC funded research) and TCEQ, and between TCEQ and TERC and is accountable to all of these groups. HARC has existed since 1982 and has been involved in a number of these “bridging” processes (see representation in Figure A-2). What did it do? HARC facilitates communication between scientists and regulatory agencies, scientists and stakeholder decision makers, and scientists of different disciplines using formal structures such as the Science Advisory Committee and informal communication through e-mail and phone calls. The Science Advisory Committee assesses the “state of the science” as applied to the specific project region and makes recommendations regarding research to be conducted. In conjunction with the Science Advisory Committee, HARC gives guidance to the TERC board about appropriate prioritization of research topics. HARC works with the TCEQ Science Coordinating Committee to identify projects of importance to the state. There is typically a tension between funding projects developed through TCEQ’s Science Coordination Committee and TERC’s own research management process. Because TCEQ staff expect as many projects from the TCEQ internal list as possible to be funded, HARC has occasionally had to defend the independent selection of projects. For example, one project, which deployed a novel technique for constraining a model’s behavior with observations, was opposed by the agency as “too ambitious,” although in the end 3 scientific papers were submitted to a peer-reviewed journal based on the project. The PI of the project later remarked that it was one of their most productive projects. HARC also collects research results into synthesis documents for translation to non-scientist decision makers. During this process, it is careful to give proper credit to the research teams that actually perform the key research. To what extent was it accountable to both users and producers for achieving its goals? HARC is accountable to the TERC board for management functions such as financial stewardship, project management, reporting, etc. It is also responsible to maintaining the nonpartisan nature of process, thus insuring its credibility to all stakeholders. Because the TERC

APPENDIXES 63 board and the Advisory Council are composed of a variety of stakeholders, HARC is not accountable to a single partisan faction, but rather to multi-partisan groups. HARC is also accountable to the scientific community, but in a different context. Because of the highly politicized nature of the air quality conflict in the region, suspicions of “partisan science” are always just under the surface. To earn the confidence and respect of the scientific community, HARC sticks to the science management process and avoids staking a pro-business, pro-health, or pro-environment position. HARC also has an active internal culture that emphasizes sound science and resistance to any partisan pressure. The accountability to the science community is formal in the form of HARC’s annual evaluation survey, and informal in the form of ongoing interaction. The science community has thus far engaged in the process without being labeled as partisan for a particular faction. 4. The decision-support system To what extent is the decision support system developed by your program an end-to-end system? The decision support system is almost, but not quite, end to end. The design of the process allows for delivery of research results to a variety of stakeholders on the TERC board (business, health, environment, and local government). Each board member can do with the information what he or she wishes. The process also delivers research results to the state’s environmental commission. The paid staff of the commission considers the research results to be timely and significant, and is free to incorporate (or not incorporate) the information into their models and SIP negotiations. Finally, the information is posted on a Web site for the public. In this regard, the delivery of information to “affected parties,” to the public, and to the leading regulatory agency represents an end-to-end process. On the other hand, Texas (as most states) is a haven for backroom politics, and in this manner the process is end-to-end but not necessarily equitable. The process was designed so that information flows to the TERC board and to the state environmental commission. As an organization, TERC does not lobby the state legislature, the governor’s office, or the lieutenant governor’s office as a unified entity, but presumes that each board member will use the information as he or she sees fit. This design was intended to minimize conflict within TERC and allow the process to focus only on improving the quality of information, while allowing board members to have better information in their traditional role outside of TERC. The problem arises in that certain board members (e.g., local government officials, business leaders) have more ability to penetrate into the “back room” than other board members (health community, environmental community). The result is a process that is designed to be fair but has some challenges in how political access varies across the board. This disparity is probably the biggest criticism of the process among some in the air debate. What are its discrete elements? • Request for Qualifications (RFQs) to qualify research teams • Strategic Research Plan (with public commentary) • Development of discrete projects, selection of the appropriate research team, (if none of the previously qualified research teams has the technical expertise required, initiate a RFP) • Research on a subcontract basis to HARC

64 LINKING KNOWLEDGE WITH ACTION • Delivery of discrete (single project) research results to TERC board, the public, and the state environmental commission • Synthesis of (multiple) research results and delivery to TERC board, the public, and state environmental commission • Incorporation of model input improvements (e.g., better equipment and emissions inventories) into airshed models used for demonstrating compliance with federal regulations • Recommendations to EPA regarding improvements to the airshed models themselves, with expectation that EPA will approve use of improved models in the region Which were the hardest elements to put in place? Why? It was all hard, but the most perplexing was determining how to organize TERC. Getting stakeholder buy-in was complicated by a long history of distrust among prospective board members. Individual leadership resulted in important breakthroughs. Separating TERC, with its diverse stakeholders largely based in government, from HARC -the research manager, was important for accountability and credibility. However, this created many legal complications in structuring a process in which funds could flow and terms and conditions of the relationship could be defined. The rules and regulations concerning handling federal and state funds required caution and expertise. Setting up the business processes for such a program is a challenging undertaking. What changes in research, decision-making, or both have occurred as a result of the system? Collaboration between the state environmental commission and TERC, facilitated by HARC, led to the identification and execution of several projects that had not been deemed as priorities before the communication process was initiated. For example more emphasis was placed on transportation and exposure research than was originally planned. Results of numerous studies have improved model inputs in both the Houston and Dallas regions. Results of one major study provided convincing evidence that was used to draw the boundaries of the Dallas / Ft. Worth nonattainment region. 5. Learning orientation Did your program have an expressly experimental orientation? The process was not designed to be experimental, but it has responded as issues arose and in this way has adopted a learning posture. For example, HARC initially modeled proposal management after the National Science Foundation, using an open request for proposals and peer review. This type of proposal process takes time and the need to provide research in time to meet policy deadlines made it difficult to follow the NSF model. HARC modified the proposal process to one based on a request for qualifications (RFQ) from research teams. Currently, after the RFQ is peer reviewed, research teams are given subcontracts to complete specified research projects. The initial research focus was to gain information crucial to improving emissions inventories, air pollutant monitoring and computer modeling of atmospheric conditions. Changes in research priorities have been made in response to a new ozone standard. Additionally, the realization that there are knowledge gaps about other air quality parameters, such as fine particulates and toxics, which may soon face regulation, has led to broadening the

APPENDIXES 65 research scope in general. The program has also broadened its scope geographically. This geographic expansion is due to a new understanding of emission transport. How did it identify which risks to take? How did it identify success and failure? How did it engage outside evaluators to help it reflect on its own experience? The risks associated with research are that the research directions and results are not valid or valuable. These risks were minimized by using the Science Advisory Committee, a diverse group of renowned scientists, to evaluate and direct the selection and prioritization of research projects. How did it identify success and failure? Successes were identified by the numerous references to TERC research projects which supported policy decisions in the State Implementation Plan. What are the most important lessons you have learned regarding pitfalls to be avoided, or approaches to be followed in the future? If HARC does its job poorly in this program, it will offend all stakeholders, with significant consequences for the organization’s credibility and future prospects. If it does its job well, it will only offend many of the stakeholders most of the time. In Texas, the management of air quality has multi-billion dollar consequences. Examples: Texas will lose $2-3 billion per year in highway funds if it fails to comply with SIP agreements by a specified future date; the cost to Houston-Galveston area related to annual health and productivity impacts from air pollution is approximately $3 billion; and the collective cost for the Houston region petrochemical complex to upgrade facilities to meet SIP commitments is from $7-13 billion. Even in a well designed process with stakeholders showing support and good will, when findings fail to support a particular position, the economic, environmental, and health consequences can be so large that pressures and tensions rise. Despite the rewards of contributing to a challenging science/policy process and being paid to do so, HARC understands the significant risks associated with the project. The stakeholders did two things right: (a) They separated TERC, the policy organization, from HARC, the research management organization, allowing the research manager to focus on the science process without getting consumed by the political and economic tug-of-war; and (b) they “agreed to disagree” at the policy level so that all parties are provided scientific information but are not required to come to consensus within the room, allowing the various factions to be united in seeking data and analysis while using the analysis to support their causes outside the room. 6. Continuity and flexibility How do budgetary requirements and/or human resource pressures influence your program? The program was initially funded by a Coastal Impact Assistance Program grant, a one time appropriation. The State of Texas has since appropriated funds for new research and technology to support air quality management. This funding is ongoing but the amount changes yearly depending on the sale of automobiles, making it difficult to schedule disbursement of funds for research projects. Additionally, HARC is limited to charging no more than 20% of total program funds for management of the program. This goal is somewhat arbitrary and only can be met if the program funds are sufficient. (In other words, if the program’s funds drop by one-half but the work load drops only by one-quarter, then service will decline.) This math has created stress and some tensions in the program administration.

66 LINKING KNOWLEDGE WITH ACTION What, if any, collaborative funding mechanisms have you developed to ensure continuity and relevance to users' needs? In return for this process of performing some research important to the state environmental commission, TCEQ provided additional support. TERC has received new foundation grants and HARC has solicited and won program awards from foundations and EPA in parallel research themes. Additionally, HARC is able to leverage its current position to qualify for external awards that benefit the entire process. If applicable, how do you maintain public funding, or incorporate private funding, for the provision of a partially private good? The goal is in fact a public good, so the justification to foundations and public agencies is relatively straightforward. What, if any, innovative approaches have you developed for enhancing human capacity in your program area (e.g. building curricula or providing incentives to reward interdisciplinary activities)? The Science Advisory Committee provides a venue for scientists of different disciplines and in different institutions to interact and share information. Additionally, HARC manages research teams from across the nation in large, intersecting studies. An example is the upcoming summer study of Houston and Dallas which requires communication and joint planning among several research teams. The program itself provided the funds for HARC to hire two air scientists, and the complex demands of air science and policy provide a real world “capacity building” training program. Apart from normal workforce development (organizational training, travel to conferences, etc.) HARC has no unique programs for building additional capacity within this air research program. Within the organization, the air science team has shared information with researchers from other disciplines in informal seminars. 7. Other insights? What other insights or conclusions emerge from your experience about the factors responsible for success and failure in activities designed to link knowledge to action? • A non-partisan process can involve strongly partisan membership • Separate the policy function from the management function • Design the process for the science-to-policy handoff • Adjust your mission, strategy, timetables and deliverables to match the problem 8. Other issues? It is important to design effective mechanisms to communicate scientific findings to laymen who are in decision making positions, and to the general public. The process described in this case study places emphasis on this aspect, and our sense is that this feature of the program’s design has contributed to the success. Efforts include written documents prepared for distinct audiences (e.g., public, media, legislators), as well as Web sites, seminars, speaker presentations, etc.

APPENDIXES 67 9. Contact information Jim Lester, PhD. Director, Environment Group (policy adviser for program described in this case study) jlester@harc.edu More about Jim: [http://www.harc.edu/harc/Content/About/Directory/ShowUser.aspx/37] Eduardo “Jay” Olaguer Sr. Research Scientist (program manager for program described in this case study) eolaguer@harc.edu More about Jay: [http://www.harc.edu/harc/Content/About/Directory/ShowUser.aspx/527] David Hitchcock Director, Sustainable transportation Programs (key architect for program described in this case study) dhitchcock@harc.edu More about David: [http://www.harc.edu/harc/Content/About/Directory/ShowUser.aspx/32] Todd Mitchell, President, HARC tmitchell@harc.edu More about Todd: [http://www.harc.edu/harc/Content/About/Directory/ShowUser.aspx/43] 10. Representative publications/products List key publications or products that would help us to understand the program you have described, including web sites. Research Management Organization: (Houston Advanced Research Center) [www.harc.edu] Regional Air Quality Program [http://www.harc.edu/harc/Projects/AirQuality/] 2003 Strategic Research Plan for Texas Environmental Research Consortium (TERC is the “Policy Organization” in this case study). Available at: [http://www.harc.edu/harc/Projects/AirQuality/Projects/Status/Files/TERCStrategicPlan2 003.pdf]

68 LINKING KNOWLEDGE WITH ACTION FIGURE A-2 The positioning of HARC as a boundary organization. Engineers Without Borders-USA: building capacity in underdeveloped communities while developing internationally-responsible engineering students and professionals Bill Wallace Engineers Without Borders 1. Problem definition The problems associated with poverty are well known, yet there appears to be a substantial disconnect between the underdeveloped communities of the world desperate for assistance and the engineering knowledge and resources needed to address their problems. These communities lack the facilities and infrastructure to meet even the most basic needs: clean water, sanitation, health, food, and shelter. EWB-USA believes that these problems can be readily addressed by working directly with these communities: identifying the most important problems and applying the appropriate technologies and practices. In contrast, the focus of national assistance programs seems to be somewhat detached from these everyday realities. Addressing these problems requires the application of engineering knowledge and resources in a developing world context. Solutions must be practical: implemented and maintained using available skills and resources, and consistent with local culture and customs. These facets of engineering are not taught in schools nor are they acquired in an engineer’s normal career experience. Furthermore, the practical “low tech—high content” technologies needed to solve these problems do not receive much attention.

APPENDIXES 69 2. Program management EWB-USA’s program started with a single project to provide a reliable water supply to a small remote village in Belize. Our primary objective was to design and deliver a water system that met the expressed needs of the people: an adequate supply of clean water through a system that was easy to maintain with local skills and locally available materials. A second objective was to educate engineering students in the design, fabrication and operation of a water supply system, working closely with the host community. EWB-USA professional engineer-mentors provided project oversight. Following the success in Belize, EWB-USA was created to extend this model to other developing communities. In each project, EWB-USA students and mentors work closely with the community leaders to understand community needs and set project objectives. A report is prepared after every project, detailing the results and lessons learned. This model has been followed for all subsequent EWB-USA projects. 3. Program organization EWB-USA’s program starts with a dialogue with the user, or in our case, the host community. We start by identifying the host community’s critical needs as well as the practical issues and limitations. We then work directly with the community to identify and describe alternative solutions from which the community selects the one that most suits their needs. After the solution is designed, the EWB-USA project team and the people of the community work together to build the system. Before leaving, the EWB-USA team makes sure that the people of the community know how to operate and maintain the system. Follow-up visits enable EWB-USA to learn how the system performed and work with the community to make any necessary system alterations. 4. The decision-support system EWB-USA’s program is set up as an end-to-end, integrated system. The program can be thought of as an aggregated set of projects, each of which is in some stage of development, implementation, or completion. To the extent practical, each new project incorporates the knowledge and lessons learned from previous projects. Over our four years of operation, we learned about the importance of pre-planning, to expect the unexpected, the likelihood of in-the- field design changes, and the knowledge and resourcefulness of the people in the host communities. 5. Learning orientation Although the EWB-USA program is not expressly experimental, it is a program that, to our knowledge, has not been done before: students, academics and professional engineer–mentors working with people in host communities planning, designing, and implementing facilities and infrastructure projects in underdeveloped nations. In conducting these projects we are faced with a number of issues and unknowns. Some examples include the ebb and flow of local and regional politics and unrest, health and safety for the project participants, and unarticulated project issues and constraints. To address these matters we have established detailed procedures for project screening, site and community assessment, project planning, project execution, project reports, and post project reviews for lessons learned. To date, the most important lessons learned were:

70 LINKING KNOWLEDGE WITH ACTION • Team building: the need for chartering the project team, making sure that the participants understand their roles and responsibilities, and how to conduct themselves as they work in an unfamiliar culture. • Health and safety: making sure that the project participants understand and are prepared to deal with health and safety issues and emergencies. • Expecting the unexpected: activities are carried out differently in other parts of the world, particularly in the underdeveloped countries. What is clear to one person may not be clear to someone else. Schedules may not be met. Materials may not be available at the time and place requested. 6. Continuity and flexibility EWB-USA is now in the throes of developing a cadre of staff and reliable sources of funding for the overall management of its programs. To date, this work has been accomplished mostly through volunteer staff. However having delivered over 50 projects in 23 countries, the work is becoming overwhelming. To remedy this situation, we are now working with professional societies, engineering trade organizations, companies, foundations, and public sector agencies to improve our sources of funding. We are more successful in securing project funding, since the donors can see direct benefit to a underdeveloped community, and the amount of monies needed to design and implement an EWB-USA project are very low, averaging $15,000. Based on what we’ve learned, we are now revisiting the EWB-USA business model, assessing the value we bring to our members and stakeholders and pricing our services accordingly. For example, in our four years of operation, we have built a unique and extensive knowledge base on how to successfully plan and run a community assistance project in underdeveloped countries. We also are becoming part of the engineering curriculum for many universities, developing students who are sought after by employers in part because of their EWB-USA project experience. 7. Other insights? • Saving the world one community at a time. EWB-USA’s approach has been both celebrated and criticized by various groups. The critics note that our approach, while well meaning, cannot possibly have much of an impact on the world’s problems. “Wouldn’t our efforts and resources be better spent on broader capacity-building activities,” they ask? Our reply is no. EWB-USA project not only improve the lives of people in developing communities, they also build relationships among the project participants, as well as “hands-on” learning for the students. Furthermore, each project experience creates positive stories about the project experience and the appreciation of the people in the host community for the immediate benefits obtained. These stories are told time and time again to other communities and to other stakeholders, which, in turn, helps build the interest for more EWB-USA projects. • Two plus two equals ten. Frequently EWB-USA projects provide more than the immediate benefits in terms of clean water, sanitation, etc. Many are transformational, changing substantially the lives of people in the host community. For example, the project in Belize not only provided a reliable supply of fresh water, it eliminated the need for the young girls of the village to spend their days carrying water from the river to the village. This enabled them to go to school and help them break out of the cycle of poverty.

APPENDIXES 71 • Improving U.S. competitiveness. Today’s graduating engineers are facing an increasingly competitive world. The market forces of globalization have created a world where anyone can buy or sell anything to anybody all the time. Over the past decade a significant portion of U.S. manufacturing has been outsourced to low-wage countries, closely followed by some basic knowledge work services such as software programming, travel services, and help desk support. However, there is no reason to believe that outsourcing will stop there. Columnist Thomas Friedman noted recently that the state of information technology has reached the point that most all knowledge work, including engineering, can be disassembled and assigned anywhere in the world based on the best skills at the lowest cost. In this new world of competition, the traditional engineering curriculum will only enable students to perform the sort of commodity engineering tasks that can be outsourced at one-quarter of the salary cost. Having an EWB-USA project experience provides an engineering student with at least some unique skills and experience. However, the U.S. and the other developed countries have some serious challenges before them. NetTel@Africa: Informing the Telecommunications Regulatory Process Jeff Cochrane USAID 1. Problem definition Africa in the mid-1990's was failing to advance into the Internet age. National regulatory authorities, were ill equipped to judge the merits of emerging technologies, economic models and legal structures that might eventually support the widespread adoption of promising new technologies. These authorities asked for assistance, which had traditionally been provided through ad hoc training workshops, usually in places like Washington or Geneva. The NetTel@Africa program, however, offered the opportunity to link academic, legal and other technical experts to national regulators through university programs within Africa itself. 2. Program management While “projects” (formally defined as sets of activities leading to clear and measurable objectives within defined timeframes) were important components of NetTel@Africa, the process of project formulation was even more critical. This project formulation took place in what might be called “alliance” mode, where the program manager assisted a diverse set of potential alliance members to articulate a common goal. Then, through a process of iterative consultations, consensus building, and workshops, the alliance was formed, with specific roles for each alliance member, and near- and intermediate-term objectives. Individual components were essentially stand-alone projects in and of themselves, but all contributed to a common goal. The overall success of the program manager was represented by the accumulation of results from the components, all contributing toward that common goal. 3. Program organization The knowledge producers were a partnership among African and US universities, as well as among African and US regulatory practitioners. The users were national regulatory authorities. The boundary-spanning organization was a non-profit center based at one of the US universities,

72 LINKING KNOWLEDGE WITH ACTION and in particular a program manager within that center. This particular program manager had substantial prior experience with old models of training regulators, recognized the shortcomings of those old models, and saw the potential of incorporating universities into a new model. The program manager was the key innovator. 4. The decision-support system The discrete elements are: 1. Identification of best practices for regulatory policy formulation and implementation, 2. Understanding the political and economic context in which African regulators operate, 3. Development of appropriate curricula and certification programs with feedback mechanisms for program enhancement, and 4. Regulators putting their acquired principles and techniques into practice. Perhaps the hardest element continues to be “2”, where the principles that seem to work fine in places like the USA are sometimes difficult to contextualize in Africa. 5. Learning orientation NetTel@Africa itself is experimental. We know of no other program like it. Technology and the associated business models are evolving rapidly. Hence, NetTel is dynamic, in that the course modules are designed to be adaptable to changing circumstances, technologies and approaches. The alliance between regulators/practitioners on the one hand and the academic experts on the other hand places a premium on examining whether the approaches to regulation are effective in achieving their public-policy purposes. In addition, good pedagogy is employed that brings experts virtually into the classrooms to discuss their current regulatory cases and challenges. Finally, the NetTel partners have identified a specific research agenda that will be implemented by the member universities. Ultimately, success will be about telecommunications regulatory authorities making better decisions, and better fulfilling their proper function as arbiters and as managers of the stage upon which telecommunications operators conduct their business. The fact that regulators themselves have embraced the program and are prepared to take advantage of it to enhance their own capacities to carry out their functions is sufficient evidence of success at this stage of the program. The most important lesson to emerge from NetTel is the importance of a commitment to collaboration, and sufficient time to implement the collaborative process. In NetTel’s case, the collaborative process required almost 18 months. 6. Continuity and flexibility The program is a partnership, with substantial resources contributed by the participating universities and regulatory bodies on both sides of the ocean, which will in turn ultimately be the operators of the certification programs. Almost all of the expertise has been donated and USAID funding alone could not have implemented the program. The unusual and potent “chemistry” of the collaborators is proving to be particularly fruitful.

APPENDIXES 73 7. Other insights? While partnerships are key, and the program could not be implemented without partnerships, the capacities of particular partners are not always sufficient. Creative program managers work with the cards they are dealt, but pick the right game to play. Contact information Dr. Maria Beebe Director of Global Networks Center to Bridge the Digital Divide Washington State University beebem@mail.wsu.edu +01 (509) 358-7947 Additional contacts: Dr. Jeffrey Cochrane, jcochrane@usaid.gov; Mr. Lane Smith, lasmith@usaid.gov. Representative publications/products Available at: [http://cbdd.wsu.edu/atc/overview4.htm] and [http://www.nettelafrica.org/] The Green Chemistry Institute John Warner The Green Chemistry Institute is an organization with a mission to promote sustainability through fundamental molecular science and technology. Green chemistry is defined as the design, development and implementation of chemical products and processes that reduce or eliminate the use and generation of hazardous substances. Therefore, through the design of next- generation material and energy sources, green chemistry seeks to eliminate a wide range of threats to man, the biosphere, and the systems that support it. The approach used embeds the desire to make these innovations in a manner that will be economically beneficial and increase quality of life. The Green Chemistry Institute was initially conceived in the mid-1990’s as an “institutionalized partnership” between academic, governmental, NGOs and industrial interests. While the institute was formally incorporated in 1997 and now is in a formal alliance with the American Chemical Society, it maintained the same emphasis on partnership types of programs and activities. By working with all interested parties, the Green Chemistry Institute has been able work both in the U.S. and in a wide range of countries having 25 national chapters in the Americas, Europe, Africa, Asia, and Oceania. There was a general recognition in the early and mid-1990’s among the nascent green chemistry community that the traditional flows of government funding were incompatible with the necessity of longer-term mission of green chemistry. While the earliest funding engagement had originated from the U.S. Environmental Protection Agency and subsequently from the National

74 LINKING KNOWLEDGE WITH ACTION Science Foundation, it was agreed that diversification was needed not only for funding purposes but also to facilitate engagement with the much broader community that needed to be involved. An initial group of individuals from industry, academia, national laboratories, research centers, and government agencies convened to identify pathways for partnership in the areas of green chemistry. The development of the Green Chemistry Institute faced several challenges in moving from concept to reality. 1. Differing institutional missions: It was recognized that all of the partners around the table were essential pieces of the puzzle. However, due to very different missions of these institutions there were often barriers to fitting these puzzle pieces together. When the Department of Energy and the Environmental Protection Agency recognize that there is large overlap yet consider collaboration dangerous in certain circumstances for statutory or policy reasons, this needs to be overcome. 2. Fundamental vs. applied: Because green chemistry deals with environmental issues facing commercialization and manufacturing, there is a definite “applied” sense to the research. However, due to the lack of a historic focus on inherent environmental implications at the molecular level, much research at the fundamental level needs to be pursued. 3. Strategic relevance: Green chemistry presents a research strategy that seeks to introduce a unique aspect of environmental protection, avoiding the use of hazardous materials at the design stage of an R&D effort. This focus brings a distinct relevance to society and links chemical research directly to goals of community and environmental improvement. The main areas of science for sustainability and green chemistry that are pursued by the Institute include: • Research • Education • Information dissemination • Awards and recognition While each of these areas has obvious overlapping audiences, they also reach particular target communities ranging from the bench scientist to the journalist to the consumer to the policy maker. In the development of these programmatic areas, there were many challenges that were cultural, institutional, financial and in some cases technical. Contact Information John C. Warner Professor of Chemistry Director, UMB Center for Green Chemistry University of Massachusetts Boston 100 Morrissey Boulevard Boston, MA 02125-3393 john.warner@umb.edu

APPENDIXES 75 617-287-6165 (Tel) 617-287-6127 (Fax) www.greenchemistry.umb.edu EPA/NSF Partnership for Technology for a Sustainable Environment Steve Lingle, EPA Bob Wellek, NSF 1. Program definition Competitive academic research on designing environmentally benign chemicals, products, and processes is central to a strategic approach to sustainability. Research in these areas is the primary mechanism for reducing or eliminating the use of toxics, materials and energy, as well as reducing or eliminating wastes and emissions at the source. Innovative and rigorous academic research, such as that funded under the Technology for a Sustainable Environment program (TSE), provides the essential foundation to real long-term progress in achieving these goals. TSE research historically has focused on: (1) green chemistry and engineering (e.g., benign solvents, catalysis, reaction engineering, etc.); (2) bioprocessing (use of biological feedstocks, biocatalysts, enzyme reactions, etc.); (3) environmentally benign manufacturing (i.e., machining, metal casting, product design) and (4) industrial ecology (i.e, life cycle analysis, materials tracking). By leveraging funds through a partnership with NSF, twice as many research projects were funded than if EPA engaged in this research program independently. A number of success stories are already emerging with the implementation of research funded through TSE leading to demonstrated environmental benefits. 2. Program management This program was explicitly designed to address the need for the federal government to fund pre- competitive research in green chemistry, green engineering, industrial ecology, and materials flow for source reduction. The Environmental Protection Agency and the National Science Foundation jointly created this program in 1995 and continues to implement as a partnership. The program is based on the release of a solicitation that offers funds for fundamental and applied research in the physical sciences and engineering that will lead to the discovery, development, and evaluation of advanced and novel environmentally benign methods for industrial processing, manufacturing, and construction. The competition addresses technological environmental issues of design, synthesis, processing, and the production, use, and ultimate disposition of products in construction and in continuous and discrete manufacturing industries. Projects must employ fundamental new approaches, and address or be relevant to current national concerns for pollution avoidance/prevention (at the source). Projects that are "on the cutting edge" or are "high-risk/high-payoff" are encouraged. Projects that show the potential to change research infrastructure by developing teams, using systems approaches, and introducing new ways of conducting research will also be considered.

76 LINKING KNOWLEDGE WITH ACTION Researchers are required to submit a 15-page proposal describing their project including a discussion of the potential environmental and economic benefits of the research. Proposals received by the host agency (alternates between EPA and NSF) are peer reviewed on a competitive basis by a panel of reviewers with the appropriate expertise for evaluation of scientific and technical merit. Those proposals that are recommended for funding by the peer review panel are then circulated to program offices at EPA (i.e., Office for Pollution Prevention and Toxics, Office of Solid Waste, Office of Policy, Economics, and Innovation) and to the program managers at the National Science Foundation. The EPA funds projects that are most relevant to the goals of the program offices and the mission of the agency. Program Managers at NSF fund projects that are most promising in advancing the scientific and technical knowledge necessary to move toward sustainability. The goal of the TSE program is to research, develop, and promote implementation of scientific and technical advances to reduce water, material and energy intensity and increase the use of benign material and energy. To date, the TSE program has many success stories and is developing a suite of metrics to describe the measurable outputs and outcomes of the program. This program has relied heavily on anecdotal evidence from the academic researchers to capture the environmental and economic benefits of the implementation of science and technology that resulted from this funding. In addition, this program has continually tracked scientific peer- reviewed journal articles, patent applications filed and granted, industry collaboration formed, and contribution of this research to the “state of science” in a given area (i.e., alternative solvents, bioengineering, etc.). One of the most important outcomes from this program that has not been quantified is the education and training of the future workforce, particularly scientists and engineers, to have an awareness of the potential impact of their work on the society, the economy, and the environment. 3. Program organization The TSE program is bounded to only include the highest quality scientific and engineering research that advances the discovery, development, and use of innovative technologies and approaches to avoid or minimize the generation of pollutants at the source. Other than aspects of materials flow and reuse, this competition is not intended to address issues related to waste monitoring, treatment, remediation, recycling, or containment other than in-process recycling of waste. Research in remediation and treatment of hazardous materials, while very important, is largely supported by other program activities in both agencies and elsewhere. Beyond the scope that the TSE research must aim toward preventing pollution at the source, the molecules, processes, products, systems, and industrial sectors are not specified or limited. It is the intent of this program to solicit the most relevant, innovative, and promising ideas from the academic community. This broad approach to the TSE program has resulted successful scientific and engineering research that has had a significant impact across disciplines and in many sectors. The boundary between initial producers (academia) and users (other academics, industry, and agency decision-makers) included a mix of scientific literature, professional conferences, and internal agency briefings (at EPA). Easily searchable web sites at both EPA and NSF also contributed, as did associations with professional societies such as ACS, including the Green

APPENDIXES 77 Chemistry Institute, and AIChE. In retrospect, this system was perhaps not as strategically designed as was needed. 4. The decision-support system From EPA’s perspective, the program has been steered by the mission of EPA and the program goals associated with that mission, including a strategy and multi-year plan that includes this and related “prevention” research. In addition, input from the EPA “media offices” on which highly peer rated projects to fund –based on their view of relevance to their program mission – is an integral part of the decision-making process for funding grants. However, the program has relied heavily on the innovation and knowledge of the academic community to propose work in areas with the most promising potential results for scientific advances and for environmental and economic benefit. In the final analysis, environmental results from this program are realized only if the research ultimately leads to development of new products and processes by industry. The communication with industry, the intended end users of this research, regarding their priorities and needs has been more indirect than direct. It has come mostly from informal interactions through professional and scientific conferences. In the early years of the program it was guided also in a general way by the priorities established in a document called Chemical Industry Vision 2020. 5. Learning orientation The implementation of this program provided significant opportunity for feedback, learning and mid-course correction. In particular, by monitoring the number, quality, and focus of proposals received we were able to judge whether the program was likely to lead to significant new knowledge. The actual quality of new knowledge being developed was transparent through a reporting process with public web sites and by monitoring both publications and presentations at professional conferences. This program, as with all government programs, is held accountable by the White House Office of Management and Budget through the Program Assessment Rating Tool (PART) and Government Performance and Results Act. The recent PART analysis indicated that this program did not effectively demonstrate quantifiable outcomes in terms of environmental and economic benefit. This raises questions about the nature of long-term research, the government’s role in helping to commercialize pre-competitive research, and the process of moving from bench-scale to implementation. This specific program has been reviewed twice by an outside panel of experts. The first panel found that 1) TSE has been extremely successful in meeting its goals of getting very high-quality research on the very important focus area of pollution prevention; 2) the funding level needs to be significantly increased (10-fold) to sustain in strength and the core research community; and 3) EPA and NSF should work to shepherd the successful research coming out of TSE to other federal programs which lead to commercialization and the private sector, e.g., through workshops. The second panel is in the process of completing their report. EPA’s extramural grants program as a whole was reviewed by the National Research Council which found that the STAR program was 1) "excellent"; 2)…has provided EPA with

78 LINKING KNOWLEDGE WITH ACTION independent analysis and perspective that has improved the agency’s scientific foundation," and 3) "the STAR program should continue to be an important part of EPA’s research program.“ The several most important lessons learned from this program to date include: 1. Science and technology for sustainability can contribute simultaneously and directly to environmental and economic benefit 2. Design quantifiable metrics and reporting mechanisms into programs at inception, wherever possible 3. Demonstrate quantifiable benefits that are a direct result of these efforts 4. Effectively communicate with stakeholders for data collection and dissemination 5. Engage a wide variety of stakeholders in the process such as industry and review panels in suggesting research topics 6. Solicit participation from a wider range of government agencies to ensure continuity and stability of the program 7. Expand the funding base beyond the government to include additional resources possibly in the form of matching funds 6. Continuity and flexibility This program is directly affected by budget resource pressures given that the program is designed to directly support extramural research through grants. The EPA and NSF have for the past nine years contributed similar levels of funding to this program. The EPA budget for this program was not included in the President’s FY05 Budget Request and this is in part a result of the recent failing score in the PART analysis. THEME III: AGRICULTURE AND ECOSYSTEMS Joint Fact Finding: Using Scientific Information to Build Collaboration and Community Decision Making Herman Karl USGS 1. Problem definition Why do decision-makers, including community-based groups, often ignore scientific information even as the call for decisions grounded in sound science escalates? My on-the-ground experience as a research scientist and project chief over the years has led me to conclude that in many cases it is necessary for producers and users to co-produce knowledge in order to enable its effective use in management decisions and policy making. Decision-making is often driven by a variety of nonscientific, adversarial, and stakeholder dynamics. Thus, even though science helps inform choices, it is only one of many values and interests considered by each stakeholder. The inadequacy of established mechanisms and institutional frameworks for natural resource management and environmental problem solving has become increasingly apparent in recent decades owing to the ever-increasing contentious nature of the disputes. An adversarial approach

APPENDIXES 79 exacerbates conflict and makes difficult the crafting of wise and durable policy solutions. An alternative and, in the view of many, better approach to ecosystems/natural resources management and environment policy is one based on a process of collaborative problem solving that seeks consensus. Joint fact-finding (JFF) is a procedure for involving those affected by policy decisions in the continual process of generating and analyzing the scientific and technical information used to inform those decisions. JFF allows for the consideration of local/cultural knowledge while preserving the independence of the scientists, as well as their commitment to the best practices of scientific inquiry. In a “high quality” JFF process, knowledge users and producers frame the questions to be addressed, choose who will do the studies, and discuss and interpret the results together. Our hypothesis is that the more you involve people affected by a policy decision in the framing of the scientific inquiry and the generation and interpretation of the scientific date, the more likely they are to value the results and use the information. 2. Program management Our goal is to improve on-the-ground outcomes by developing a project structure that integrates: 1) joint fact-finding as part of a meaningful participatory, community-based approach to ecosystems and natural resources management and land use planning, 2) adaptive management as a principle for making decisions that allows flexibility to accommodate new information from ongoing investigations, and 3) societal learning by monitoring and assessing the impacts of management and policy decisions. The U.S. Geological Survey is in initial stages of exploring the role of scientists and science in collaborative processes that include joint fact-finding. A pilot project was undertaken to study the decision-making processes with respect to the role of science in a local watershed, San Francisquito Creek (SFC), California. The project was designed by a group of citizens in dialogue with scientists. Four citizens and two scientists comprised the project steering committee. The project takes a problem-focused, in contrast to discipline- focused approach. Our goal was to help the community “create a solution” with respect to four issues: flooding, aquatic habitat restoration, dam removal, and TMDL impairment. A committee, composed of subgroup of citizens and scientists, decided that a sediment budget needed to be established for the watershed to aid in decisions regarding the four issues. Two discipline program projects funded through USGS programs were developed to study sediment issues in the watershed. These studies are ongoing. Measurable goals were determined with the representatives from the community and coordinated with committees of the San Francisquito Creek Joint Powers Authority and the San Francisquito Watershed Council. The decision-making body, the San Francisquito Creek Joint Powers Authority, in the watershed makes decisions using a traditional public involvement process. In other words, it uses advisory committees and public comment. As such it is not a participatory process whereby decisions are made through consensus of the stakeholoders. The project team also “studied the problem.” The team began its research by studying the issues and the ways land use and environmental decisions are made in the watershed. The team organized into three principal study groups: Social Dynamics Studies, Biophysical and Geographic Scientific Studies, and Communication and Learning Studies. Overall, we focused on the role of science in decision-making. The research team identified more than 30 obstacles and barriers to collaborative processes and the consideration of sound science in the decision-making processes in the SFC watershed. These have been grouped into five major categories: 1) Lack of scientific understanding, 2) Ineffective communication of scientific understanding, 3) Lack of trust, 4) Fragmentation of responsibility and conflicting interests and 5) Distribution of power.

80 LINKING KNOWLEDGE WITH ACTION We compared the decision-making process in the San Francisquito Creek watershed with that in the Tomales Bay watershed. The Tomales Bay Watershed Council is comprised of all relevant stakeholders, including representatives of government organizations, in the watershed. Although, people holding different views sit on the council, they have found that they can and have moved away from conflict toward cooperation. The people in this community have learned that when they become stewards of the land, they focus more on what they share in common and less on how they differ. In their watershed, the council makes use of the best available science to help inform consensus-based decisions to address their land use and environmental problems. The functioning of the Council as a consensus-based decision-making body continues to be studied and it serves as a comparison to the traditional public involvement process of decision-making in the San Francisquito Creek watershed. 3. Program organization The U.S. Geological Survey essentially served a boundary spanning function by bringing together end users and scientists to foster an ongoing dialogue. The boundary spanning function was built in to the project from the beginning. We did not use the term “boundary spanning function or organization.” Our goal was to provide a forum or environment for citizens, decision- makers and scientists to interact. The achievement of mutually determined research goals was tied to the ability to get funding. Scientists at the U.S. Geological Survey (USGS) are embarking upon research that explores the problems of incorporating science into value-laden societal decisions. This research includes designing experiments that will assess the appropriateness of using JFF as a component of a collaborative problem solving process. This line of research is especially appropriate for USGS because it is unique among the DOI agencies in that it has no regulatory authority. USGS scientists do not advocate for a specific policy although their science can help inform policy choices. The exploration of the role of science and scientists in collaborative processes that include JFF is one of the primary mission goals of the USGS’s Science Impact Program (SIP). Science Impact represents a focused USGS effort to improve and expand the use of science information to inform and support decisions at all levels of society. This effort encompasses developing and implementing improved methods and processes to enhance linkages between science and decision-making. 4. The decision-support system We did not use a computer-based decision support system in pilot projects exploring joint fact- finding. Parenthetically, it is my view that if such systems are used that they should be developed with the end-user. Commonly these systems are developed by software engineers based on what they “think” the end user needs or wants. Consequently, these systems are often not used by the intended user. Decision support systems, predictive models, and other forms of scientific information when used to inform a collaborative process can be thought of as “aids to the conversation” that occurs as part of the multi-party negotiation. In USGS we intend to explore the use of predictive models in collaborative processes. The “decision support system,” in my view, is the conversation that occurs among the parties in the case. Computer tools and scientific products are “aids to that conversation.”

APPENDIXES 81 5. Learning orientation The project was established with Venture Capital awards from the Geology Discipline and from the Director’s Office of the USGS. These awards are highly competitive and encourage highly innovative research that pushes the envelope of USGS programs and that is outside of traditional USGS programs. The Venture Capital programs foster high-risk research that has the potential of high payoff. Significantly, lessons are learned from failure. Although planned, external review boards were not convened. As a consequence of the Venture Capital research, USGS is establishing a MIT-USGS Science Impact Collaborative as part of the Science Impact program. An external panel will be assembled to advise the Collaborative and to help it reflect upon its experiences. We gauged our success by impact that was broader in scope than the products and results enumerated in the original project proposals. A major pitfall for experimental programs is rigidity. Experimental programs by their very nature must be adaptive and flexible. It is necessary at times to “take a leap of faith.” This is the most important lesson to be learned. As is the case with most path-breaking research, the project team found that a mid-course change in research was necessary to best test the hypotheses and accomplish the objectives set forth in the original proposal. Specifically, it was essential to hold a pilot training course for USGS scientists to introduce them to Joint Fact Finding, an emerging new approach to balancing science and politics in ecosystems management and building an informed consensus. USGS now intends to offer this pilot course as a regular course three times per year. As a direct result of research undertaken as a part of this Venture Capital project, Joint Fact Finding as a component of a collaborative approach to address environmental policy and ecosystems and natural resources management decisions is now discussed at the highest levels in the USGS and Department of the Interior. An outcome of broadening the scope of the originally proposed research is that findings of the Venture Capital project team may significantly influence the decision-making process used by the Department of the Interior for setting environmental policy and making ecosystems and natural resources management decisions. For example, the USGS Director, Charles Groat, convened a workshop on joint fact finding for the USGS Executive Leadership Team. I have been invited to brief the Secretary of the Interior, Gale Norton, on joint fact finding. An outcome of broadening the scope of the originally proposed research is that findings of the Venture Capital project may significantly influence the decision-making process used by the Department of the Interior for setting environmental policy and making ecosystems and natural resources management decisions. However, it must be noted that such risk taking is not rewarded in the traditional programs. There are no incentives in traditional programs to take risk. Champions are required to allow such risk taking and risk takers need to be prepared to pay a price when evaluated through the traditional process. 6. Continuity and flexibility Decisions that involve ecosystems and natural resources management and environmental policy are part of a continual process. Often these management decisions are associated with very contentious issues. Conflict does not go away and a decision-making process must be put in place that can deal with conflict and contentiousness. It is extremely difficult especially for Federal agencies to maintain both a financial and human resources commitment over the long term. As an instructor in the BLM Community Based Stewardship course I have seen the

82 LINKING KNOWLEDGE WITH ACTION exponential growth of community stewardship groups. The principles of consensus building and multi-party, interest-based negotiation developed over the past thirty-five years provide a framework for a model of decision-making in which citizens and government share the responsibility for land use planning, ecosystems/natural resources management, and environmental policy. This shared-governance model is a citizen-centered, community-based approach. Such an approach is best achieved through the alignment of informal community networks and formal government systems. In this model citizens take responsibility for being stewards of the land. Citizen stewardship groups play a leadership role in working with government to seek consensus on vexing and complex environmental issues. Many believe this process is a better way to arrive at wise solutions that result in stable policy. Innovative partnerships will help leverage resources both financial and human to help ensure continuity and relevance to users’ needs. Courses such as those offered through the BLM Partnership Series and the USGS Joint Fact Finding course build the capacity in institutions and among citizens to co- produce knowledge and to find ways to leverage public and private funding. Academic institutions need to introduce courses to train a new generation of scientists and applied social scientists in the integrated tools and techniques of using joint fact finding in science-intensive policy making. 7. Other insights? Mutual respect and trust are essential to a joint fact finding process that involves diverse stakeholders. Face-to-face conversation is important. For insights on linking knowledge to action, I’ll let someone with practical experience, Michael Mery, chair of the Tomales Bay Watershed Council (California) speak: “Sharing a sense of place is the first step. We will all benefit from an exploration of what we share, that is, those points of common interest. Understanding that we share an ecosystem or watershed is essential. Our watershed is relatively small, 220 square miles, small enough to wrap our mental arms around it. Given that common conceptual framework, it becomes obvious, for example, that if agricultural producers are to stay in business they must be concerned with soil loss. Similarly, environmentalists will be interested in minimizing stream siltation and habitat integrity. We might describe our needs differently, but agree on the outcome for complementary reasons. This might seem obvious, but there must be the appropriate atmosphere for collaborative planning efforts to be possible based on a common understanding of place. Through the process of watershed characterization, describing those realties on the ground we share and mutually value, we can create the basis for sufficient trust and the acceptance of the other’s legitimate interests. Sufficient, that is, to begin to focus on mutually agreeable outcomes for similar, and sometimes different, reasons. If we take the time to work through this process, one obvious consequence will be the blurring of lines between interest groups; the distinction between the regulator, environmentalist, Ag producer, chamber of commerce member will become less and less clear as we focus on what we share rather than how we differ. In this context, we will begin to see that the health, stability and restoration of the watershed includes all of us as responsible stewards. The Watershed Council members and others do share this understanding based on the necessity of the best available science for our decisions. The entire effort rests on a common sense of place, a strong sense of joint responsibility, and a commitment to the long-term enhancement of the watershed resulting from joint effort. We continue to differ, conflict arises and will continue to do so, as we work our way through this process. In my view, however, it is through this community-based collaborative

APPENDIXES 83 effort that we have the best chance to address our land use and environmental problems successfully. In this commitment to our grandchildren and their grandchildren, we will proceed.” Contact Information Herman Karl Chief Scientist, Western Geographic Science Center U.S. Geological Survey Currently Visiting Lecturer Department of Urban Studies and Planning Massachusetts Institute of Technology Room 9-330 77 Massachusetts Avenue Cambridge, MA 02139 617-324-0262 hkarl@mit.edu (hkarl@usgs.gov) SERVIR—A Regional Environmental Monitoring and Visualization System for Central America (in collaboration with NASA’s Earth Science Applications Program) Woody Turner NASA 1. Problem definition SERVIR has arisen from over five years of collaboration between NASA and the Central American Commission on the Environment and Development (CCAD). Early cooperative activities between NASA and CCAD included archaeological research on the ancient Maya and the provision of imagery and data products by NASA to CCAD for its use in developing the Mesoamerican Biological Corridor (MBC). CCAD was established by the seven national governments of Central America to promote the sustainable development of the entire isthmus. The MBC is a unique international experiment in which seven governments are attempting to manage large portions of their territory in a collaborative and sustainable manner in order to conserve the rich biological and cultural diversity of this region. Our previous work with CCAD demonstrated the utility of remote sensing imagery for both long-term environmental management and short-term response to natural disasters (i.e., Hurricane Mitch). The SERVIR project leads had worked with a number of Central American government officials and NGO personnel prior to proposing the SERVIR concept to NASA. Our personnel developed the concept not only in concert with the Central Americans, who will ultimately take over the operation and management of SERVIR, but also with officials from USAID and the World Bank, two organizations that are providing funding support. 2. Program management Yes, SERVIR is very much in the “project” mode. As an ongoing project, it has definitive milestones and a schedule for meeting them. NASA funding for this five-year project occurs in annual increments with each new year’s funding contingent on completion of work in the

84 LINKING KNOWLEDGE WITH ACTION previous year. Working with our Central American partners to define a limited set of requirements at the outset is crucial to success. The program manager frequently reminds the project manager of this concern. 3. Program organization As the term is described above, the NASA Earth Science Applications Division is essentially a boundary organization. Its purpose is to take the results (i.e., observations, measures, and models) of NASA’s Earth Science research programs and, working closely with our partner organizations, apply them to the decision support systems of these partners. Thus, it is the role of the Applications Division to translate NASA’s research results into tools that directly address the requirements of our partner agencies for decision support. The Applications division has been following this “measures to models to decision support” approach for approximately two years. Under this approach, NASA will work with a partner organization to verify and validate the decision support tools generated. Furthermore, we will benchmark the performance of these new tools in order to provide a measure of the added value gained by their use within the partner’s decision support system. 4. The decision-support system SERVIR is an end-to-end system with three primary elements: monitoring and measurements; Earth system models; and decision support tools. The monitoring and measurements component provides direct observations for SERVIR. These observations are also used in the Earth system models component to make predictions. The observations and predictions, in turn, enable the decision support tools that comprise SERVIR. The outputs of the decision support tools produce value and benefits for society. The Integrated System Solutions diagram in Figure A-3 depicts the end-to-end system for SERVIR.

APPENDIXES 85 FIGURE A-3 Integrated Systems Solution for SERVIR. 5. Learning orientation SERVIR is very much a work in progress. To my knowledge, it is a first of its kind activity and is thus in an experimental mode. Meeting deadlines and delivering the pieces of the system to our Central American colleagues constitute initial measures of success. For this program manager, a key to success is first implementing known technologies with a proven track record, e.g., integrating the MODIS Rapid Fire products into the system. While the technical challenges of implementing SERVIR are significant, the political challenges are likely to be greater. So, working at the outset with known technical solutions is key to keeping that portion of the project as straightforward as possible. Managing the expectations of partners is also crucial, i.e., ensuring that both sides have an adequate understanding of what SERVIR will and will not do. 6. Continuity and flexibility Funding for SERVIR comes primarily from three institutions: NASA, USAID, and the World Bank. NASA is contributing roughly half of SERVIR funding over its first five years. The World Bank is funding many of the Central American contributions. The involvement of international donor organizations is critical to the success of SERVIR in that NASA will rely upon them for the long-term support of the system. If they are to remain involved, SERVIR will have to demonstrate over the next 3-4 years its basic utility for environmental managers and policy makers in the region. Training has been a significant component of NASA’s activities since the outset of its work in Central America. We have been pleased to find a significant level of indigenous capacity for

86 LINKING KNOWLEDGE WITH ACTION using remote sensing and geographic information systems (GIS) in the region. The skill and enthusiasm of local collaborators has been tremendous. NASA’s challenge is to help these individuals refine their skills so that they can take full advantage of the new data sets, Earth system models, and visualization software that together constitute SERVIR. 7. Other insights? The importance of finding the right individuals to manage a project of this nature cannot be overemphasized. People matter a great deal!! NASA has been fortunate to find very capable project managers in Tom Sever and Dan Irwin. They have worked in Central America for many years. They are extremely personable and very committed to the sustainable development of the region. Tom is an archaeologist who studies the ancient Maya and a pioneer in the application of remote sensing to archaeology. Dan is an expert in the use of remote sensing and GIS, who was employed by a conservation NGO working in the region before coming to NASA. The number one trait these two bring to the project day in and day out is enthusiasm, which leads to a willingness to work through the problems that regularly arise. They tend to view such problems or challenges as opportunities to teach others about the project. This tendency has paid tremendous dividends to the project, both in Central America and Washington. SERVIR will succeed or fail due to the efforts of these two people. Contact information Mr. Woody Turner Program Scientist, Biological Diversity NASA Office of Earth Science Woody.Turner@nasa.gov 202-358-1662 Dr. Tom Sever Archaeologist Global Hydrology and Climate Center NASA Marshall Space Flight Center Tom.Sever@msfc.nasa.gov 256-961-7958 Mr. Dan Irwin Earth Sensing Applications Expert Global Hydrology and Climate Center NASA Marshall Space Flight Center daniel.irwin@msfc.nasa.gov 256-961-7945 Representative publications/products a. SERVIR website: [http://servir.nsstc.nasa.gov/home.html] b. NASA’s Earth Science Applications website: [http://earth.nasa.gov/eseapps/]

APPENDIXES 87 c. NASA’s Ecological Forecasting Program (the home for SERVIR in the applications program): [http://earth.nasa.gov/eseapps/theme13.htm] d. NASA’s Earth Science for Society Brochure, which discusses all 12 of our applications areas. Please go to the drop box at [http://ese-dropbox.hq.nasa.gov/ese-dropbox/] and click on “Science for Society brochure” Soil Quality and the Soil Management Assessment Framework (SMAF) Michael Jawson USDA 1. Problem definition Soils play a central role in agriculture production and environmental quality. Despite its importance, however, the soil resource is generally not appreciated by the public or even always by others working in agriculture and environmental management. The concept of soil quality or health was partly developed in the 1990’s to provide a means of assessing the status of the soil resource. However, among concerns raised was whether soil quality per se should be the end point. The soil quality concept is only truly useful when it can guide sustainable production and management decisions. A team of more than 15 ARS scientists from across the country was formed to develop the Soil Management Assessment Framework (SMAF), a decision support tool that is intended to help in the management of soils for both production and environmental quality endpoints. It goes beyond just describing the status of the resource like many water and air quality criteria because it also provides information on how to improve management for producers and technology transfer partners such as NRCS who are involved in producing this tool. The primary dialogue was between a USDA action agency (NRCS) and a USDA research agency (ARS). Discussions were ongoing and the NRCS Soil Quality Institute provided initial funding for post-doc to work with ARS on what became the Soil Management Assessment Framework. This post–doc was hired by NRCS and her duties include continuing to work on SMAF. Although the problem definition has not changed, the priorities of NRCS have. NRCS is in the midst of reorganization and the future of their Soil Quality Institute is uncertain. They also have become focused on determining the effects of conservation practices. The major program developed, the Conservation Effects Assessment Program (CEAP), is initially focused on water quality, although soil quality is a consideration and SMAF may be the assessment tool used for soil quality. 2. Program management SMAF was developed specifically to develop a solution; in this case, a tool to address the concern that soil quality was only a theoretical conceptualization. SMAF was developed to provide an utilizable tool for the assessment of management on the soil resource. Therefore, SMAF was specifically designed to “create a solution” because of the issues previously raised about only “studying the problem”. The production of the tool was its measurable goal. This goal was driven by both scientists and decision makers as stated above. Leaders were not held formally accountable as SMAF was developed by its field leaders and not their supervisors or because of a congressional or other department or agency mandate.

88 LINKING KNOWLEDGE WITH ACTION 3. Program organization The NRCS’ Soil Quality Institute may be considered to have served a boundary spanning function. Meetings were also held with farmers and other land mangers as the ultimate end-users before and during SMAF development. NRCS created the Soil Quality Institute to develop information and other technology transfer tools to serve its customers, private land managers. The Soil Quality Institute supplied a small, but critical, amount of the resources for the project and continues to be involved in SMAF’s improvements. The Soil Quality Institute serves as a linkage between the scientists (producers) and land managers (users) of the tool, although there are other linkages (e.g., between ARS and university scientists and the extension service, private industry, other government agencies, farmers, etc.). The Soil Quality Institute is involved in getting SMAF technology transferred to the users as part of its technology transfer activities. It does this primarily by training and providing tools to its first line field staff. 4. The decision-support system SMAF is an end to end system in that it is self contained. It has three components: (1) choice of management goal (production, waste recycling, or environmental quality), (2) indicator selection based on management goal, and (3) interpretation. It requires data from the users. Collecting this data requires field measurements to be made at each site. The development of the dimensionless response functions for the indicators was the most difficult element to put in place, because all of these needed to be generated by the project as none were previously available from the literature. This does not mean that other elements were easy. Changes in research occurred to develop the tool in the sense that scientists needed to think about their experiments and data in a new context and interpret them in a new way. It is too early to tell if and what changes in decision making will result. 5. Learning orientation Yes, it did have an experimental orientation. It is not clear how deliberate this was. Risks were not explicitly identified (really not sure what this necessarily means within this context). End- users were used to test the tool as it was being developed to provide feedback. New scientists were continuously brought into the project for evaluation and assistance. By focusing on management effects, SMAF is designed more for learning than knowing. That is, the soil quality “index” or “scores” are the least important outcome. The desired outcome is a change in management to correct problems identified. Lessons learned (many of these are from involvement in other projects and not just the SMAF experience): 1. First attempts at producing a tool for soil quality were based on theoretical conceptualizations, which were too abstract to gain involvement. A concrete “something” (e.g., a product) is necessary for there to be something to react to. 2. Giving a name to this activity or product (e.g., SMAF, GRACEnet, LTARs) helps develop focus and cohesion. 3. Teams for the sake of teams aren’t effective. Each member of a team needs a definitive role. (A team of all center fielders isn’t an effective team.) Team members should have a

APPENDIXES 89 willingness to become trans-disciplinary (i.e., willing to learn beyond their current discipline and knowledge base.) 4. Field leadership is critical. Program leaders are important as supporters, but they aren’t able to provide the day-to-day oversight often needed. Scientists in the field must carry out this function. There also needs to personnel that can be dedicated full time to the activity. 5. Field scientists and users must be integrated into the entire process as much as possible to develop and maintain a sense of ownership. 6. “Traditions” and institutional barriers must often be overcome. We (i.e., our own agencies) can be our own worst enemies. Reward and evaluation systems should reinforce team activities. 6. Continuity and flexibility ARS is a “hard” funded agency that conducts in-house research so that budget restraints are limited mostly by the personnel available to work on a topic and their limited resources and multiple commitments. These internal restraints, however, can be considerable. ARS National Program Leaders do not directly control budgets, so their influence results from visioning, articulating the strength of concept and focusing on relevance. The hard funding does provide continuity. Relevance is maintained by continuous interactions with users. Multi-location and interdisciplinary incentives are not of themselves rewarded. However, the most important factor for all agency personnel (both scientists and program managers) is impact. Interdisciplinary activities conducted in cooperation with users almost always leads to greater impact. Human capacity is garnered to work on multi-location interdisciplinary projects by focusing on relevance and impact and the need for “teams” to produce the products to address national problems. Contact information Dr. Doug Karlen Dr. Susan Andrews National Soil Tilth Lab NRCS Soil Quality Institute Ames, Iowa Ames, Iowa Phone: 515-294-3336 dkarlen@nstl.gov sandrews@nstl.gov Representative publications/products There is both a stand alone (i.e., CD) and web version of SMAF. The Collaborative Agricultural Biotechnology Initiative Bhavani Pathak and Josette Lewis USAID 1. Problem definition The need to incorporate new strategies for achieving improvements in the amount and quality of food in developing countries over the next two decades to keep pace with rapidly increasing populations, is compelling. The use of biotechnology in agriculture is one important tool for

90 LINKING KNOWLEDGE WITH ACTION achieving this goal. Crops improved through genetic engineering offer benefits of enhanced agronomic, nutritional, and marketing qualities, contributing not only to increased production of food, and hence increased income, but also to new food-based, and therefore, more sustainable strategies for addressing the issue of malnutrition in developing countries. In 2002, USAID launched the Collaborative Agricultural Biotechnology (CABIO) Initiative— the agency’s new comprehensive strategy to assist developing countries with accessing, managing, and applying the tools of biotechnology to improve agricultural productivity, environmental sustainability, and nutrition. This new initiative, which builds on a foundation of the agency’s twelve-year experience, was developed to facilitate the expansion of agency biotechnology efforts and demonstrate U.S. leadership in a comprehensive approach to promoting the safe access and use of biotechnology to alleviate hunger and promote economic growth in developing countries. CABIO incorporates lessons learned and carries forward successes of previous agency programs, while recognizing changes in international agricultural biotechnology, more broadly. The initiative has the following goals: • Research and Technology Development to address developing countries’ crop and animal production needs with a better understanding of potential impacts on biodiversity and the environment. • Strengthening Public Institutions to use research, development of policy and regulatory frameworks, particularly in biosafety and intellectual property rights, informed decision- making, and public outreach to promote safe use of biotechnology. • Local Private Sector Development to help to deliver new technology and integrate it into local agri-food systems. • Communication and Outreach activities to local stakeholders on use of technology. Background USAID started to explore opportunities for integrating biotechnology to developing country agricultural systems in the 1980s, and as part of designing a new program called upon the National Research Council (NRC) of the U.S. National Academy of Sciences for assistance in identifying broad priorities for consideration in an international biotechnology development program. Among the recommendations, the NRC panel placed equal weight on addressing institutional management issues, particularly the capacity to address issues of intellectual property rights (IPRs) and biosafety, as on research and technology development. The panel also recommended that USAID consider the role of the private sector, suggesting private-public or private-private sector linkages between U.S. companies and developing country research institutions and companies. Based on these recommendations, the agency entered into an agreement with Michigan State University, in September 1991, for the Agricultural Biotechnology Support Project (ABSP)—a consortium of public sector institutions and private companies in the United States and developing countries. During its twelve-year life, ABSP was designed to move from the research and development stage to field-testing and towards commercialization of potential products. While ABSP’s scientific objectives included transfer of host plant resistance genes into

APPENDIXES 91 developing country crops, training scientists, administrators, and policy makers in biosafety procedures and intellectual property rights was an equally important component of the project. ABSP was active in a number of developing countries, conducting research on a diverse set of food crops and involved in numerous policy assistance activities both in individual countries and in regional efforts. In addition to partnering with public sector institutions in the U.S. and developing countries, ABSP was active in promoting private sector involvement. U.S. companies such as ICI Seeds (now Syngenta), DNA Plant Technology, Pioneer Hi-Bred, Asgrow Seed Company and Monsanto, and developing country companies such as Fitotek Unggul in Indonesia and Agribiotechnologia de Costa Rica were involved in the project. Summary of ABSP activities Following is a partial list of project activities in key countries: 1. Technology Development a. Egypt: Development of Bt. potatoes resistant to potato tuber moth, maize resistant to stem borers, tomatoes resistant to yellow leaf curl virus, potyvirus resistant cucurbits, drought and salinity tolerant wheat. b. Indonesia: Development of potatoes resistant to potato tuber moth, maize resistant to Asian corn borer, micropropagation of topical crops. c. Kenya: Development of virus resistant sweet potatoes d. India: Development of high beta carotene mustard oil 2. Biosafety Development: Training and technical assistance with biosafety policy development a. Egypt: guidelines adopted in 1995 b. Indonesia: guidelines adopted in 1997 c. Kenya: training and capacity building d. Southern Africa: Regional network established and training provided 3. Intellectual Property Rights and Technology Transfer: Training, capacity building at institutional and national level to manage IPR issues a. Established technology transfer offices in Egypt and Indonesia b. Indonesia PVP law passed in 2001 Lessons learned from the ABSP experience: The ABSP experience broadened the agency’s understanding for making biotechnology available to developing countries. Described below are some of the lessons learned from the project’s experience in various countries: • The process of agricultural biotechnology development and deployment in developing countries is complex and generally involves building capacity and a whole host of factors beyond just research and development. • The integrated approach taken by the ABSP project to address both the technology development and create the enabling policy framework, in biosafety and IPR issues, was very successful in the overall strategy of product development. By taking an integrated approach the ABSP project was able to leverage resources, particularly in biosafety and IPR that individual scientists would not have been able to do on their own.

92 LINKING KNOWLEDGE WITH ACTION • Local ownership of technologies provided an impetus for moving the policy development process forward. • Biotechnology outreach and communication involving various stakeholders right from the technology development phase are important to ensure final commercialization of the product. Key gaps While successful at various levels in increasing awareness of biotechnology, the project did not accomplish commercialization of products, a goal that it set out to accomplish. Key reasons for these included: 1. A lack of clear understanding of product development by public sector institutions, both in developed and in developing countries: In contrast to previous achievements in crops improvement, almost all commercially released bioengineered crops to date have been developed by the private sector. As a result, public sector institutions, both in developed and in developing countries, have a limited understanding of this process. While the project promoted private sector involvement, their market interest did not match in terms of expectation of developing crops for small holders’ markets. Additionally, given issues of the proprietary nature of enabling technologies, increasingly complex biosafety considerations and other stewardship issues, the private sector was hesitant in being fully committed to enabling product development. 2. Priority setting and choice of technologies not based on adequate economic and market analysis: Technology development, at public sector institutions, was often directed by host of factors not always anchored in a rigorous analysis of impact. In the case of biotechnology access to enabling technologies, investigator interest and capability of the institutions all contributed to there being, in many cases, either a mismatch between priority constraints and technologies developed or a priority constraint being addressed in a crop variety not relevant to local needs. In the case of ABSP, this was further compounded by the institutional constraints within USAID, particularly as there was not a good intersection of global priorities with country specific allocation of resources. 3. Increased complexity of policy issues made it difficult for implementing partners to adequately access and manage the needed expertise: An integrated project that addressed policy issues with technology development was extremely successful in the early years of the project when policy development was spurred by technology development. In later years however, especially as biosafety concerns gained a higher profile internationally, this became a complex portfolio of activities for one institution to manage. 4. Unforseen global developments in perception and acceptance of biotechnology that could not be predicted in the early years of the project had a major impact on the project goal of commercializing products of biotechnology. As a result of the international debate about the technology, many developing countries were hesitant to proceed with product development. An unfortunate consequence of the global skepticism of the technology was decreased funding for technology development, especially from other donors, that resulted in fewer technologies developed overall for small holder farmers in developing countries. In some cases, other capacity building initiatives, particularly in

APPENDIXES 93 biosafety policy development, also promoted an overtly cautionary approach that further hindered development of biotechnology. New approaches and strategies Incorporating lessons learned and taking into account global developments in the field, USAID designed CABIO as a comprehensive set of activities to develop and deploy biotechnology more broadly to meet needs of small holder farmers in developing countries. Key features of CABIO include: 1. Development of an increased portfolio of activities and discrete programmatic mechanisms to address technology development, biosafety policy development and commercialization of technology: In the second generation of agency biotechnology portfolio, multiple projects were developed to address specific issues. Thus, the Agricultural Biotechnology Support Project II (ABSPII), a U.S. university project, deals primarily with technology development and provides assistance on intellectual property rights IPR) issues, while the Program for Biosafety Systems (PBS), a consortium of several institutions led by the International Food Policy and Research Institute, primarily provides support for development and implementation of biosafety regimes. Commercialization issues will be addressed by yet another program to be developed while new projects have been designed to address other important issues such as research on addressing micronutrient deficiency. 2. A priority-setting process for technology selection based on economic and market analysis, taking into account input from a broad range of local stakeholders: To ensure that technology development will result in development of products, especially under the ABSPII project, a much greater emphasis has been placed on taking these issues into consideration early in the priority setting process for selection of technologies. 3. With the goal of spreading the technology more broadly and pooling limited technical capacity, particularly in Africa, projects under CABIO promote regional approaches for technology and policy development. 4. Promoting new models for technology access and management: Recognizing the increased complexity in access and in regulation of biotechnology, USAID has looked increasingly at new approaches addressing these issues. For example, USAID is partnering with the Rockefeller Foundation in supporting the African Agricultural Technology Foundation (AATF) to promote increased access to proprietary technologies for use in Africa, while programs such as PBS will look at new models to address biosafety amenable to developing country needs with a greater emphasis on consideration of implementation and promoting complementarities and synergies across existing relevant policies Conclusions Although developing countries stand to benefit both in terms of food security and in environmental sustainability by using biotechnology, there are major barriers that have to be overcome before the technology can be accessed by farmers. These include not only a lack of availability of a range of technologies to address local agricultural constraints, but also considerations of limited capacity of public sector institutions to develop and promote development of biotechnology products. In addition, policy issues, especially in consideration of

94 LINKING KNOWLEDGE WITH ACTION biosafety and issues pertaining to access and stewardship surrounding the use of biotechnology add to the complexities of product development. Finally, there needs to be an increased effort in ensuring that a greater number of products are being developed for the benefit of small-holder farmers in developing countries. USAID’s CABIO attempts to address a number of these issues under its programs. Decision Support Tools for Forecasts of Global Agricultural Productivity and Yield in Collaboration with NASA’s Earth Science Applications Program Ed Sheffner NASA 1. Problem definition The World Agricultural Outlook Board (WAOB) of the U.S. Department of Agriculture issues global monthly estimates of the productivity and yield of major agricultural commodities. These estimates are a primary source of information for managers and policy makers in the agricultural community including farm operators, agribusiness, commodities traders, government agencies world-wide and non-government organizations. The WAOB estimates are based on convergence of evidence methodology, utilizing the best available information from a number of sources including assessments of agricultural condition derived from analysis of observations from Earth orbiting satellites. These analyses are made by the Production Estimates and Crop Assessment Division (PECAD) of the Foreign Agricultural Service (FAS) of the US Department of Agriculture (USDA). The FAS has used satellite observations for crop condition assessments for many years and such observations are essential for PECAD. Landsat and AVHRR data are, to date, the most common sources of satellite data for PECAD-Landsat because of its high ground resolution (30m) and low cost, and AVHRR because of its high temporal resolution (1-2 day global coverage). New Earth observation satellites launched by NASA in the last 5-10 years, especially Terra, Aqua, Jason and Topex-Poseidon, acquire data with the potential of improving the accuracy and timeliness of the assessments provided by FAS to the WAOB. The “problem” addressed in this program is how to define, develop, verify, and validate new satellite-based products into the PECAD crop assessment procedures. Explicit in the program is the modification of PECAD procedures to accommodate the products and the implementation of FAS supported operational mechanisms to produce, deliver, archive, and analyze the products. The products delivered to date to FAS for evaluation emerged from several separate initiatives, all of which were either initiated by FAS or involved substantial involvement of FAS in the formulation of the proposal. In all instances, the products being tested differ from the standard products developed from the satellite systems to meet the initial Earth science objectives of the system. The modifications to the standard products are based on FAS requirements, and further modifications can be expected as new sample products are evaluated by FAS analysts. 2. Program management The NASA collaboration with FAS is comprised of several peer-reviewed proposals funded directly or in response to NASA solicitations. Four separate FAS/NASA partnership projects are currently underway. Each project is directed toward specific products and has a duration of no more than three years. Each project will measure its success by the incorporation of satellite- based products in the operational procedures of PECAD and demonstration of the impact of

APPENDIXES 95 NASA observations and measurements on the decision support tools of FAS. The product requirements were established through interaction between the scientists and FAS staff. The overall program goal, i.e., the impact on decision support tools, is a programmatic objective of NASA’s Earth Science Applications Division. The program managers within the Division are accountable to achieve the program goals. 3. Program organization The program involves a “boundary spanning function” in the sense that organizations other than the user (FAS) and the producer (the government and university organizations producing the products) delimited the existing analysis environment and methods of PECAD, so that the impact of the new products could be properly benchmarked (i.e., the improvements in the performance characteristics of the PECAD system resulting from the new products can be described quantitatively.) These same organizations will also play a vital role in making recommendations to FAS on how to alter existing, or implement new, procedures for use of the new products operationally. These organizations are from the academic community—Universities of Arizona and Missouri—and NASA field centers—Stennis Space Center. Their participation in the program is supported not by the projects but by the Program Planning and Analysis and Crosscutting Solutions functions within the Earth Science Applications Division. The functions performed are vital to the success of the program. Without a thorough understanding of the existing decision support tools, in this instance, the elements and operation of PECAD, it would be difficult for the producers to generate products suitable for the PECAD system. It would also be more difficult for FAS to understand how then new products would benefit the analysis functions. The description of the PECAD system was the first task of the program. It was completed in the first year of the projects and was described by FAS and the best documentation of the PECAD system compiled to date. It will serve as the baseline against which the new products will be benchmarked. 4. The decision-support system The Earth observation products incorporated into the PECAD analysis procedures are part of an end-to-end system, although the boundaries of that system, especially the final end, are for the user to select. The WAOB provides a resource monitoring function. The inputs considered by WAOB to make its crop production and yield estimates are decision support tools. The estimates that emerge from the board are “decisions.” The output from the WAOB, i.e., the estimates, are, in turn, decision support tools used by the community to make decisions on what, where and how much to plant, how to manage funds and where to direct resources. The outputs from PECAD are country and regional production and yield assessments based in large part on remotely sensed data. These assessments can also be considered “decisions” and the Earth observation products used by FAS are decision support tools. This program is ongoing. Initial benchmarking of the satellite based products is expected by the end of FY05. The products will improve the accuracy (i.e., reduce the error terms) and timeliness (better estimates earlier in the growing season) of the information supplied to the WAOB. As such, the estimates from the WAOB should also be more accurate and published earlier than currently. The ultimate outcome should be increased economic efficiency among the individuals and organizations that base their decisions, in whole or in part, on WAOB estimates, and the social benefits derived from such economic efficiency (see Figure A-4).

96 LINKING KNOWLEDGE WITH ACTION FIGURE A-4 NASA’s Agricultural Efficiency Integrated Efficiency Solution: From NASA observations and measurements to social and economic benefits for agriculture (involving collaboration with NOAA and USDA). 5. Learning orientation This program is focused on achieving practical, operational goals based on peer reviewed science. All the work underway stems from proposals that received high marks from peer review. Consequently, the community believes the products can meet their intended purposes. Risks are relatively low because the products were defined by the user rather than by the producer— “technology pull.” The products developed will be evaluated by the user before they are incorporated into the user’s operational procedures. In addition, the products will be verified and validated, and their impact benchmarked, independently from the producer and user. Although the success of the program is yet to be determined, to date it retains the enthusiastic support of the user in large part because the program incorporated lessons previously learned and documented by the NRC, e.g., Transforming Remote Sensing Data into Information and Applications, NRC, Space Studies Board, 2001 [http://www.nap.edu/catalog/10257.html]. The crucial lessons include: 1) Form a partnership with the user. It is important that the partnership should include a sharing of risk, through co-funding, as an indication that the user-partner is committed to the program and is willing to assume operational responsibility if the project meets its goals; 2) The user-partner has to be involved in design phase of the program and the establishment of project requirements; 3) Products developed for the user, whether observations, measurements or predictive models, must be evaluated by the user and/or an organization other than the user and producer; and 4) The current capabilities of the user and the procedural

APPENDIXES 97 improvements derived from the new products must be benchmarked to document the impact of the project. 6. Continuity and flexibility Budgetary restraints and potential changes are assumed risks in any project. These risks are minimized in this program through the commitment to joint funding of the projects, setting of objectives and milestones, and adherence to the programmatic objectives of the partner organizations as described in the IBPD of both agencies. 7. Other insights Concurrent with the individual FAS projects, NASA and USDA have formalized an Interagency Working Group (IWG) to identify more systematically USDA operational mandates that may be served through the integration of NASA Earth science observations, measurements and predictive models. The IWG provides guidance for enhanced collaboration between USDA and NASA, and the work with FAS has been a high priority identified in the Agricultural Efficiency component of the IWG. The IWG is an example of a mechanism that assures user involvement and commitment to a project from inception through acceptance of operational responsibility. Contact information Mr. Brad Doorn Mr. Ed Sheffner USDA/FAS NASA/Office of Earth Science doorn@fas.usda.gov edwin.j.sheffner@nasa.gov 202-690-0131 202-358-0239 Representative publications/products a. Information on PECAD with links to collaborative work with NASA: [http://www.fas.usda.gov/pecad/] b. “Crop Explorer” – PECAD on-line information on crop condition: [http://151.121.3.218/rssiws/] c. Global Reservoir and Lake Monitor tool – under development as a NASA/USDA partnership project: [http://www.pecad.fas.usda.gov/cropexplorer/global_reservoir/] d. PECAD’s MODIS Rapid Response Imagery tool – under development as a NASA/USDA partnership: [http://www.pecad.fas.usda.gov/cropexplorer/modis_summary/] e. Summary of NASA Earth science applications and related information: [http://earth.nasa.gov/eseapps] f. On-line press release on use of NASA Earth observations by FAS (January 2004): [http://earthobservatory.nasa.gov/Newsroom/NasaNews/2004/2004012016417.html] g. Decision Support Tools Evaluation Report for FAS/PECAD, Version 2.0, NASA/ Stennis Space Center, January 2004. h. Hutchinson, Chuck, S. Drake, W. vanLeeuwen, V. Kaupp. T. Haithcoat. “Characterization of PECAD’s DSS: a zeroth-order assessment and benchmarking preparation” Version 1.3, August 2003.

98 LINKING KNOWLEDGE WITH ACTION i. NASA’s Earth Science for Society Brochure, which discusses 12 NASA applications areas: Please go to the drop box at [http://ese-dropbox.hq.nasa.gov/ese-dropbox/] and click on “Science for Society brochure” The State of the Nation's Ecosystems: periodic, high quality, non-partisan reporting on key aspects of the condition of the nation’s ecosystems Robin O'Malley The H. John Heinz III Center for Science, Economics and the Environment 1. Problem definition Prior to the initiation of the State of the Nation's Ecosystems project, the United States did not have an agreed-upon suite of indicators describing the key characteristics of the nation’s ecosystems, and no mechanism for identifying and reporting such indicators. (This is in contrast to the relatively stable and generally accepted set of indicators describing economic activity at the macro level, and the several institutions and processes that report and periodically refine these indicators.) The H. John Heinz III Center for Science, Economics and the Environment, which produces The State of the Nation's Ecosystems (The Heinz Center, 2002; see item 10 for link), utilized a process involving participants from business, environmental advocacy organizations, academic institutions, and federal, state, and local governments. Each working group for the project included representation from these major societal sectors. In practice, the project was managed through a series of committees. One of these (the Design Committee) had individuals who were relatively senior in their organizations (e.g., titles such as Vice President for XX, Director of XX, etc.) and who provided a policy level (decision maker) perspective. This group was complemented by several working groups, with individuals generally at a more technical level. Chairs of each working group were also members of the Design Committee, ensuring a strong link and open dialogue between the two potentially divergent spheres of thinking. (In the interest of full disclosure, it should be noted that Dr. William Clark, Harvard University, and co- convener of the workshop for which this material is being prepared, is the Chair of the Design Committee). The involvement of both highly technical individuals and those with policy experience and expertise was crucial. Reporting on the state of the nation’s ecosystems requires communicating complex information in a manner that is accessible to non-specialists while maintaining the scientific integrity of the information. Issues such as the number of indicators and the tone, technical content, and amount of supporting information provided in the report, and the degree to which the report was dominated by indicators that are already well known by the public or included those that are seen as important by the ecological community, but are not well known by non-specialists, are examples of areas in which the report was shaped by the different viewpoints of these two communities. 2. Program management The program WAS developed in a “project” mode, with specific dates by which the report’s prototype and first edition were to be completed. These deadlines were driven primarily from the

APPENDIXES 99 decision maker end of the spectrum and were influenced by the need to demonstrate the potential for such a report, and to justify a significant expenditure in a reasonable time period. The target for issuing the prototype report was met, while the first full edition was about nine months late in completion. Internal organization pressure, the fact that deadlines were relatively widely known and thus a delivery-date expectation had been created, and the real fact of having utilized a significant fraction of available funds provided pressure that the program managers could not ignore. 3. Program organization The project did involve a boundary-spanning function. The boundary between technical and decision maker / user communities was touched upon in an earlier response (#1). I would also stress very strongly an additional boundary that is NOT captured by the traditional “research- versus-user” paradigm. As noted previously, the State of the Nation's Ecosystems project, and indeed all Heinz Center projects, included boundary spanning between business, environmental advocacy organizations, academic institutions, and government. Each of these major sectors may have both “researchers” and “decision makers” – but the researchers in each of these four sectors will often have very different perspectives, values, assumptions, and strategic ways of approaching an issue. Thus, it is inappropriate to lump all “researchers” together, much as it would be inappropriate to lump a decision maker from a resource extractive industry with a government regulator in an environmentally-leaning state or federal agency as “decision makers.” Inclusion of multiple research perspectives, and multiple decision maker perspectives, is a crucial design element that will strengthen many programs. 4. The decision-support system Successful reporting on the state of the nation’s ecosystems requires an end-to-end system that involves: 1. Collection of individual bits of raw data about the Earth or a component of an ecosystem 2. Aggregation of that data at larger geographic scales 3. Appropriate statistical manipulation 4. Reporting of statistical data at a regional and national level 5. Identification of key indicators of the condition of ecosystems 6. Gathering statistical data from multiple sources on multiple indicators 7. Reporting of these indicators in a form accessible to the target audience (i.e., decision makers and opinion leaders). While many of these elements are in place, there are huge substantive gaps (i.e., areas in which data are not collected or (#1) data collected by multiple entities is not aggregated (#2). In addition, prior to the initiation of The Heinz Center’s effort, there was no entity charged with identification of indicators, gathering data, and reporting (#5,6,7). In the initial phases of our work, there was an assumption that an ecosystem reporting effort could focus on items 5, 6 and 7. We have grown to understand that—because there is no single entity that focuses on the overall task of monitoring the nation’s ecosystems (i.e., items 1-4)—a successful indicator reporting effort will require attention to both filling the gaps in the

100 LINKING KNOWLEDGE WITH ACTION underlying data collection enterprise and assuring the continuity and maintenance of existing data collection efforts. Thus, the program has added elements that will focus on the resources necessary to both ensure continued flows of basic statistical data and filling of gaps, and has begun a policy-level conversation that will address institutionalization of the indicator selection and reporting effort (#5-7). 5. Learning orientation The State of the Nation's Ecosystems project is designed as an iterative, adaptive effort. The overall perspective is that getting a set of indicators “right” will take some time, and that the goal should be to reduce the level of change in the indicator set over time, until eventually a relatively conservative set is established, and changes are at the margin. (This is the case with economic indicators—they are revised periodically, but the system as a whole consists of a relatively stable set.) Evaluation and reflection have involved large numbers of presentations to many different groups, reviews of the report by outside experts, and synthesis of these feedback inputs by staff. That said, this process is relatively informal and probably could be developed into a more structured one. The most successful element of the entire venture, which has been highlighted by respondents from across the political spectrum and by people from both the research and decision maker/user communities, was the report’s steadfast refusal to adopt normative positions to describe environmental conditions. Multiple value-laden choices underlie the selection of indicators— which, after all, represent what is “important” to society. However, once that value-driven process was complete, information about the indicators and their values and trends was presented in a strongly neutral fashion. Trends or conditions were not described as “good” or “bad”— because any single trend may be viewed quite differently by different stakeholders. We will clearly continue this successful element of the experiment. 6. Continuity and flexibility The State of the Nation's Ecosystems project has been supported with both public and private funds. Across both Democratic and Republican administrations, the fact that the project was supported, in more than a rhetorical sense, by both foundations and corporations was viewed quite positively. Maintaining this diversity of funding is in large part due to two factors. The first is the neutral position taken by the report (see previous response). Essentially, all funders see the report as providing information they believe is important, but doing so in a way that is not overly influenced by political agendas they disagree with. The second factor is the report’s strong linkages within, particularly, federal agencies. We have involved both political appointees AND large numbers of career staff in the process. When the project moved through an administration transition, these career staff were crucial in highlighting the project to incoming appointees, assuring continuity of funding, and maintaining momentum on the project itself. Contact information

APPENDIXES 101 Robin O'Malley Senior Fellow and Program Director The H. John Heinz III Center for Science, Economics and the Environment 1001 Pennsylvania Avenue NW, Suite 735 South Washington, DC 20004 202-737-6307 (ph) 202-737-6410 (fax) omalley@heinzctr.org Representative publications/products http://www.heinzctr.org/ecosystems/index.htm THEME IV: PUBLIC HEALTH AIDS International Training and Research Program (AITRP) Kenneth Bridbord Fogarty International Center The mission of the Fogarty International Center (FIC) of the National Institutes of Health (NIH) is to promote and support scientific research and training internationally to reduce disparities in global health. The longest standing FIC program designed to achieve this goal is the AIDS International Training and Research Program (AITRP). AITRP was initiated in 1988 to respond to what was believed even at that time to represent a global health emergency, which necessitated an unprecedented level of international scientific cooperation. AITRP operates through grants to U.S. universities, which establish long term collaborations with scientific and public health institutions in one or more developing countries. AITRP provides long and short-term training opportunities in the U. S. and short-term in-country training opportunities for developing country scientists. Since its inception nearly 2,000 foreign scientists have received training in the U.S. AITRP’s are strongly linked to NIH-supported research in the home country of trainees, which has been vital to its success, allowing trainees to find career opportunities and to use their newly acquired skills to help their country combat HIV/AIDS. Another key feature of AITRP has been its flexibility as well as the provision of advanced in country research support for trainees upon completion of their formal training. Today AITRP involves two dozen awards to U.S. universities, which are active in more than 60 developing countries. Many of the leading developing country health scientists involved in NIH international AIDS Research Programs, in awards form the Global Fund to Combat AIDS, TB and Malaria, as well as awards from the Bill and Melinda Gates Foundation and the Elizabeth Glaser Pediatric AIDS Foundation have received training through AITRP. Most of NIH- supported HIV/AIDS research in developing countries, e.g., the Vaccine Trials Network, the

102 LINKING KNOWLEDGE WITH ACTION Prevention Trials Network, the Popular Opinion Leader Studies and the CIPRA program, relies on foreign collaborators trained through AITRP. One measure of the impact of AITRP is that 25% of all of the presentations at the last two international AIDS conferences in Durban and Barcelona were authored or co-authored by current or former AITRP trainees. Contact Information Kenneth Bridbord, M.D., MPH Director, Division of International Training and Research Fogarty International Center National Institutes of Health 31 Center Drive Room B2C39, Building 31 Bethesda, MD 20892-2220 Phone: (301) 496-2516 Fax: (301) 402-0779 e-mail: Ken_Bridbord@nih.gov Centers for Disease Control and Prevention (CDC) TB Genotyping Network (The first phase involved a five year pilot study, the second universal implementation) Chris Braden 1. Problem definition Traditional methods in surveillance and outbreak investigation have been insensitive in detecting, monitoring and studying the emergence of pathogens or pathogen strains and new modes of transmission. In the U.S., a major TB epidemic emerged in the late 1980s, with a 20% increase in the number of cases nationally by 1992. The TB rates in New York City advanced beyond those seen in the most severely affected developing countries—an epidemic fueled by lethal strains of multidrug-resistant TB. Traditional investigations were unable to identify the sources and circumstances of infections for a large proportion of TB cases. Unidentified sources of infection meant TB continued to be spread in the communities. Laboratory scientists, epidemiologists, and decision makers responsible for public health and TB control were all aware of the seriousness of the problem. By 1992, TB genotyping was shown to accurately discriminate among strains of TB. By comparing isolates from among multiple TB patients, one could determine which ones were closely related, and thus share the same source. Investigation of the relationships among the patients with related isolates could then identify sources and circumstances of infections. The question remained, could this be done on a large scale and what was the overall benefit to universal application of this technology? 2. Program management The National TB Genotyping and Surveillance Network was established as a pilot research project involving 7 state departments of health and 7 matched genotyping laboratories at departments of health or universities. The study sites were funded through CDC cooperative agreements and a protocol was developed, which received human subjects research exemption.

APPENDIXES 103 The participants had performance goals and targets pertaining to quickly identifying cases and obtaining isolates, conducting interviews, performing the genotyping, supplying results, etc. The outcomes were not set as quantitative goals but rather descriptive objectives- the reason for the research was to see what quantitative outcomes might be achieved by the process. The objectives of the research were to: 1. Determine the relative frequency of TB strains in specific geographic areas 2. Determine the extent of spread of related TB strains in communities 3. Describe the geographic mobility of TB strains and the mode in which they spread 4. Determine the relatedness of TB strains in patients and determine high risk of TB through conventional epidemiologic studies 5. Develop the capacity of local TB controllers to identify patients with related TB strains who deserve careful investigation, and compare the results to those of traditional investigations. 6. Assess the use of TB genotyping in guiding TB control activities. If TB control is successful, then fewer patients should have isolates that cluster by genotyping analysis. 3. Program organization In this case, the users are long time collaborators with and fund recipients of the CDC’s Division of Tuberculosis Elimination (DTBE). Some of the local sites had DTBE employees working full time as public health advisors (there are over 300 CDC TB public health advisors assigned to state public health departments around the country). One independent professional organization, the National TB Controllers Association, provided a conduit for communication and external review that was very useful. 4. The decision-support system The main elements of the TB genotyping project are the results of laboratory analysis of TB isolates from patients and the epidemiologic investigation based on those results. The laboratorians and the epidemiologists are rather distinct groups, often geographically separated, with a long history of poor communications. Some epidemiologists considered the best scientific method to include “blinding” the laboratorians to the details of the patients of the outcomes of investigation, lest the laboratorians be biased in their analysis of genotype patterns. Overcoming this obstacle such that all participants were sharing information took constant effort through multiple project officer site visits, objectives for internal conferences, and annual meetings whereby the results of good communications could be shared as examples.

104 LINKING KNOWLEDGE WITH ACTION 5. Learning orientation The program was set up as a research study. We did not know what the outcome would be and participants were eager to learn and apply the best genotype interpretation and epidemiologic investigation decisions, which necessarily changed with experience. The program received informal, internal CDC evaluation, and at the project period end, a working group established by the National TB Controllers Association reviewed results to provide guidance to TB controllers generally in the use of genotyping in TB control. I consider the project to have suffered from little external review, however. Ultimately, success was based on the ability to meet the objectives, disseminate results and impact TB rates in the study localities. Probably the most difficult problems were administrative- participants falling behind in their investigations, slow or confusing genotype results from laboratories, and poor communications. The original cooperative agreement mechanism for funding made it difficult to fund based on set requirements. In the second phase for universal implementation, contracts have been established for genotyping laboratories. 6. Continuity and flexibility The pilot study was funded through cooperative agreement over a 5-year study period for both laboratory and surveillance sites. This funding was subject to available funds of the Division of TB Elimination. No other direct federal funding source was available, though some academic laboratories also had NIH funded projects. State funds were also applied to this project in the form of human resources as these were people responsible for TB control in the participating states. In phase 2, new methodologies allow high throughput at just two laboratories covering the whole country and ultimately responsible for genotyping about 10,000 isolates a year. These laboratories operate under contract with CDC. State epidemiologists currently do not receive federal funds specifically for this activity, though TB control programs receive state and federal support for general surveillance and investigative capacity. 7. Other insights? Success builders: 1. The project must be based on sound scientific theory with demonstrable impact- it’s what people can believe in. 2. People must be acknowledged, especially those who may not often receive acknowledgment. They should be given the chance to present the fruits of their toil at meetings and conferences and publish their results. 3. People must feel that their career is enhanced, both by personal satisfaction and on their resume. 4. Communications need to be enhanced at every opportunity. One of the best is a general meeting of participants, face-to-face and sharing experiences and problems in an encouraging atmosphere. Communications are also enhanced by a communicative project officer.

APPENDIXES 105 5. An energetic leader who listens, but is also not afraid to direct, is critical. Part of the direction is to keep the objectives focused on outcomes rather than process. Failure signs 1. Starting too ambitious and big. Make sure what you start can be administratively well managed and has the best chances for success. 2. Trying to grow and implement without adequately demonstrating impact in the right way and to the right people, leading to poor support. 3. A very narrow source of funding support. 8. Other issues? Question: Are there any other issues that you would like to discuss during the workshop? 9. Contact information CDC TB Genotyping Network Dr. Lisa Rosenblum Centers for Disease Control and Prevention MS-E10 1600 Clifton Road Atlanta, GA 30333 Ph: 404-639-8116 10. Representative publications/products CDC TB Genotyping Network Castro KG, Jaffe HW. Rationale and methods for the National Tuberculosis Genotyping and Surveillance Network. Emerg Infect Dis [serial online] 2002 Nov [date cited];8. Available at: [http://www.cdc.gov/ncidod/EID/vol8no11/02-0408.htm] Crawford JT, Braden CR, Schable BA, Onorato ID. National Tuberculosis Genotyping and Surveillance Network: design and methods. Emerg Infect Dis [serial online] 2002 Nov [date cited];8. Available at: [http://www.cdc.gov/ncidod/EID/vol8no11/02-0296.htm] Ellis BA, Crawford JT, Braden CR, McNabb SJN, Moore M, Kammerer S, et al. Molecular epidemiology of tuberculosis in a sentinel surveillance population. Emerg Infect Dis [serial online] 2002 Nov [date cited];8. Available at: [http://www.cdc.gov/ncidod/EID/vol8no11/02-0403.htm] Braden CR, Crawford JT, Schable BA. Quality assessment of Mycobacterium tuberculosis genotyping in a large laboratory network. Emerg Infect Dis [serial online] 2002 Nov [date cited];8. Available at: [http://www.cdc.gov/ncidod/EID/vol8no11/02-0401.htm]

106 LINKING KNOWLEDGE WITH ACTION The TB Genotyping program application instructions and users guide (available through contact listed previously). NASA Earth Science Results for Public Health Surveillance: Robert Venezia NASA 1. Problem definition The program integrates NASA Earth science results into public health surveillance systems. NASA and Centers for Disease Control and Prevention (CDC) officials dialogued for nearly three years to match Earth science results with public health surveillance needs. Earth scientists and aerospace engineers met with epidemiologists and public health policy makers to explore requirements. The primary difference between the initial formulation and the final problem definition reflected the difference between science and operations or "public health practice." The initial formulation considered interesting public health science questions that could be addressed using Earth science results. However, what was needed was the ongoing, systematic collection, interpretation, and analysis of data for health events. This is not research. 2. Program management The program sought existing decision support systems or those under construction by the public health practice community. These systems became the focus of the program or the "project." The goals of these efforts were to enhance the descriptive and predictive capabilities of the surveillance systems using NASA Earth science results. The public health practice community set the performance measures for descriptive and predictive capability after discussing the strengths and limitations of the Earth science data for potentially doing so. In one project with the CDC, enhancements to the decision support system were driven by an MOU with NASA. Congress mandated the system addressed by this MOU and maintained interest in the collaboration. 3. Program organization Boundary conditions were spanned by addressing only recognized public health priority subjects. For example, it would have been interesting to study several mosquito-borne diseases using Earth science results. However, asthma and air pollution proved to be more appropriate subjects based on documented morbidity, mortality, lost economic productivity and research spending by the public health community. Users and producers could readily agree to focus in these areas. 4. The decision-support system The program did not develop the decision support system. In fact, the key point is that NASA Earth science results are merely enhancing one owned and operated by another agency and community. NASA's role will be to provide data and observations (ironically a "single piece of the chain") to describe the attributable risk of disease from environmental factors. If that attributable risk is 40%, then NASA will have contributed to understanding that 40% of the cause of the disease in question. The challenge was to integrate those data and observations into a decision support system that was not originally designed to handle that type and amount of

APPENDIXES 107 information. Data pipelines between global change producers and public health decision-makers simply did not exist. 5. Learning orientation Meeting deadlines in delivering components of the system to our CDC colleagues is one critical measure of success. Another will be implementing technologies on the NASA side that have proven track records. Managing CDC’s expectations will also be very important. To do this, both sides require a solid understanding of what each will do. 6. Continuity and flexibility The program is fortunate to be driven by congressional mandate. Therefore, NASA's partners in the effort have relatively secure funding streams. The program seeks to bridge three disparate disciplines (Earth science and aerospace with public health). Curriculum development and interdisciplinary research are encouraged at the national level. NASA is working with the Association of Schools of Public Health, the American Public Health Association, and other public health academic leaders to address the issue. NASA and NCAR are co-sponsoring a summer institute for graduate students interested in linking climate change science with public health. 7. Other insights? At some point, those responsible for action must be made aware of the wealth of pertinent knowledge. At the same time, those responsible for generating that knowledge, must recognize that it is not available to those who need it for decision-making in a timely manner and in a readily useful format. Contact information Dr. Robert Venezia Program Manager, Public Health NASA Office of Earth Science robert.a.venezia@nasa.gov 202-358-1324 Representative publications/products a. NASA’s Earth Science Applications website: [http://earth.nasa.gov/eseapps/] b. NASA’s Public Health Program: [http://earth.nasa.gov/eseapps/theme11.htm] c. NASA’s Earth Science for Society Brochure, which discusses all 12 of our applications areas: Please go to the drop box at [http://ese-dropbox.hq.nasa.gov/ese-dropbox/] and click on “Science for Society brochure”

Next: APPENDIX B: Workshop Agenda, »
Linking Knowledge with Action for Sustainable Development: The Role of Program Management: Summary of a Workshop Get This Book
×
Buy Paperback | $46.00 Buy Ebook | $36.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

This report summarizes a workshop organized by the National Academies’ Roundtable on Science and Technology for Sustainability. The workshop brought together a select group of program managers from the public and private sectors to discuss specific cases of linking knowledge to action in a diverse set of integrated observation, assessment, and decision support systems. Workshop discussions explored a wide variety of experiments in harnessing science and technology to goals of promoting development and conserving the environment. Participants reflected on the most significant challenges that they have faced when trying to implement their programs and the strategies that they have used to address them successfully. The report summarizes discussions at the workshop, including common themes about the process of linking knowledge with actions for sustainable development that emerged across a wide range of cases, sectors, and regions.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!