2
THE ROLE OF COLLABORATIVE, USER-DRIVEN DIALOGUE IN LINKING KNOWLEDGE WITH ACTION2

This chapter highlights one of the most ubiquitous and important features of programs that successfully link knowledge with action: collaborative, user-driven dialogues. In particular, it explores the role of user-producer dialogues, the boundary organizations that facilitate such dialogues, and the importance of user-driven problem definition and ongoing user-driven dialogue.

The Knowledge-Action Supply Chain

Linking knowledge from research and development systems with action for sustainable development is not a simple process, such as one that requires a single step from basic science to end use. Efforts to link knowledge with action entail undertaking some R&D in response to articulated needs of decision makers, rather than only in response to interests of researchers. It has proven difficult to ensure that research informs decisions, even in circumstances where a system is developed explicitly with the goal of affecting decisions, such as some decision-support systems; for example, one workshop participant pointed out: “Commonly (computer-based decision-support systems) are developed by software engineers based on what they think the end user needs or wants. Consequently, these systems are often not used by the intended user. Decision-support systems, predictive models, and other forms of scientific information, when used to inform a collaborative process, can be thought of as aids to the conversation that occur as part of the multiparty negotiation.” Systems that successfully link knowledge with action tend to involve various groups in the conversation about research priorities, including knowledge producers (e.g., climate scientists, engineers, or economists); knowledge users (decision makers, such as city managers, farmers, consumers, or politicians (e.g., those who ultimately take action or make the decisions that initiate action); and program managers who often bridge those two groups, attempting to ensure that what the knowledge producers develop assists the users in making their decisions and in taking action.


Successful programs tend to involve end-to-end integrated systems that connect basic scientific predictions or observations (e.g., a forecast of higher probability of drought) to outputs directly relevant to decision making (e.g., implications of changed crop outputs for national balance of payments), often involving a number of intermediary analytic steps (e.g., converting lower rainfall into likely crop outputs). For example, a fairly sophisticated end-to-end system described by one participant included products ranging from: “global ENSO3 forecasts, using state-of-the-art climate models, through higher resolution regional forecasts co-produced with local forecasting entities, to fairly localized, practical forecast products that incorporate input from potential users of the information.”

2  

This section draws heavily on the background material for the workshop, including materials supplied by the participants.

3  

ENSO is the El Niño-Southern Oscillation a “continual but irregular cycle of shifts in ocean and atmospheric conditions that affect the globe” (see http://www7.nationalacademies.org/opus/elnino_PDF.pdf).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop 2 THE ROLE OF COLLABORATIVE, USER-DRIVEN DIALOGUE IN LINKING KNOWLEDGE WITH ACTION2 This chapter highlights one of the most ubiquitous and important features of programs that successfully link knowledge with action: collaborative, user-driven dialogues. In particular, it explores the role of user-producer dialogues, the boundary organizations that facilitate such dialogues, and the importance of user-driven problem definition and ongoing user-driven dialogue. The Knowledge-Action Supply Chain Linking knowledge from research and development systems with action for sustainable development is not a simple process, such as one that requires a single step from basic science to end use. Efforts to link knowledge with action entail undertaking some R&D in response to articulated needs of decision makers, rather than only in response to interests of researchers. It has proven difficult to ensure that research informs decisions, even in circumstances where a system is developed explicitly with the goal of affecting decisions, such as some decision-support systems; for example, one workshop participant pointed out: “Commonly (computer-based decision-support systems) are developed by software engineers based on what they think the end user needs or wants. Consequently, these systems are often not used by the intended user. Decision-support systems, predictive models, and other forms of scientific information, when used to inform a collaborative process, can be thought of as aids to the conversation that occur as part of the multiparty negotiation.” Systems that successfully link knowledge with action tend to involve various groups in the conversation about research priorities, including knowledge producers (e.g., climate scientists, engineers, or economists); knowledge users (decision makers, such as city managers, farmers, consumers, or politicians (e.g., those who ultimately take action or make the decisions that initiate action); and program managers who often bridge those two groups, attempting to ensure that what the knowledge producers develop assists the users in making their decisions and in taking action. Successful programs tend to involve end-to-end integrated systems that connect basic scientific predictions or observations (e.g., a forecast of higher probability of drought) to outputs directly relevant to decision making (e.g., implications of changed crop outputs for national balance of payments), often involving a number of intermediary analytic steps (e.g., converting lower rainfall into likely crop outputs). For example, a fairly sophisticated end-to-end system described by one participant included products ranging from: “global ENSO3 forecasts, using state-of-the-art climate models, through higher resolution regional forecasts co-produced with local forecasting entities, to fairly localized, practical forecast products that incorporate input from potential users of the information.” 2   This section draws heavily on the background material for the workshop, including materials supplied by the participants. 3   ENSO is the El Niño-Southern Oscillation a “continual but irregular cycle of shifts in ocean and atmospheric conditions that affect the globe” (see http://www7.nationalacademies.org/opus/elnino_PDF.pdf).

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop Communication in the form of ongoing dialogues is needed among producers, users, and program managers. In the absence of such dialogues, suggested one participant, “the S&T community often persists in offering its newest nanoswitches, while decision makers keep asking for old-fashioned hammers, and no one figures out that superglue would do the job at hand better than anything else.” Setting up and maintaining effective user-producer dialogues along the whole “supply chain” from basic research through decision making can impose strains on both scientists and decision makers. One participant explained that carrying out those dialogues in science-based organizations often leads to perceived capture of the dialogue by science, leading to the pitfall of science-push solutions that are irrelevant to action. On the other hand, carrying them out in operational or political contexts often leads to the perceived capture of the dialogue by politics, leading to the pitfall of politics-pulled solutions that are disowned by science. This leads to the important role of program managers and the boundary organizations within which they operate in promoting effective dialogues between knowledge producers and users. The Importance of Program Managers and Boundary Organizations Program managers and boundary organizations that successfully link knowledge with action tend to bridge both the barriers that separate disciplines and those that separate knowledge production and application. Many of the program managers at the workshop either work for a boundary organization or work to strengthen systems for linking knowledge with action that involve other boundary organizations. A few brief descriptions of some of those organizations are included in Box 2-1 as examples. More detailed descriptions of the programs represented at the workshop and how many of them serve as boundary organizations are included in the case summaries in Appendix A. BOX 2-1 Examples of Boundary Organizations* The H. John Heinz III Center for Science, Economics and the Environment: State of the Nation’s Ecosystems Project The Heinz Center’s State of the Nation’s Ecosystems project was designed to develop and report on an agreed-upon suite of indicators describing the key characteristics of the United States’ ecosystems. Reporting on the state of the nation’s ecosystems requires communicating complex information in a manner that is accessible to nonspecialists while maintaining the scientific integrity of the information. For this and other reasons, the Heinz Center used a process involving participants from business, environmental advocacy organizations, academic institutions, and federal, state, and local governments. These groups served on design committees and working groups that were structured to ensure strong links and open dialogue among the members. Examples of areas in which the report was shaped by the different viewpoints of these communities are the number of indicators and the tone, technical content, and amount of supporting information provided in the report, and the degree to which the report was dominated by indicators that are already well known by the public or included those that are seen as important by the ecological community but are not well known by nonspecialists. The Heinz Center program served as a forum for direct dialogue, bringing users and producers together to jointly design and implement the project.

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop The National Oceanic and Atmospheric Administration’s Regional Integrated Sciences and Assessment Program The problem to be solved that led to the creation of the Regional Integrated Sciences and Assessment (RISA) Program was that NOAA’s Climate and Global Change Program lacked integration and the ability to connect well (and by design) to issues faced by decision makers (whose problem was rarely framed as climate). The program was launched in order to define problems or challenges for which climate information and data might be useful to decision makers. Each set of investigators within a region was asked to design a research agenda in partnership with stakeholders in their particular regions. In their longer-term collaboration, the investigators experimented with public forums, regular and sustained meetings, proactively seeking opportunities to participate in technical or professional meetings, disseminating material through websites and targeted publications, identifying research partners who sit in resource management agencies, and a range of other techniques. Through this process, each RISA project has developed its own version of end-to-end integrated systems and its own region-specific decision-oriented research agenda. The approach to user-driven dialogue in each RISA case was designed and implemented by individual teams in close collaboration with users from a specific region, resulting in uniquely tailored research questions and approaches to answering those questions. The U.S. Agency for International Development’s NetTel@Africa: Informing the Telecommunications Regulatory Process Africa in the mid-1990s was failing to advance into the Internet age. National regulatory authorities were ill equipped to judge the merits of emerging technologies, economic models, and legal structures that might eventually support the widespread adoption of promising new technologies. These authorities asked for assistance. The NetTel@Africa Program offered the opportunity to link academic, legal, and other technical experts to national regulators through university programs within Africa. The knowledge producers were a partnership among African and U.S. universities, as well as among African and U.S. regulatory practitioners. The users were national regulatory authorities. The boundary-spanning organization was a nonprofit center based at one of the U.S. universities, and in particular a program manager within that center. Project formulation took place in what might be called alliance mode, where the program manager assisted a diverse set of potential alliance members to articulate a common goal. Then, through a process of iterative consultations, consensus building, and workshops, the alliance was formed, with specific roles for each alliance member, and near- and intermediate-term objectives. The program’s discrete elements are: (1) identification of best practices for regulatory policy formulation and implementation; (2) understanding the political and economic context in which African regulators operate; (3) development of appropriate curricula and certification programs with feedback mechanisms for program enhancement; and (4) regulators putting their acquired principles and techniques into practice. The alliance between regulators (practitioners) on the one hand and academic experts on the other, places a premium on examining whether the approaches to regulation are effective in achieving their public policy purposes.

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop NetTel is a partnership, with substantial resources contributed by the participating universities and regulatory bodies on both sides of the ocean. It devoted substantial time to implement the initial phase of the collaborative process (almost 18 months). Houston Advanced Research Center: Informing the Development of Clean Air Policy in Houston Public policy leaders in the Houston-Galveston area needed better scientific information upon which to base policy decisions regarding compliance with federal clean air standards. The Houston Advanced Research Center (HARC) was designated to manage research with the aim of improving air quality models, model inputs, and understanding of ozone formation in an area of unique geography, climate, industry, and transportation. The process was designed to allow for the delivery of research results to a variety of stakeholders on the board of the Texas Environmental Research Consortium (including stakeholders with diverse interests in business, health, environment, and local government). Each board member could use the information as he or she pleased. The process also delivered research results to the Texas Commission on Environmental Quality. In addition, the research results were posted on a public website. HARC served as a facilitator between researchers and users, communicating priorities, needs, feedback, research results, and other pertinent information. *   These examples are adapted from the case summaries in Appendix A. More detail on these examples and additional examples are included in that appendix. The concept of boundary organizations as used here is developed in Hellstrom and Jacob (2003), Guston (2001), and Jasanoff (1990). Although the details of the design and roles of the boundary organizations represented varied substantially, a general model was broadly applicable to many of the programs discussed at the workshop. Figure 2-1 is a slightly modified version of a diagram provided by a workshop participant and is a fairly accurate representation of this generalized model of boundary organizations.4 The diagram places the boundary organization and some of its typical characteristics and roles in the center. On each side are the groups it bridges: the producers of knowledge to the left and the users of information to the right. The producers and users are brought together either directly or indirectly by the boundary organization or program manager who provides a critical link between the two. The diagram is appropriately not sequential; there is and should be an ongoing, tangled back-and-forth among the various groups involved (Cash and Buizer, 2005). Boundary organizations vary somewhat in approach but often share features or strive toward such features as nonpartisanship, experimental orientation, coproduction of information, and user-driven approaches. 4   This figure is a slightly modified version of a model Todd Mitchell submitted (see Appendix A), informed in part by task force discussions, as a representation of his organization, HARC. The figure is a good representation of boundary organizations more generally.

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop FIGURE 2-1 The positioning of a boundary organization in an end-to-end system. As Figure 2-1 illustrates, an end-to-end system requires all the necessary components to exist and be connected with one another, including necessary input and feedback loops. Although one organization does not provide all pieces of the chain to ensure that knowledge leads to action, it is important to consider issues such as whether all the pieces exist, whether they are adequately connected, how various organizations will fit into that system, and who will manage the supply chain and how. Some program managers expressed hesitation regarding end-to-end systems, emphasizing that they should be responsible only for their parts of the system, preferring the idea of science-to-policy handoff, meaning the scientists understand what information is needed to make the decisions but do not make the decisions themselves. Science-to-policy handoff, as described by some participants, and end-to-end systems as discussed more generally, are not mutually exclusive, however. One participant pointed out that in an end-to-end system, researchers do not try to make the decisions, nor should one organization provide all the parts of the system. Instead, decision makers understand that they will likely receive the information they need but not necessarily the scientific outcome they would like. Many participants emphasized that although all of the pieces of the chain need to be in place, it is important to respect boundaries between the different parts of the chain. Different groups provide different pieces, with some systems being more complex than others, requiring more or less tailoring depending on the decisions to be made. An effective program manager does not assume that all pieces of the chain will be addressed in his or her program. Instead, he or she ensures that all the pieces are in place and are being addressed by the required organizations. These systems or supply-chain perspectives on the design of decision-support systems are critical to assuring that no crucial link is missing or mismatched. Defining the Problem Workshop participants suggested that programs seeking to link knowledge with action are more likely to be effective when program managers and other parties strive first to understand both the problem and what information decision makers need in order to develop a solution. To ensure

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop that a program is decision relevant, dialogue and cooperation between the scientists and engineers (producers) who ultimately create new knowledge and the decision makers (users) who ultimately use it is critical. The problem to be solved, which ultimately determines the focus and approach of the program, needs to be defined collaboratively but ultimately be user-driven (decision-maker-driven). One program manager described the gradual realization within her program of the importance of working with users to frame the problem, the questions to be asked, and the program approach if a program is to be decision relevant. The role of stakeholders … evolved over time, through an adaptive learning and management process. Recognizing the potential applications of climate research during the late 1980s, (we) initiated a series of workshops on behalf of the research community. Participation in this applications dialogue was initially focused on the physical scientists involved in understanding the dynamics of the climate system, and the creation of observations systems and models to support this work. During the early 1990s, (we) began to realize that the internal focus of these discussions was not likely to lead to the realization of any socioeconomic applications and value, so the program management staff began to purposefully incorporate an increasingly wider range of participants in this dialogue. The expansion began with social scientists, to help articulate the impact of climate on society and to begin to understand the potential applications of climate information in decision making (including opportunities and barriers related to its use). While the inclusion of another type of academic was useful, we soon realized that actual decision makers needed to be at the table to help frame their challenges and information needs, and to participate in a “negotiation” with the scientific community about what was desirable and feasible.5 Such dialogues can lead to the framing of the problem to be addressed in a new, more decision-relevant way; for example, rather than having a broader, more traditional research agenda in which understanding the phenomenon of climate change is the focus, some programs evolved to have a region-specific, impact-oriented problem definition and focus. One workshop participant explained: “In the (Southeast), the problem was defined in terms of the vulnerability of important crops. In the Pacific Northwest, the problem was variations and changes in water supply. In California, one prominent stakeholder-defined issue is the restoration of the San Francisco Bay Delta and the resolution of competing resource demands. In the (Southwest), fire risk and the spread of disease have dominated parts of the research agenda.” Such joint problem definition 5   This program’s gradual inclusion of more groups to articulate the problem to be addressed in a more useful way highlights the need to draw upon a variety of perspectives. Another program manager explained that his program included “boundary spanning between business, environmental advocacy organizations, academic institutions, and government. Each of these major sectors may have both ‘researchers’ and ‘decision-makers,’ but the researchers in each of these four sectors will often have very different perspectives, values, assumptions, and strategic ways of approaching an issue … Inclusion of multiple research perspectives, and multiple decision-maker perspectives, is a crucial design element that will strengthen many programs.” Many workshop participants expressed recognition of the variations within those groups and the need in many cases to draw upon multiple perspectives from those communities, but for the purposes of the workshop discussions and case summaries users and producers are grouped together to highlight the under-recognized need for greater collaboration between the groups.

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop can result in shared ownership of the problem and the research agenda, which in turn can change the entire dynamic of how the problem is addressed. Early and Ongoing User Engagement The importance of a collaborative approach involving researchers, decision makers, and program managers highlights the central role of process in effective systems. One workshop participant noted: “We view decision support as a process, as opposed to a product (particularly a product requiring a specific methodology like integrated end-to-end modeling). In the course of the process, particular approaches and tools that are best suited for answering the questions being asked by decision makers are identified.” Among the programs represented at the workshop, a variety of processes for engaging users were employed. Commonalities that were identified include the need to identify potential users at the earliest stage possible; undertaking a joint exercise for problem definition and other goals very early in the project; and ongoing user engagement. Despite the effort it takes to identify and engage users, many workshop participants emphasized that doing so not only leads to significantly more relevant work that is much likelier to result in action but also improved perceptions of the legitimacy of the process and credibility of the knowledge it produces given the increased transparency and openness that user engagement entails. An important consideration that was mentioned was how to bring together users and producers. Options vary, but one tool that was commonly cited as useful for effective problem definition was a problem formulation exercise, in which users and producers exchange perspectives.6 Several program managers stressed the importance of undertaking a deliberate exercise at the beginning of a project, or ideally before beginning the project, during the problem formulation stage. One participant described problem formulation exercises as useful for researchers to “inform a particular group of users about the best available science on a particular topic” and for the users to then “identify the specific issues and questions of concern to them.” Based on these discussions, an assessment plan is then formulated. This early user engagement was described as important in order to both develop a relevant program and provide time to identify additional users. Several program managers found that it could be helpful to engage users throughout the process, especially in research and the communications stage. A participant explained: “Researchers/assessors and stakeholders are not necessarily distinct communities. In many cases, the stakeholder community can offer data, analytic capabilities, insights and understanding of relevant problems that can contribute to the assessment.” Another participant, who trains other program managers in a user-engagement process called “joint fact finding,” finds that enabling decision makers and stakeholders to have input into the science or to participate in fact finding helps them find common ground. He stated: “Mutual respect and trust are essential to joint fact finding that involves diverse stakeholders. Face-to-face conversation is important.” It builds trust. In addition, one potential complication that some participants seemed particularly concerned about was the potential inequities that can arise when one user group is selected over another; for 6   Although several participants raised the concept of a problem formulation exercise, Joel Scheraga provided the term in his case summary (see Appendix A).

OCR for page 7
Linking Knowledge with Action for Sustainable Development: The Role of Program Management - Summary of a Workshop example, if a certain group is engaged in identifying what information is needed and/or they are the only group to receive such information, does it benefit them to the disadvantage of other groups? Participants pointed out that although it is important to try to identify all of the appropriate users early on, program managers should expect to learn of additional users later and should design programs so that there are numerous opportunities to engage additional users throughout the project; this occurs when those involved gain a better understanding of the various groups that have a stake in their work. Adaptively including more users as more is learned can help avoid longer-term inequities and can ensure that the most appropriate users are engaged. Benefits of Collaborative, User-Driven Dialogue In brief, discussions at the workshop suggested that collaborative, user-driven dialogues could help identify: What problem needs to be addressed: For example, moving from the more general “reduce uncertainty about climate change feedback loops” to the more decision-specific “improving predictions of variations in the water supply for a region”; What information users need to address a problem, what producers can offer, and how those two converge: In many cases, the user comes to a different understanding of what he might need and the producer comes to a different understanding of what he can offer or how he should offer it. How that information should be communicated: According to one participant: “If the purpose of the effort is to convey insights to decision makers, communication during the problem formulation stage is important to ensure that useful assessment endpoints are identified and pursued. Not only should information needs be identified but analysts should also understand how and when stakeholders would use assessment information. Will end users find and read a scientific journal article? Would they prefer a tool or model to help them evaluate and employ assessment results? If the audience is the public, is it best served by a pamphlet that simply and accurately relates the findings? Understanding the audience’s ultimate needs shapes the communications strategy.” How the local context varies: For example, practices that have proven successful in the United States might not prove as effective in other contexts. One participant explained: “Addressing (problems associated with poverty) requires the application of engineering knowledge and resources in a developing-world context. Solutions must be practical [and be] implemented and maintained using available skills and resources, and consistent with local culture and customs. These facets of engineering are not taught in schools nor are they acquired in an engineer’s normal career experience. Furthermore, the practical ‘low-tech, high-content’ technologies needed to solve these problems do not receive much attention.”