What are the general attributes of such relatively effective systems? Workshop participants discussed a hypothesis based on earlier work that systems for linking knowledge to action are more likely to be effective to the extent that they produce information that is perceived by users to be simultaneously salient, credible, and legitimate (Cash et al., 2003; Clark et al., in review). These three terms are not precisely defined but have been differentiated as follows:
Salience relates to the perceived relevance of information: Does the system provide information that decision makers think they need, in a form and at a time that they can use it? For example, farmers in some areas need to know something about the timing of first rains, as opposed to average expected precipitation over a season; or emergency preparedness managers need to have forecasts early enough to start preparations for potential natural hazards.
Credibility addresses the perceived technical quality of information. Does the system provide information that is perceived to be valid, accurate, tested, or, more generally, at least as likely as alternative views to be “true”? For example, do other scientists agree with underlying assumptions of a model? Or does ground-truthing of general research reveal that the general findings hold in a specific place?
Legitimacy concerns the perception that the system has the interests of the user in mind or, at a minimum, is not simply a vehicle for pushing the agendas and interests of other actors. The term fairness has been used to characterize legitimacy, but was felt by some workshop participants to convey an overly negative or suspicious view of the system. Questions about legitimacy often take the form of concerns about process or the peer group that the forecaster belongs to. For example: Who is involved in producing the knowledge? How were those involved selected? When and how are stakeholders engaged? How are R&D agendas set?
As described in the paper, the challenge of designing effective systems for linking knowledge and action—systems that produce information that is perceived to be salient, credible, and legitimate—is complicated by two aspects of the linkages among these criteria (or dimensions). First, it appears that if a system is perceived to be seriously lacking on any one of these dimensions, its likelihood of producing influential information falls significantly. (In other words, no amount of investment in, say, credibility, will make up for a serious shortfall in salience.) Second, it appears that the attributes of salience, credibility, and legitimacy are tightly linked: efforts to enhance one may either enhance or degrade another, depending on the circumstances and the strategies used. For example, greater involvement of stakeholders may increase salience (the right questions are asked) and legitimacy (a more transparent process ensues), but credibility might decrease (the science may appear to be politicized).
The art of designing effective systems for linking knowledge to action thus can usefully be viewed as the art of designing institutions (processes, organizations, norms) that balance such tradeoffs in ways that produce information perceived by users to meet simultaneously at least minimum standards of saliency, credibility, and legitimacy. How this has been done in efforts to bring knowledge about climate and its variability to bear on practical problems around the world was the focus of most of the comparative analysis conducted at the Workshop. Lessons learned emerging from those comparisons are summarized in the following chapter “Components of Effective Systems.”