Chapter 4

Development of a Decision Framework

THE NEED FOR AND VALUE OF A DECISION FRAMEWORK

The preceding chapters identified the need for a consistent decision framework that can be used to strengthen sustainability linkages. Drawing from a number of the fact-finding examples and the literature, the committee identified the common elements of an effective decision framework, which form the basis for the framework presented here.

Decision frameworks provide a way to facilitate and enhance decision making by providing conceptual structures and principles for integrating the economic, social, ecological, and legal/institutional dimensions of decisions. Their application can result in consistent and effective results. Decision frameworks refer to principles, processes, and practices to proceed from information and desires to choices that inform actions and outcomes (Lockie and Rockloff, 2005).

While decision frameworks vary in design and purpose, they generally have common elements that include:

Problem identification and formulation,

Identification of clear goals,

Illumination of key questions that help decision participants scope problems and management options,

Processes for knowledge-building (including scientific, technical, experiential, and cultural knowledge) and application of appropriate analytical tools to assess actions, options, trade-offs, risks, and uncertainties,

Connection of authorities tasked with making decisions to outcomes associated with those decisions.

In addition to these common elements, decision frameworks generally provide transparency about goals, information, and decision processes; inclusiveness of relevant participants; and structures or processes to adapt decisions over time in response to new goals, changing circumstances, or new knowledge.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 70
Chapter 4 Development of a Decision Framework THE NEED FOR AND VALUE OF A DECISION FRAMEWORK The preceding chapters identified the need for a consistent decision framework that can be used to strengthen sustainability linkages. Drawing from a number of the fact-finding examples and the literature, the committee identi- fied the common elements of an effective decision framework, which form the basis for the framework presented here. Decision frameworks provide a way to facilitate and enhance decision making by providing conceptual structures and principles for integrating the economic, social, ecological, and legal/institutional dimensions of decisions. Their application can result in consistent and effective results. Decision frame- works refer to principles, processes, and practices to proceed from information and desires to choices that inform actions and outcomes (Lockie and Rockloff, 2005). While decision frameworks vary in design and purpose, they generally have common elements that include:  Problem identification and formulation,  Identification of clear goals,  Illumination of key questions that help decision participants scope problems and management options,  Processes for knowledge-building (including scientific, technical, expe- riential, and cultural knowledge) and application of appropriate analytical tools to assess actions, options, trade-offs, risks, and uncertainties,  Connection of authorities tasked with making decisions to outcomes associated with those decisions. In addition to these common elements, decision frameworks generally provide transparency about goals, information, and decision processes; inclu- siveness of relevant participants; and structures or processes to adapt decisions over time in response to new goals, changing circumstances, or new knowledge. 70

OCR for page 70
Development of a Decision Framework 71 In this chapter, the principles that form the basis for the decision frame- work the committee recommends are first articulated, followed by the frame- work itself. Recommendations concerning its implementation and use are also presented. PRINCIPLES The decision framework described in this chapter was developed to be:  Flexible and scalable to a wide range of complex sustainability issues  Based on the broad and diverse literature and practice of effectively and widely used frameworks  Inclusive of the major elements of such frameworks As illustrated in the examples addressed in this report, any framework must be flexible enough that it can be applied to a broad range of sustainability linkage challenges. Consequently, for the decision framework presented here to be broadly useful, it must be sufficiently flexible to be adapted to a wide range of applications. As also illustrated in this report, sustainability linkage applica- tions vary both temporally and geographically. Consequently, the decision framework must also be scalable. A broad and diverse literature and significant practical experience with decision frameworks exist (see Box 4-1). This literature and experience provide the foundation for describing an effective and broadly applicable decision framework.1 Moreover, the committee has concluded that this literature and ex- perience are broadly applicable to the examples considered and evaluated in this report. The decision framework as applied to sustainability linkages must also in- clude the major elements of relevant frameworks. These generally include the following elements:  Agreement on the problem or issue and its scope  Agreement on objectives and goals  Agreement on “who’s at the table”  Engagement of all relevant stakeholders  Capacity building to overcome asymmetries in stakeholder knowledge and resources 1 In addition to the literature cited in Box 4-1, the World Bank has developed guidance for how to design a results framework, defined as “an explicit articulation (graphic dis- play, matrix, or summary) of the different levels, or chains, of results expected from a particular intervention—project, program, or development strategy. The results specified typically comprise the longer-term objectives (often referred to as ‘outcomes’ or ‘im- pact’) and the intermediate outcomes and outputs that precede, and lead to, those desired longer-term objectives” (World Bank 2012).

OCR for page 70
72 Sustainability for the Nation: Resource Connections & Governance Linkages BOX 4-1 Relevant Decision Framework Literature Numerous National Academies reports include frameworks for decision- making on issues ranging from the environment to public health to transportation. Some selected reports include IOM 2010; NRC 1996, 2005, 2008a, b, 2009a, b, 2011a, b, 2012. In particular, NRC 2009b summarizes key issues related to deci- sion support systems and distills six principles that are broadly related to the committee’s framework that characterize these systems, including the benefits of following them. These include:  “Begin with users’ needs. Decision support activities should be driven by users’ needs, not by scientific research priorities. These needs are not always known in advance, and they should be identified collaboratively and iteratively in ongoing two-way communication between knowledge producers and decision makers.  Give priority to processes over products. To get the right products, start with the right process. Decision support is not merely about producing the right kinds of information products. Without attention to process, products are likely to be inferior—although excessive attention to process without delivery of useful products is also ineffective. To identify, produce, and provide the appropriate kind of decision support, interactions between decision support providers and users are essential.  Link information producers and users. Decision support systems re- quire networks and institutions that link information producers and users. The cul- tures and incentives of science and practice are different, for good reason, and those differences need to be respected if a productive and durable relationship is to be built. Some ways to accomplish this rely on networks and intermediaries, such as boundary organizations.  Build connections across disciplines and organizations. Decision sup- port services and products must account for the multidisciplinary character of the needed information, the many organizations that share decision arenas, and the wider decision context.  Seek institutional stability. Decision support systems need stable sup- port. This can be achieved through formal institutionalization, less-formal but long- lasting network building, new decision routines, and mandates, along with commit- ted funding and personnel. Stable decision support systems are able to obtain greater visibility, stature, longevity, and effectiveness.  Design for learning. Decision support systems should be structured for flexibility, adaptability, and learning from experience” (NRC, 2009b).  Mutually negotiated and agreed upon decision rules (e.g., “how much agreement is sufficient to constitute approval”) to ensure perceived legitimacy and accountability (may or may not require unanimity)  Clarification of participant roles, responsibilities, and accountability  Boundary processes/organizations at the intersection of scientists/tech- nical experts and decision makers, managers, and stakeholders

OCR for page 70
Development of a Decision Framework 73  Maintenance of flexibility to adapt to new information and/or changing circumstances  Understanding of structural barriers that could limit success and ways to address them. A DECISION FRAMEWORK Figure 4-1 presents a graphic representation of the decision framework recommended by the committee. The purpose of this framework is to lay out a structured but flexible process, from problem formulation through achievement of measureable outcomes, which engages agencies and stakeholders in goal- setting, planning, knowledge building, implementation, assessment, and decision adjustments. It is designed to be used when addressing place-based sustainabil- ity challenges as well as in policy formulation and rulemaking. The framework incorporates an iterative (or incremental) process that yields solutions to a wide range of issues that vary in scope, characteristics, and time. As an iterative pro- cess, the framework can also be viewed as a learning tool that begins with prob- lem formulation and includes knowledge regarding key drivers and their rela- tionship to key stakeholders, as well as access to scientific knowledge regarding the connections among components of the system. The framework is consistent with and extends the sustainability framework developed for the U.S. Environ- mental Protection Agency (EPA) in the “Green Book” (NAS, 2011a). As per the statement of task, the decision framework presented here will help “examine the consequences, trade-offs, synergies, and operational benefits of sustainability- oriented programs. The decision framework will include social, economic, and environmental dimensions of sustainability.” The framework is depicted in four phases: (1) preparation and planning; (2) design and implementation; (3) evaluation and adaptation; and (4) long-term outcomes. A description of each of the framework elements is given below. The framework is meant to apply to the creation of a sustainability program (an on- going, interagency effort such as a crosscutting program to support sustainable development in cities) and projects (single interagency efforts focused on a spe- cific task, such as a project to design sustainable water use and agricultural pro- duction in the Great Plains Ogallala Aquifer). Phase 1: Preparation and Planning This phase has three major steps that need to occur before the actual pro- gram or project is designed. This important phase and its associated steps are often overlooked or done in an incomplete or piecemeal fashion. The examples and other research done by this committee found that this phase and its elements were critical to the success of sustainability programs and, if not done well, con- tributed to the demise of programs. Because of the importance of this phase, a more detailed view is provided in Figure 4-2.

OCR for page 70
74 FIGURE 4-1 Conceptual Decision Framework. Four phases are shown, along with the relevant steps within each phase. The framework could be applied in creating either programs or projects related to sustainability.

OCR for page 70
FIGURE 4-2 Phase 1 of the decision framework in expanded detail. Each step identified in Figure 4-1 of Phase 1 now includes specific actions and outputs/outcomes for that action (see key). 75

OCR for page 70
76 Sustainability for the Nation: Resource Connections & Governance Linkages Phase 1: Preparation and Planning This phase has three major steps that need to occur before the actual pro- gram or project is designed. This important phase and its associated steps are often overlooked or done in an incomplete or piecemeal fashion. The examples and other research done by this committee found that this phase and its elements were critical to the success of sustainability programs and, if not done well, con- tributed to the demise of programs. Because of the importance of this phase, a more detailed view is provided in Figure 4-2. The steps that need to be taken in Phase 1, and their associated actions and outputs, include: Frame the problem. A sustainability issue of sufficient complexity to war- rant a multi-agency approach is first identified. Issues requiring a coordinated response are those of national significance due to their broad geographic extent, potential to impact long-term health and economic well-being, or crosscutting impact. Next, the issue must be framed so that the problem to be solved is clear- ly understood. This is analogous to problem formulation in human or ecological risk assessment. Effectively framing the problem requires a coordinated effort by an appropriate combination of federal, state, local, tribal, nongovernmental, and/or private-sector entities. An issue may be framed through a number of dif- ferent avenues ranging from engaging key stakeholder partnerships to agency leadership and executive action. All dimensions of the problem must be identi- fied, including the environmental resource connections, societal connections, and economic connections. These elements of the problem will inform the selec- tion of agencies and nonagency organizations that should be involved in the program or project. It is important to note that agencies need not await structural overhauls in order to strengthen their capacity to address sustainability linkages. Agencies can begin by preparing a high-level systems map illustrating key link- ages that can then be deployed widely across federal agencies for any sustaina- bility-related program or project in order to incentivize policy coordination. Some baseline analysis is typically required at this point to generally de- scribe the magnitude of adverse impacts if the issue is not successfully ad- dressed, and the magnitude of the benefits to be gained when it is. An initial estimate of the extent of the effort that might be reasonably expected to address the problem is also useful when framing it. These initial estimates will be re- fined as the decision process proceeds; thus, the process is iterative. An initial group of relevant parties—representatives of at least some of the relevant agen- cies, as well as some of the affected parties and those needed to implement po- tential solutions—are typically engaged at this point to assist with the framing. Some of these individuals often function as champions whose actions can en- gage relevant parties in the next step, as well as get buy-in from key agency ad- ministrators (“champion the cause”).

OCR for page 70
Development of a Decision Framework 77 Identify and enlist stakeholders. The next significant step is to identify the relevant agency linkages. Depending on the natural resources and social and economic aspects of the problem, it will be critical to engage all of the federal agencies affected by it. For example, a project to develop a sustainability plan for the Ogallala Aquifer would require participation by the U.S. Geological Sur- vey (USGS), the Bureau of Reclamation (BOR), the Bureau of Land Manage- ment (BLM), the U.S. Fish and Wildlife Service (FWS), and the U.S. Depart- ment of Agriculture (USDA), as well as states, tribes, and others. This participation also illustrates the highly collaborative nature of the process, which continues throughout. The issue framing conducted during the first step should provide know- ledge as to which organizations and individuals need to be involved and the in- formation needed to engage them. The interpersonal skills of the individuals engaged in the first, problem-framing step become critical in this phase, as they will often not have the positional authority to engage all of the relevant organi- zations and individuals. The initial group must collectively possess sufficient collaborative leadership skills to engage the relevant parties. It may also be nec- essary for them to identify and engage sponsors who have the influence to bring relevant parties to the table, along with necessary resources to support the efforts of the team. At this step of the process, the technical skills and professional ex- pertise needed to design and implement the program or project are identified. Identifying relevant nonagency stakeholders is part of this step as well. Nonagency stakeholders are frequently those who must use or implement the approach or solution developed to address the problem, as well as those impact- ed by the approach or solution. These stakeholders may be individuals or entities at the local, state, tribal, regional, or national scales. They may include nonfed- eral governmental agencies, nongovernmental organizations (NGOs), private- sector interests, or others who have significant interests in the outcome of deci- sions and actions. It is critical that all relevant players be involved; if a repre- sentative of a sector that is a key driver in the issue is missing, the likelihood of success is greatly diminished. In the next step, the actual Project or Program Team (“Team”) is identi- fied. The Team, which may be deployed either to design a sustainability pro- gram or to address a specific sustainability problem at the project level, should include individual representatives from the relevant organizations (“stakehold- ers”) identified during step 2. This group must have the necessary background, experience, and leadership skills to successfully design the project or program. Team members must be carefully selected by their member organizations; they must have the right commitment, expertise, and skill sets, and they must have appropriate authority from their organizations so that their participation leads to success. Sufficient expertise in the fields of environmental science, ecology, social science, economics, and public health should generally be included. Each Team member should be a collaborative leader, and each should add value to the Team. Members must be provided support and resources by their respective organizations. Attention should be paid to the informal and formal relationships

OCR for page 70
78 Sustainability for the Nation: Resource Connections & Governance Linkages that already exist across these organizations, as success can be strongly influ- enced by the trust that exists or is built among Team members. If it has not been done previously, it is essential during this step to deter- mine and specify the role the federal agencies will play relative to the other players. The agencies may be principal leaders, or facilitators, or deal-makers, or they may act as a backstop using their legal authorities, with regional, state, or other participants taking the leadership role. In several examples studied by the committee, federal agencies successfully provided (legal) cover for regional or local programs. Other successful examples highlighted federal agencies in a leadership role. Often it was the scale of the program (city vs. interstate) and willingness of the federal agencies to partner with and engage stakeholders ef- fectively when they were in the position of leadership that contributed to suc- cess. Develop project management plan. The importance of this step, in which the Team develops a management plan for the program or project, cannot be overstated. The plan should clearly delineate the roles, responsibilities, and ac- countability of each member organization or participant, as well as a business plan for the funding of the project design, implementation, and maintenance (thus assuring its longevity). Other partners may be identified at this point whose involvement will be necessary in order to meet the project goals and to balance any asymmetries in the capacity of the Team. This plan should be de- veloped prior to any project design or implementation so as to avoid missing critical pieces and to avoid conflict among players as to who does what. Phase 2: Design and Implementation A more detailed version of Phase 2 is shown in Figure 4-3. Set project goals. The Team establishes goals for the program or project— a step that should be taken with engagement of stakeholders and relevant mem- bers of the public. In addition, the short- and long-term outcomes and their asso- ciated measures are identified, and an evaluation process is developed. A project timeline for measuring and achieving goals is agreed upon. Goal and outcome settings may also inform the partnerships needed to achieve success. Evaluating baseline conditions before implementing a sustainability solution or approach is necessary so that a future evaluation can gauge the impact of the program. Design action plan. Now the Team develops a comprehensive design of the approach, strategies, actions, etc. that are needed to address the sustainability issue and meet the goals established in the previous step. The necessary tools, knowledge, and information to accomplish the goals must be identified and pur- sued. The Team also needs to identify who will implement the plan, how the program will be maintained, and by whom. This plan must include “decision

OCR for page 70
FIGURE 4-3 Phase 2 of the decision framework in expanded detail. Each step identified in Figure 4-1 of Phase 2 now includes specific actions and outputs/outcomes for that action (see key). 79

OCR for page 70
80 Sustainability for the Nation: Resource Connections & Governance Linkages rules” (e.g., consensus or majority vote) for what constitutes acceptable actions or outcomes. The Team must build in the principles of adaptive management— that is, provide for flexibility in altering goals, design, and implementation as knowledge is gained in assessing the course of the program implementation and short-term outcomes (e.g., CRS, 2011). It is critical to include a systematic and explicit process for projecting the outcomes of the program or project in order to anticipate the consequences, both intended and unintended, added benefits in terms of efficiencies and cost- savings, the short- and long-term trade-offs of implementing the plan (vs. doing nothing), and any synergies gained from the program or project. This can be done with a variety of tools, including scenario analysis (Schmitt Olabisi et al., 2010) and policy analysis (Bardach, 2012). Implement action plan. The design phase is where the action plan is de- veloped for addressing the sustainability issue that was identified in the first step of Phase 1. This includes the “what” that needs to be done as well as the “how” and “by whom.” It also includes a determination of the kinds of decision-making tools or models that might be needed for implementation. In this step, the action plan developed in the previous step is actually implemented, either by the Team that designed the program or project or by the implementers determined during the design step. A key action in this step is determining the kinds of boundary organizations or processes that are needed. (A boundary organization or process is one that bridges the scientific and technical people with the policy people and stakeholders either within or across entities, horizontally or vertically. Such or- ganizations often facilitate ongoing dialogue among experts and others (Guston et al., 2010). Approaches to sustainability challenges generally take time and require maintenance to ensure their longevity, adoption, and success. The Team must develop and implement a maintenance plan that describes who is responsible for long-term maintenance, who pays for it, and who evaluates its effectiveness. Phase 3: Evaluation and Adaptation Realize short-term outcomes, assess outcomes, and adjust. This is where the “rubber meets the road” as results are achieved. Outcomes are assessed and evaluated relative to the baseline established in Phase 2. Short-term outcomes are on the scale of a year to a few years. Are the trends observed on track with goals? Significant learning typically occurs during this step as knowledge and actual experience are obtained, which allow modifications to framing the prob- lem, the approach, design, and methods. At this point the evaluation plan identi- fied above becomes critical, because it allows actual results to be compared to the original goals and for adjustments to be made. Additional stakeholders may also be identified and engaged at this point.

OCR for page 70
Development of a Decision Framework 81 Phase 4: Long-Term Outcomes Long-term outcomes are on the scale of several years or more, and should closely track the goals. While performance is assessed and adjustments are made during this phase, as in the previous one, a point is reached where a formal as- sessment is needed. Using the outcome measures developed under Phase 2, at this stage evaluations are conducted to see if short- and long-term outcomes are meeting goals. Ideally, the results of this evaluation should be able to be com- pared to the results of the baseline evaluation conducted in Phase 2. Based on this evaluation, necessary changes to the Team, Goals, Outcomes and Measures, Management Plans, Design, Implementation, or Maintenance are made. When well executed, this framework process will enhance legitimacy, en- courage systems thinking and the relevance of government actions, and most importantly, result in streamlined and more efficient governance. An additional benefit is that the experiences and lessons learned while applying this process are fed back to the participating organizations and individuals, improving both future efforts and government efficiency. Finally, a decision framework for sustainability is unlikely to lead to con- sistently favorable actions unless several additional elements are also in place. An important factor is building sustainability into the fabric of an organization: its mission statement, its goals and objectives, and its organizational and man- agement structure. A previous NRC report (2011a) that addressed sustainability at EPA discussed the importance of incorporating sustainability into an agency’s culture and thinking. This committee (NRC, 2011a) found that integrating sus- tainability into the agency’s work and culture will be most effective when based on clear principles, vision, strategic goals, and implementation processes. Also, the report recommended that the agency institute a focused program of change management to achieve the goal of incorporating sustainability into all agency thinking to optimize the social, environmental, and economic benefits of its de- cisions, and create a new culture among all EPA employees. Similarly, this committee found the incorporation of a culture of sustainability within the oper- ations of the agency is essential. Also very important are structuring sustainabil- ity decision making on long time frames and assessing ways to maximize bene- fits in all sustainability solutions and approaches. RECOMMENDATIONS 1. Federal agencies should adopt or adapt the committee’s decision framework described above. Several key elements of the framework include the need to: a. Build sustainability into the fabric of an organization. b. Structure sustainability decision making on long time frames, in- corporating adaptive management approaches.

OCR for page 70
82 Sustainability for the Nation: Resource Connections & Governance Linkages c. Assess co-benefits and trade-offs in all sustainability solutions and approaches, and communicate these along with the primary out- comes. d. Engage locals, states, and NGOs through an iterative processes to the extent possible, stressing inclusiveness, receptiveness, and good communications. 2. Agencies need not await structural overhauls in order to strength- en their capacity to address sustainability linkages. Agencies can begin by preparing a high-level systems map illustrating key linkages, which can then be deployed widely across federal agencies for any sustainability-related program or project in order to incentivize policy coordination. REFERENCES Bardach, E. 2011. A Practical Guide for Policy Analysis. 4th ed. Washington, DC: CQ Press. EPA (U.S. Environmental Protection Agency). 2012. EPA Announces Framework to Help Local Governments Manage Stormwater Runoff and Wastewater. Online. Available at http://yosemite.epa.gov/opa/admpress.nsf/0/AB2035971BAB1AD48 5257A1B006F25B1. Accessed September 4, 2012. Fiksel, J. 2006. A framework for sustainable materials management. Journal of the Min- erals Metals & Materials Society 58(8):15-22. Congressional Research Service. 2011. Adaptive Management for Ecosystem Restora- tion: Analysis and Issues for Congress. Guston, D. H., W. Clark, T. Keating, D. Cash, S. Moser, C. Miller, and C. Powers. 2010. Report of the Workshop on Boundary Organizations in Environmental Policy and Science. New Brunswick, NJ: Bloustein School of Planning and Public Policy, Rutgers University. IOM (Institute of Medicine). 2010. Bridging the Evidence Gap in Obesity Prevention: A Framework to Inform Decision Making. Washington, DC: National Academies Press. Jabareen, Y. 2008. A new conceptual framework for sustainable development. Environ- ment, Development and Sustainability 10:179-192. Lockie, S., and S. Rockloff. 2005. Decision Frameworks: Assessment of the social as- pects of decision frameworks and development of a conceptual model. Coastal CRC Discussion Paper. Norman Gardens, Australia: Central Queensland Universi- ty. NRC (National Research Council). 1996. Understanding Risk: Informing Decisions in a Democratic Society. Washington, DC: National Academies Press. NRC. 2005. Decision Making for the Environment: Social and Behavioral Science Re- search Priorities. Washington, DC: National Academies Press. NRC. 2008a. Research and Networks for Decision Support in the NOAA Sector Applica- tions Research Program. Washington, DC: National Academies Press. NRC. 2008b. Public Participation in Environmental Decision Making. Washington, DC: National Academies Press. NRC. 2009a. Science and Decisions: Advancing Risk Assessment. Washington, DC: The National Academies Press.

OCR for page 70
Development of a Decision Framework 83 NRC. 2009b. Informing Decisions in a Changing Climate. Washington, DC: National Academies Press. NRC. 2009c. Sustainable Critical Infrastructure Systems: A Framework for Meeting 21st Century Imperatives. Washington, DC: National Academies Press. NRC. 2011a. Sustainability and the U.S. EPA. Washington, DC: The National Acade- mies Press. NRC. 2011b. Improving Health in the United States: The Role of Health Impact Assess- ment. Washington, DC: National Academies Press. NRC. 2012. Science for Environmental Protection: The Road Ahead. Washington, DC: National Academies Press. Schmitt Olabisi, L. K., A. R. Kapuscinski, K. A. Johnson, P. B. Reich, B. Stenquist, and K. J. Draeger. 2010. Using scenario visioning and participatory system dynamics modeling to investigate the future: Lessons from Minnesota 2050. Sustainability 2(8):2686-2706. Waheed, B., F. Khan, and B. Veitch. 2009. Linkage-based frameworks for sustainability assessment: Making a case for driving force-pressure-state-exposure-effect-action (DPSEEA) frameworks. Sustainability 2009(1):441-463. World Bank. 2012. Designing A Results Framework for Achieving Results: A How-to Guide. Online. Available at http://siteresources.worldbank.org/EXTEVACAP DEV/Resources/designing_results_framework.pdf. Accessed February 28, 2013.