. "2 Envisioning a Meaningful Impact Evaluation for PEPFAR: Moving Beyond Counting." Design Considerations for Evaluating the Impact of PEPFAR: Workshop Summary. Washington, DC: The National Academies Press, 2008.
The following HTML text is provided to enhance online
readability. Many aspects of typography translate only awkwardly to HTML.
Please use the page image
as the authoritative form to ensure accuracy.
Design Considerations for Evaluating the Impact of PEPFAR: Workshop Summary
rush to produce quantitative outputs for goals like 2–7–101 can sometimes jeopardize sustainability. He called for a reduction in the number of indicators and a focus on a more strategic set of results. Such prioritization could be accomplished using a criteria-based approach, noted one participant. The generation of improved indicators, noted discussant Caroline Ryan of the Office of the U.S. Global AIDS Coordinator (OGAC), will ultimately provide better information for making programmatic course corrections.
Many participants argued in favor of a broader interpretation of impact as opposed to a narrower, or more pure, notion of impact. Many participants also believed that priorities and opportunities for evaluation include not only those describing the results of program implementation, but also the process of implementation—for example, coordination and capacity building—because the means of implementation are inseparable from the ends. Speaker Julia Compton of the UK Department for International Development (DFID) noted that defining the boundaries of what to measure—around what is AIDS and what is AIDS spending—is a challenge, and that defining the boundaries too widely or too narrowly involves risks. Defining boundaries includes answering the question of whether to measure direct AIDS impacts very narrowly or to measure broader impacts such as gender empowerment or land rights or health systems.
Workshop participants identified questions for evaluating impact that can be clustered into the following nine broad categories: cost-effectiveness, conceptual approach, health impacts, impacts beyond health, capacity building and health systems strengthening, coordination and harmonization, sustainability, equity and fairness, and unintended impacts.
Workshop participants described the importance of developing indicators that track the cost-effectiveness of PEPFAR. “Are we getting the biggest bang for our buck?” asked workshop speaker Christos Tsentas from the office of Representative Barbara Lee. Discussant Mead Over of the Center for Global Development emphasized that cost-effectiveness of interventions—and whether costs can be afforded in particular countries once donor funding is discontinued—has important implications for program sustainability. Cost-effectiveness measures can be used to evaluate different approaches, strategies, and interventions. Speaker Stefano Bertozzi of the National Institute of Public Health, Mexico, urged that cost-effectiveness measures be designed to assess types of prevention services delivered. He described a
PEPFAR’s 5-year goals—known as 2–7–10—are to support treatment for 2 million people, prevention of 7 million new infections, and care for 10 million people, including orphans and vulnerable children.