Impact concerns not only outcomes, but also the change that leads to outcomes, noted speaker Julia Compton of the UK Department for International Development. Impact evaluation, therefore, is a combination of both means—or processes—and ends, observed speaker Mary Lyn Field-Nguer of John Snow, Inc.

Speakers Compton and Sara Pacqué-Margolis of the Elizabeth Glaser Pediatric AIDS Foundation emphasized the long-term nature of impact evaluation. Compton observed the importance of considering longer term results, such as unintended effects and sustainability, in impact evaluation. She noted that the Organisation for Economic Co-operation and Development’s Development Assistance Committee includes such concepts in its definition of impact.

In the context of evaluating the impact of PEPFAR, several speakers called for a shift to a broader definition of impact evaluation and to a more nonlinear concept of causation. Speaker Paul De Lay of the Joint United Nations Programme on HIV/AIDS (UNAIDS) observed that although the traditional definition for infectious disease impact evaluation of prevalence, incidence, infections averted, morbidity, and mortality is important, an assessment about what PEPFAR has actually accomplished should include broader concepts such as the intensity, quality, targeting, and equity of services. Key aspects of economic development and social change should also be built into the definition, he noted. Pacqué-Margolis added that a broader definition of impact evaluation would include measuring changes in health status, systems capacity, and other social, economic, and political outcomes. All impact evaluations must be based on a conceptual model of causation and intervention, observed speaker Nils Daulaire of the Global Health Council, but cultural-, political-, and location-specific factors and shifting influences make causality in the world of HIV/AIDS highly nonlinear. Innovative thinking may therefore be required when designing an impact evaluation for such a system.

Uses of Impact Evaluation

How is impact evaluation used? Workshop participants discussed uses of impact evaluation both to assess whether a project met its goals and to inform improvement in a project or indicate the need for midcourse corrections.

Summative Evaluation for Accountability and Advocacy

Impact evaluation has an important role in judging the performance of a program in order to account to specific constituents. In the case of PEPFAR, noted speakers De Lay and Agnes Binagwaho of the Rwanda Na-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement