future impact evaluations by the IOM committee that evaluated PEPFAR implementation. Next, a vision is described for the types of questions workshop participants would like to see addressed in future impact evaluations, along with suggestions for how the process of impact evaluation would ideally be carried out. The final section addresses methodological issues that were raised by participants as being important to consider in the design of future impact evaluations.

DEFINING AND USING IMPACT EVALUATION

Meeting participants proposed a working definition of impact evaluation as a measurement of net change in outcomes attributable to a specific program using a methodology that is robust, available, feasible, and appropriate both to the question under investigation and to the specific context. Workshop participants noted that impact evaluation is not only about outcomes, but also the process that leads to outcomes; that is, it includes both means and ends. Participants argued for a definition of impact evaluation that is longer term, more broadly defined, and less linear. Although a more traditional definition of infectious disease impact evaluation (that is, one that is limited to metrics such as prevalence, incidence, infections averted, morbidity, and mortality) is important, the broader and deeper impact evaluation envisioned would also include measurement of changes in health status, systems capacity, quality of services, economic development, and social, economic, and political outcomes.

Two major uses of evaluation were described: (1) use of evaluation for judging the performance of the program for purposes of accountability (summative evaluation) and (2) use of evaluation for informing the improved decision making within a program (formative, or utilization-focused, evaluation). Formative evaluation of PEPFAR is important to inform both congressional decision making and programmatic decision making in partner countries, although decision makers at different levels may have different evaluation needs, participants said. For each evaluation question, participants noted, it is important to clarify who needs the information, what information is needed, and when.

INTERNAL EVALUATION OF PEPFAR

According to PEPFAR officials, a key aspect of PEPFAR’s monitoring process and infrastructure is to track progress toward the program’s goals in prevention, treatment, and care. PEPFAR internal monitoring supports the principles of local leadership and ownership of the HIV/AIDS response by building local capacity, using local infrastructure, implementing the program according to national guidelines, monitoring using locally developed



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement