Every year, public and private funders spend many billions of dollars on large-scale, complex, multi-national health initiatives. The only way to know whether these initiatives are achieving their objectives is through evaluations that examine the links between program activities and desired outcomes. Investments in such evaluations, which, like the initiatives being evaluated, are carried out in some of the world's most challenging settings, are a relatively new phenomenon. In the last five years, evaluations have been conducted to determine the effects of some of the world's largest and most complex multi-national health initiatives.
Evaluation Design for Complex Global Initiatives is the summary of a workshop convened by the Institute of Medicine in January 2014 to explore these recent evaluation experiences and to consider the lessons learned from how these evaluations were designed, carried out, and used. The workshop brought together more than 100 evaluators, researchers in the field of evaluation science, staff involved in implementing large-scale health programs, local stakeholders in the countries where the initiatives are carried out, policy makers involved in the initiatives, representatives of donor organizations, and others to derive lessons learned from past large-scale evaluations and to discuss how to apply these lessons to future evaluations. This report discusses transferable insights gained across the spectrum of choosing the evaluator, framing the evaluation, designing the evaluation, gathering and analyzing data, synthesizing findings and recommendations, and communicating key messages. The report also explores the relative benefits and limitations of different quantitative and qualitative approaches within the mixed methods designs used for these complex and costly evaluations.
Table of Contents
|1 Introduction and Overview of the Workshop||1-10|
|2 Overview Framework for Complex Evaluations||11-16|
|3 Framing the Evaluation||17-28|
|4 Developing the Evaluation Design and Selecting Methods||29-42|
|5 Mapping Data Sources and Gathering and Assessing Data||43-54|
|6 Applying Qualitative Methods to Evaluation on a Large Scale||55-60|
|7 Applying Quantitative Methods to Evaluation on a Large Scale||61-68|
|8 Analysis Through Triangulation and Synthesis to Interpret Data in a Mixed Methods Evaluation||69-78|
|9 Evolving Methods in Evaluation Science||79-86|
|10 Lessons from Large-Scale Program Evaluation on a Not-Quite-as-Large Scale||87-94|
|11 Using Evaluation Findings and Communicating Key Messages||95-104|
|12 Envisioning a Future for Evaluation||105-114|
|Appendix A: Statement of Task||117-118|
|Appendix B: Workshop Agenda||119-128|
|Appendix C: Participant Biographies||129-148|
|Appendix D: Evaluation Information Summary for Core Example Initiatives||149-168|
|Appendix E: Evaluation Design Resources Highlighted at the Workshop||169-170|
The National Academies Press (NAP) has partnered with Copyright Clearance Center's Rightslink service to offer you a variety of options for reusing NAP content. Through Rightslink, you may request permission to reprint NAP content in another publication, course pack, secure website, or other media. Rightslink allows you to instantly obtain permission, pay related fees, and print a license directly from the NAP website. The complete terms and conditions of your reuse license can be found in the license agreement that will be made available to you during the online order process. To request permission through Rightslink you are required to create an account by filling out a simple online form. The following list describes license reuses offered by the National Academies Press (NAP) through Rightslink:
Click here to obtain permission for the above reuses. If you have questions or comments concerning the Rightslink service, please contact:
Rightslink Customer Care
Tel (toll free): 877/622-5543
To request permission to distribute a PDF, please contact our Customer Service Department at 800-624-6242 for pricing.
To request permission to translate a book published by the National Academies Press or its imprint, the Joseph Henry Press, please click here to view more information.