National Academies Press: OpenBook

Evaluation Design for Complex Global Initiatives: Workshop Summary (2014)

Chapter: Appendix A: Statement of Task

« Previous: References
Suggested Citation:"Appendix A: Statement of Task." Institute of Medicine. 2014. Evaluation Design for Complex Global Initiatives: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18739.
×

Appendix A

Statement of Task

An ad hoc committee will plan and conduct a 2-day public workshop on “Lessons Learned in the Conduct of Large-Scale, Complex, Multi-National Global Health Program Evaluations.” The scope of the workshop will encompass key points learned in the execution of the IOM’s second evaluation of PEPFAR (to be released in February 2013), the 2009 evaluation of the Global Fund, the 2011 evaluation of the President’s Malaria Initiative (PMI), the 2012 evaluation of the Affordable Medicines Facility–malaria (AMFm), and 1–2 examples of non-health development assistance programs. The workshop will focus on transferable insights gained across the spectrum of choosing the evaluator, framing the evaluation, developing the methodological approach, implementing the study plan, gathering, assessing and analyzing the data, synthesizing the findings and recommendations, and communicating the key messages. The workshop will illuminate the relative benefits and limitations of quantitative, qualitative, and mixed methodologies used for these complex and expensive evaluations.

The workshop will be planned by an appointed, international planning committee and take place in Washington, DC; London, UK; Geneva, Switzerland; or South Africa. The participants will include individuals involved with the execution of the previously mentioned evaluations, representatives of the agencies that commissioned the evaluations, experts in a range of evaluation methods, and diverse users of the published evaluations. The discussions will reflect the reality of executing these kinds of evaluations, rather than evaluation theory. The goal is to extend evaluation methodologies by capturing lessons learned. Representatives of foundations, bilateral

Suggested Citation:"Appendix A: Statement of Task." Institute of Medicine. 2014. Evaluation Design for Complex Global Initiatives: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18739.
×

and multilateral governmental donors, and NGOs also will be invited to attend and participate as will representatives of the academic, international organization, and scientific communities involved with the execution of evaluations of this type.

The format of the workshop will include presentations, moderated panel discussions, and small group consultations for participants to explore the application of lessons learned to future work. The workshop may be webcast and archived if resources allow and a rapporteur-authored book length report will summarize the proceedings and be published for a global audience by the U.S. National Academies Press.

Specific issues to be explored in the workshop include

  1. Perspectives on terminology, levels of evidence, and the nature of evidence.
  2. Issues in program evaluation design, including determination of appropriate evaluation questions to be addressed with appropriately matched methods; prospective and retrospective approaches; longitudinal versus cross-sectional versus comparison approaches; and purposeful sampling to maximize feasibility and representativeness.
  3. Methodological issues in evaluating complex and dynamic programs while they are evolving.
  4. Data sources, data quality assurance, and post-evaluation data access issues.
  5. The complementary utility of quantitative and qualitative mixed methods approaches to evaluation; creating an effective interdisciplinary scientific team.
  6. Managing the challenge of program contribution versus attribution.
  7. Component analysis.
  8. Framing and communicating key findings and recommendations in terms that are useful to stakeholders.
  9. Using evaluations to guide future program implementation and program improvement.
  10. Using evaluations to guide or influence policy and funding decisions.
Suggested Citation:"Appendix A: Statement of Task." Institute of Medicine. 2014. Evaluation Design for Complex Global Initiatives: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18739.
×
Page 117
Suggested Citation:"Appendix A: Statement of Task." Institute of Medicine. 2014. Evaluation Design for Complex Global Initiatives: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/18739.
×
Page 118
Next: Appendix B: Workshop Agenda »
Evaluation Design for Complex Global Initiatives: Workshop Summary Get This Book
×
Buy Paperback | $50.00 Buy Ebook | $39.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Every year, public and private funders spend many billions of dollars on large-scale, complex, multi-national health initiatives. The only way to know whether these initiatives are achieving their objectives is through evaluations that examine the links between program activities and desired outcomes. Investments in such evaluations, which, like the initiatives being evaluated, are carried out in some of the world's most challenging settings, are a relatively new phenomenon. In the last five years, evaluations have been conducted to determine the effects of some of the world's largest and most complex multi-national health initiatives.

Evaluation Design for Complex Global Initiatives is the summary of a workshop convened by the Institute of Medicine in January 2014 to explore these recent evaluation experiences and to consider the lessons learned from how these evaluations were designed, carried out, and used. The workshop brought together more than 100 evaluators, researchers in the field of evaluation science, staff involved in implementing large-scale health programs, local stakeholders in the countries where the initiatives are carried out, policy makers involved in the initiatives, representatives of donor organizations, and others to derive lessons learned from past large-scale evaluations and to discuss how to apply these lessons to future evaluations. This report discusses transferable insights gained across the spectrum of choosing the evaluator, framing the evaluation, designing the evaluation, gathering and analyzing data, synthesizing findings and recommendations, and communicating key messages. The report also explores the relative benefits and limitations of different quantitative and qualitative approaches within the mixed methods designs used for these complex and costly evaluations.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!