1. A refined and clear overall research and analytic design that integrates the various research projects under SORA into a coherent whole in order to produce valid and useful findings and recommendations for democracy program improvements.

  2. An operational definition of democracy and governance that disaggregates the concept into clearly defined and measurable components.

  3. Recommended methodologies to carry out retrospective analysis. The recommendations will include a plan for cross-national case study research to determine program effectiveness and inform strategic planning. USAID will be able to use this plan as the basis of a scope of work to carry out comparative retrospective analysis, allowing the agency to learn from its 25 years of investment in DG programs.

  4. Recommended methodologies to carry out program evaluations in the future. The recommendations for future analysis will focus on more rigorous approaches to evaluation than currently used to assess the impact of democracy assistance programming. They should be applicable across the range of DG programs and allow for comparative analysis.

  5. An assessment of the feasibility of the final recommended methodologies within the current structure of USAID operations and defining policy, organizational, and operational changes in those operations that might improve the chances for successful implementation.

OVERALL RESEARCH AND ANALYTIC DESIGN

In response to the first charge, the committee unanimously recommends a four-part strategy for gaining increased knowledge to support USAID’s DG policy planning and programming. These are:


Recommendation 1: Undertaking a pilot program of impact evaluations designed to demonstrate whether such evaluations can help USAID determine the effects of its DG projects on targeted policy-relevant outcomes. A portion of these impact evaluations should use randomized designs since, where applicable and feasible, they are the designs most likely to lead to reliable and valid results in determining project effects and because their use in DG projects has been limited. USAID should begin the pilot program by focusing on a few widely used DG program categories. The pilot evaluations should not supplant current evaluations and assessments, but impact evaluations could gradually become a more important part of USAID’s portfolio of monitoring and evaluation (M&E) activities as the agency gains experience with such evaluations and determines their value. (See Chapters 5 through 7 for a discussion of impact evaluations and how they might be applied to DG projects and Chapter 9 for the committee’s recommendations.)



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement