evaluation designs described in Chapter 5. First, how many of USAID’s current projects appear suitable for randomized impact designs? Second, when projects are not suitable for randomized evaluations, what options are available and how should the other methods described in Chapter 5 be chosen and applied?

HOW OFTEN ARE RANDOMIZED EVALUATIONS FEASIBLE?

To help answer this question, project staff collected information about the DG activities that the USAID mission in Uganda had undertaken in recent years (see Appendix E for a list of these projects as well as those in Albania and Peru). The projects in Uganda included efforts designed to provide support for the Ugandan Parliament, strengthen political pluralism and the electoral process, and promote political participation—a fairly typical roster of projects and one that parallels those implemented by USAID missions in many countries. A team member then divided these projects into 10 major activities and scored them for (1) amenability of each activity to randomized impact evaluation and (2) where randomized evaluation was not deemed possible, the benefits of adding other impact evaluation techniques (better baseline, outcome, or comparison group measures) to existing monitoring and evaluation (M&E) designs.1 In doing so, the committee recognizes that current USAID project monitoring plans are largely designed to track an implementer’s progress in achieving agreed-upon outputs and outcomes. Our approach, therefore, is not to assess the quality of current monitoring plans but rather to assess and illustrate instances where additional information that could reveal the impact of DG projects is currently not being collected but could readily be acquired.

The first finding of the analysis was that all 10 of the activities examined used M&E plans that omitted collection of crucial information that would be needed if USAID sought to make impact evaluations of those activities.2 The committee does not mean to criticize current M&E plans, which focus on acquiring important information for program management and resource allocation. The committee wants to draw attention to the marked difference between the content of the currently mandated and universal M&E components of most DG projects and the information that would need to be acquired to conduct a sound and credible impact evaluation of project effects. The latter is a different task and, as noted,

1

This section is based on the work of Mame-Fatou Diagne, University of California, Berkeley.

2

See Chapter 2 for a discussion of the difference between current USAID project M&E plans and impact evaluations.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement