HIV/AIDS has evolved into one of the world’s greatest public health crises. More than 39 million people are estimated to be living with HIV/AIDS worldwide, over 60 percent of them in sub-Saharan Africa (UNAIDS,2006). HIV prevalence among adults aged 15–49 now exceeds 15 percent in many countries and has approached nearly 25 percent in Botswana, Lesotho, Swaziland, and Zimbabwe. In 2006 alone, more than 4 million people are estimated to have become infected with HIV, including nearly 2 million women and over half a million children under the age of 14. The Joint United Nations Programme on HIV/AIDS (UNAIDS) has estimated that nearly 3 million people died of AIDS worldwide in 2006, and that AIDS has reversed the gains in life expectancy that had been achieved by Africa over the past 50 years (UNAIDS, 2004a, 2006).
By 2006, an estimated 12 million children had been orphaned in sub-Saharan Africa as a result of HIV/AIDS (UNICEF, 2006). The status of girls and women makes them especially vulnerable to HIV, and they now account for nearly half of people living with HIV worldwide and 59 percent of those in sub-Saharan Africa (UNAIDS, 2006). In addition, HIV/AIDS has severely strained national economies and contributed to political instability in many of the countries experiencing an epidemic (UN, 2003b; CSIS, 2005; Rice, 2006). Chapter 2 provides more background about the HIV/AIDS pandemic.
The U.S. Leadership Against HIV/AIDS, Tuberculosis, and Malaria Act
Global funding in response to HIV/AIDS has increased dramatically since 2001 (Kates and Lief, 2006; UNAIDS, 2006). On May 27, 2003, the U.S. Congress passed the United States Leadership against HIV/AIDS, Tuberculosis, and Malaria Act of 2003 (the Leadership Act)1 and launched the U.S. Global AIDS Initiative. The provisions of the legislation that pertain most directly to the initiative (1) required the President to establish a comprehensive, integrated 5-year strategy to combat global HIV/AIDS, including specific objectives, strategies, and approaches related to prevention, treatment, and care; (2) assigned priorities for relevant executive branch agencies; (3) required improved coordination among such agencies; and (4) projected general levels of resources needed to achieve the stated goals. The legislation emphasized the establishment of programs focused on national HIV/AIDS strategies of recipient countries, the needs of women and children, strengthening of countries’ health care infrastructure and workforce, and effective monitoring and evaluation to assess programmatic success. The legislation also required the President to establish within the U.S. Department of State the position of Global AIDS Coordinator (the Coordinator), who would have primary responsibility for oversight and coordination of all U.S. international activities to combat the HIV/AIDS pandemic. On October 6, 2003, Randall Tobias was sworn in as the first Coordinator, with the rank of ambassador. On February 23, 2004, Ambassador Tobias presented the required presidential strategy, the U.S. 5-year Global HIV/AIDS Strategy, to Congress.
The President’s Emergency Plan for AIDS Relief (PEPFAR)
The U.S. Global AIDS Initiative is commonly known by the title given to the U.S. 5-year Global HIV/AIDS Strategy: “The President’s Emergency Plan for AIDS Relief,” or PEPFAR. To measure the progress of the initiative, the PEPFAR strategy establishes three overarching goals (OGAC, 2004):
To encourage bold leadership at every level to fight HIV/AIDS.
To apply best practices within bilateral HIV/AIDS prevention, treatment, and care programs, in concert with the objectives and policies of host governments’ national HIV/AIDS strategies.
To encourage partners, including multilateral organizations and other host governments, to coordinate at all levels to strengthen response efforts, to embrace best practices, to adhere to principles of sound management, and to harmonize monitoring and evaluation efforts to ensure the most effective and efficient use of resources.
The PEPFAR strategy also sets forth guiding principles for achieving the initiative’s mission and goals, including responding with urgency to the crisis; seeking new approaches; coordinating U.S. government oversight and direction of PEPFAR activities; drawing on the scientific evidence base in developing interventions; establishing and ensuring accountability for measurable goals; harmonizing program development and implementation with the host countries; integrating prevention, treatment, and care programs; building national capacity; encouraging national leadership; and coordinating with other partners (OGAC, 2004).
The U.S. Global AIDS Initiative, while encompassing activities in more than 120 countries, is focused on the development of comprehensive and integrated prevention, treatment, and care programs in 15 countries selected largely because they are heavily affected by HIV/AIDS (OGAC, 2004). Of the $15 billion authorized under the Leadership Act, $10 billion is to be allocated to efforts in these 15 countries over a 5-year period (OGAC, 2004). The remainder of the funding goes predominantly to the Global Fund to Fight AIDS, Tuberculosis and Malaria (Global Fund); international HIV/AIDS research at the National Institutes of Health; and HIV/AIDS activities in other, nonfocus countries. The 15 PEPFAR focus countries, 14 of which were named in the Leadership Act, are the Republic of Botswana (Botswana), the Republic of Côte d’Ivoire (Côte d’Ivoire), the Federal Democratic Republic of Ethiopia (Ethiopia), the Cooperative Republic of Guyana (Guyana), the Republic of Haiti (Haiti), the Republic of Kenya (Kenya), the Republic of Mozambique (Mozambique), the Republic of Namibia (Namibia), the Federal Republic of Nigeria (Nigeria), the Republic of Rwanda (Rwanda), the Republic of South Africa (South Africa), the United Republic of Tanzania (Tanzania), the Republic of Uganda (Uganda), the Socialist Republic of Vietnam (Vietnam), and the Republic of Zambia (Zambia). PEPFAR has established specific targets for its prevention, treatment, and care programs in these countries to support prevention of 7 million new HIV infections; treatment of 2 million HIV-infected people with antiretroviral therapy (ART); and care of 10 million people living with and affected by HIV/AIDS, including orphans and other children made vulner-
STUDY GOALS AND APPROACH
Study Mandate and Scope
The Leadership Act mandates that the Institute of Medicine (IOM) evaluate PEPFAR and directs the President to consider the IOM’s findings. Specifically, Section 101(c)(1) of the Leadership Act states:
Not later than 3 years after the date of the enactment of this Act, the Institute of Medicine shall publish findings comparing the success rates of the various programs and methods used under the [PEPFAR] strategy.
In prioritizing the distribution of resources under the [PEPFAR] strategy, the President shall consider the findings published by the Institute of Medicine under this subsection.
This mandate is somewhat ambiguous as to whether the IOM should conduct macro-level comparisons (considering the relative success of PEPFAR and other approaches to aid, such as the Global Fund, for example) or micro-level comparisons (looking, for instance, at the comparative success of different PEPFAR-supported ART or prevention programs). Consultations between the IOM and cognizant congressional staff and State Department officials who were involved in structuring the original mandate and study contract indicated that the true “success” of PEPFAR needed to be judged in terms of real impact—both on the lives of people and on the nature of the epidemics in the affected countries. Recognizing that an in-depth assessment of the impact of PEPFAR would not be feasible within the time frame and resources allocated for the IOM evaluation, the study charge was understood as evaluating the initial implementation of PEPFAR 3 years after authorization to provide guidance for Congress in time for its consideration of the reauthorization of the program (IOM, 2005).
To plan, conduct, and report on this short-term implementation evaluation, the IOM empaneled an independent, expert committee. The scope of the evaluation was limited to the implementation of PEPFAR in the focus countries and did not include the U.S. contribution to the Global Fund, which is also overseen by the Coordinator. Although direct evaluation of the Leadership Act was beyond its scope, the IOM Committee for
the Evaluation of PEPFAR Implementation (the Committee) examined and drew conclusions about factors that appeared to be having a pronounced effect on implementation, some of which have their roots in the Leadership Act.
The Committee began its study in January 2005 with a series of information-gathering and deliberative meetings, and in October 2005 published a letter report outlining its plan for the evaluation (IOM, 2005). Figure 1-1, from the letter report, summarizes the major foci and high-level questions of the evaluation plan (see Appendix C for the full letter report).
As Figure 1-1 illustrates, the Leadership Act is not the subject of the evaluation; rather, it is one of the major points of reference for evaluating PEPFAR implementation. Thus, the Committee did not examine the major features of the program that were determined by the legislation, such as its single-disease focus, the concentration of resources in specified countries, or the established targets and goals, but regarded them as the parameters within which the program was required to be implemented. However, the Committee did address the other major points of reference for evaluating PEPFAR shown in Figure 1-1: global consensus regarding the major components of an HIV/AIDS strategy and the evidence base for specific programs and activities. The Committee recognized that discordance was possible—both among elements of the legislation and between the legislation and other points of reference—and that such discordance could affect program implementation. In such instances, the Committee addressed aspects of the legislation that directly affect implementation and thereby the ability of the program to achieve the goals of the legislation.
Figure 1-1 also illustrates how the U.S. commitment to harmonization lies at the core of the complex structure and approach of PEPFAR, which involves numerous U.S. government agencies, is centrally coordinated by the Office of the U.S. Global AIDS Coordinator (OGAC), but is implemented by the U.S. teams in the focus countries (Country Teams) (UN, 2003a; The Rome Declaration, 2003; The Paris Declaration, 2005; OGAC, 2005). The central tenet of harmonization is that sustainable gains against the AIDS pandemic require that each country own and lead its response to its epidemic. The role of donors is to support and participate in three country-determined elements critical to an effective response—one national AIDS plan, one national AIDS coordinating mechanism, and one national AIDS monitoring and evaluation framework (UNAIDS, 2004b). Thus, the
Committee evaluated the implementation of PEPFAR primarily through the lens of harmonization and sought to determine how effectively the program is meeting its commitment to support the focus countries’ responses to their HIV/AIDS epidemics (IOM, 2005). The evolution of harmonization and PEPFAR’s participation in the process is discussed in detail in Chapter 2.
CONDUCT OF THE EVALUATION
In carrying out this evaluation, the Committee used several approaches to examine PEPFAR’s strategic development and programmatic implementation, including review of the relevant literature and PEPFAR documentation, review and analysis of budgetary and programmatic data, information-gathering Committee meetings, discussions with relevant parties, and visits to the PEPFAR focus countries to observe activities directly and hold face-to-face discussions. The Committee endeavored to respect global efforts at harmonization of monitoring and evaluation—in which the United States was participating prior to PEPFAR (Rugg et al., 2004)—and relied on existing indicators and data sources to the extent possible. Beginning with its first meeting, the Committee reviewed global monitoring and evaluation efforts already under way, systems and processes already in place, and existing data (IOM, 2005).
In addition to the scientific literature, the Committee reviewed a wide range of documents, including PEPFAR’s authorizing legislation, strategy, guidance, and reports; global consensus and guidance documents, such as those of the World Health Organization (WHO) and UNAIDS; national plans and other documents from the focus countries; and reports of other HIV/AIDS donors and PEPFAR partners. In addition to its three information-gathering meetings, the Committee held an extensive series of discussions with U.S. government officials at OGAC, at the implementing agencies, and on the Country Teams; focus country officials; partners; program officials; community groups; and officials from other donor organizations. To encourage the participants in these discussions to speak candidly, the Committee assured them that it would not attribute their statements to individuals by name, organization, or country. The Committee took this approach because consultation with a wide range of people and organizations during its development of the evaluation plan indicated that such an approach would be necessary to facilitate candid discussions (IOM, 2005). Appendix A lists the people and organizations with whom the Committee held discussions; Appendix B shows the generic agenda for the Committee’s visits to the focus countries; and the letter report reprinted in Appendix C details the issues and questions covered in these discussions.
Focus Country Visits
From October 2005 through February 2006, small delegations from the Committee visited 13 of the 15 focus countries. Each visit lasted 1 week and included discussions with the U.S. Country Team, country government officials, officials from other donor groups working in the country, partners implementing PEPFAR programs, and representatives of groups of people
living with HIV/AIDS. The Committee’s delegations visited sites of all types conducting programs focused on prevention, treatment, care, the needs of orphans and vulnerable children, system strengthening, and training. The Committee’s analysis of its country visits, as well as its synthesis of this information with that from other sources, is described in Appendix B. The visits were not designed or intended to allow the Committee to delve deeply into and reach definitive conclusions about any one focus country, program, or aspect of implementation, perhaps with the exception of PEPFAR’s overall management, coordination, and harmonization. Thus, the Committee did not attempt to draw conclusions about specific countries or programs and did not base its conclusions about any aspect of PEPFAR solely on information obtained during the visits. However, the Committee believes that the cumulative information from all of the visits—effectively 13 weeks in PEPFAR focus countries, discussions with hundreds of people, and visits to dozens of sites—provided a comprehensive and detailed picture of PEPFAR implementation overall as viewed from the focus countries.
A great deal of information about the focus countries is a matter of public record—for example, information about the nature of their HIV/AIDS epidemics, their national AIDS strategies and sometimes their operational plans, and their PEPFAR Country Operational Plans. When discussing this kind of information in the report, the Committee identifies specific countries by name. When discussing information that is based on discussions held in the focus countries, however, the Committee avoids attribution of comments even by country.
The Committee provides examples of PEPFAR-supported programs throughout the report. These examples were selected from the Country Operational Plans simply to illustrate the types of programs and activities included in the various PEPFAR categories. The Committee did not visit or review the details of all of the programs described in the examples provided, nor did it evaluate any of these programs.
Budget and Performance Data
As suggested above, the Committee attempted to respect global efforts at harmonization of monitoring and evaluation by relying on existing data sources to the maximum extent possible. The Committee reviewed and analyzed all publicly available PEPFAR budget and performance data, as well as information about HIV/AIDS funding, epidemiology, and activities in the focus countries. The primary sources for PEPFAR data were Congressional Notifications, Country Operational Plans, annual reports and other interim reports, and analyses of the Country Operational Plan Reporting System provided by OGAC. The primary sources of data on the focus countries were their own websites and publications, OGAC, UNAIDS, WHO,
the World Bank, and the Kaiser Family Foundation. The Committee did not audit or independently verify these data; however, it performed some checks for internal consistency, as well as for congruence with external sources. The Committee did not audit any aspect of the program and thus is unable to address such issues as contract compliance, diversion of funds, and corruption.
Challenges for the Evaluation
In addition to the size and complexity of PEPFAR, two features made the Committee’s task an especially challenging one: (1) PEPFAR is of necessity a dynamic, evolving program, and (2) it is still relatively early in its implementation.
The Coordinator has implemented PEPFAR on an emergency basis and, as acknowledged by the Leadership Act, has had to “maintain sufficient flexibility and remain responsive to the ever-changing nature of the HIV/AIDS pandemic” (P.L. 108-25, p. 718). Thus PEPFAR is a rapidly moving, continually evolving target for evaluation. The Committee was prepared to find considerable changes in PEPFAR throughout its evaluation, and attempted to develop an evaluation approach that would allow it to adapt not only to changes in PEPFAR implementation, but also to what the Committee learned as its work proceeded. PEPFAR indeed continued to evolve rapidly as the Committee conducted this study, and the Committee has attempted to remain current with these developments. Clearly, the Committee’s findings and conclusions, particularly its observations of activities in the focus countries, are based on its examination of a particular period in PEPFAR’s development. The Committee’s main recommendations are for the future of the U.S. Global AIDS Initiative as a whole, and so are largely independent of ongoing changes in the management of PEPFAR. However, the Committee hopes that PEPFAR’s dynamism will not diminish, and thus that parts of this report related to PEPFAR management may become outdated rather quickly.
EVALUATING THE SUCCESS OF PEPFAR
Ultimately the “success” of PEPFAR will be judged by whether it has achieved its targets of effectively supporting the prevention of 7 million HIV infections, treatment for 2 million people with HIV/AIDS with ART, and care for 10 million people living with and affected by HIV/AIDS, as well as its longer-term goal of achieving sustainable gains against the HIV/AIDS
epidemics in the focus countries. However, although the Leadership Act was passed in May 2003, the initial funds for the program were not appropriated until January 2004, and the majority of the first year’s funding was not obligated until September 2004. Thus at the close of the Committee’s short-term evaluation, PEPFAR had been supporting the implementation of programs for prevention, treatment, care, and orphans and vulnerable children in the focus countries for less than 2 years—less time, perhaps, than Congress had envisioned when it wrote the mandate for this study. The Committee recognized that it would not be reasonable or feasible to judge PEPFAR’s success solely against the above targets and goal this early in the program’s implementation, and therefore planned instead in this short-term evaluation to examine how well PEPFAR has been establishing the foundation for and making reasonable progress toward achieving and measuring these targets and this goal. In so doing, the Committee aimed to make constructive suggestions for improvement to ensure that the program ultimately meets the targets and goal. The Committee recognized that it was too early in the program for this short-term evaluation to provide the information necessary to judge true success, that is, to adequately measure what matters most—the program’s impact on the lives of the people whom the Leadership Act sought to serve. The Committee urges that a long-term evaluation be conducted to determine whether the U.S. Global AIDS Initiative has ultimately succeeded in improving the lives of people in the focus countries by preventing infections, treating patients, and caring for people. To this end, the Committee is planning to conduct a workshop to encourage collaboration among evaluators, and to discuss and develop considerations for designing an evaluation of PEPFAR’s impact on the focus countries, their HIV/AIDS epidemics, and, most important, their people.
ORGANIZATION OF THE REPORT
This report is organized into three main parts (see Box 1-1). Part I describes the nature and object of the study—the U.S. Global AIDS Initiative—and its context. Following this introductory chapter, Chapter 2 briefly describes the HIV/AIDS pandemic that the Leadership Act was designed to address, as well as the global context for the U.S. Global AIDS Initiative, briefly highlighting the challenges for the implementation of PEPFAR, and indeed any such donor program. Chapter 2 also provides background on the U.S. Global AIDS Initiative, beginning with the historic legislation that enabled it and concluding with the current structure for implementing the first 5-year strategy of the initiative—PEPFAR. Part II describes the progress of PEPFAR to date; its structure is aligned with how the Coordinator reports to Congress. Chapter 3 presents the Committee’s assessment of the Coordinator’s management of the U.S. Global AIDS Initiative, including
development of the strategy required by the Leadership Act. Chapters 4 through 7 provide progress reports on the implementation of the strategy according to its four major categories of activities and programs—prevention, treatment, care, and orphans and vulnerable children—and suggest how the Coordinator could improve programming in each of these categories. Part III looks to the future—both the immediate future of the remaining years of the PEPFAR 5-year strategy and the longer-term future of the U.S. Global AIDS Initiative. The one chapter in this section, Chapter 8, focuses on the common themes that emerged when the Committee considered PEPFAR as a whole, that is, without being bound by the program’s four categories. This final chapter draws on what the Committee learned from the implementation of PEPFAR thus far to suggest how the U.S. Global AIDS Initiative could advance most effectively toward achieving the primary goal of its landmark enabling legislation—U.S. leadership to address the HIV/AIDS pandemic.
CSIS (Center for Strategic and International Studies). 2005. More than humanitarianism: A strategic U.S. approach toward Africa. Washington, DC: CSIS.
IOM (Institute of Medicine). 2005. Plan for a short-term evaluation of PEPFAR implementation. Washington, DC: The National Academies Press.
Kates, J., and E. Lief. 2006. International assistance for HIV/AIDS in the developing world: Taking stock of the G8, other donor governments and the European Commission. Washington, DC: Kaiser Family Foundation.
OGAC (Office of the U.S. Global AIDS Coordinator). 2004. The President’s Emergency Plan for AIDS Relief: U.S. five-year global HIV/AIDS strategy. Washington, DC: OGAC.
OGAC. 2005. The President’s Emergency Plan for AIDS Relief: Defining U.S. support for national strategies (PowerPoint Presentation, July 2005). Washington, DC: OGAC. http://www.pepfar.gov (accessed July 19, 2005).
The Paris Declaration on aid effectiveness: Ownership, harmonisation, alignment, results and mutual accountability. 2005. High level forum on harmonization, February 28–March 2, 2005, Paris, France. http://www.aidharmonization.org (accessed November 10, 2006).
The Rome Declaration on harmonization. 2003. High level forum on harmonization, February 24–25, 2003, Rome, Italy. http://www.aidharmonization.org (accessed November 10, 2006).
Rice, S. 2006. Global poverty, weak states, and insecurity. The Brookings Blum Roundtable. Washington, DC: The Brookings Institution.
Rugg, D., G. Peersman, and M. Carael. 2004. Global advances in HIV/AIDS monitoring and evaluation. Minneapolis, MN: Wiley Periodicals, Inc.
UN (United Nations). 2003a. Monterrey Consensus of the International Conference on Financing for Development: The final text of agreements and commitments adopted at the International Conference on Financing for Development, Monterrey, Mexico, 18–22 March 2002. Geneva, Switzerland: UN.
UN. 2003b. Organizing the UN response to the triple threat of food insecurity, weakened capacity for governance and AIDS, particularly in southern and eastern Africa. Document discussed at the UN high-level committee on programmes, 6th Session, Rome, Italy, September 18–19, 2003. Geneva, Switzerland: UN.
UNAIDS (Joint United Nations Programme on HIV/AIDS). 2004a. 2004 report on the global AIDS epidemic: 4th global report. Geneva, Switzerland: UNAIDS.
UNAIDS. 2004b. “Three ones” key principles. Conference paper 1, Washington consultation of April 4, 2004. Geneva, Switzerland: UNAIDS. http://www.unaids.org/NetTools/Misc/DocInfo.aspx?LANG=en&href=http://gva-doc-owl/WEBcontent/Documents/pub/UNA-docs/Three-Ones_KeyPrinciples_en.pdf (accessed September 20, 2006).
UNAIDS. 2006. 2006 report on the global AIDS epidemic. Geneva, Switzerland: UNAIDS.
UNICEF (United Nations Children’s Fund). 2006. Africa’s orphaned and vulnerable generations: Children affected by AIDS. New York: UNICEF.