Cover Image

Not for Sale



View/Hide Left Panel
Click for next page ( 2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
INTRODUCTION HIV/AIDS has evolved into the world's greatest public health crisis. Forty million people are estimated to be living with HIV/AIDS, 64 percent of them in sub-Saharan Africa (UNAIDS, 2004a). HIV prevalence among adults 15-49 years of age now exceeds 15 percent in many coun- tries and has approached nearly 30 percent in Botswana, Lesotho, Swaziland, and Zimbabwe. In 2004 alone, almost 5 million people are thought to have become infected with HIV, including 2 million women and 800,000 children. UNAIDS estimated that 3.1 million people died of AIDS worldwide in 2004, and that AIDS has reversed the gains in life expectancy that had been achieved by Africa over the past 50 years (UNAIDS, 2004a). By 2003, an estimated 12 million children had been orphaned in sub-Saharan Africa as a re- sult of HIV/AIDS, half of whom are between the ages of 10 and 14 (UNAIDS, 2004a). Girls and women are especially vulnerable to HIV, and now account for 50 percent of people living with HIV worldwide and 57 percent of those in sub-Saharan Africa (UNAIDS, 2004a). In addition, HIV/AIDS has severely strained national economies and contributed to political instability in many of the countries experiencing an epidemic. The U.S. Leadership against HIV/AIDS, Tuberculosis, and Malaria Act Global funding in response to HIV/AIDS has increased dramatically since 2001. On May 27, 2003, the U.S. Congress passed Public Law 108-25: the United States Leadership against HIV/AIDS, Tuberculosis, and Malaria Act of 2003 (the Act). Provisions of this legislation 1) required the President to establish a comprehensive, integrated 5-year strategy to combat global HIV/AIDS, including specific objectives, strategies, and approaches related to HIV/AIDS prevention, treatment, and care; 2) assigned priorities for relevant executive branch agencies; 3) required improved coordination among such agencies; and 4) projected general levels of resources needed to achieve the stated goals. The legislation emphasized the establishment of programs that focus on national HIV/AIDS strategies of recipient countries, women and children, strengthening of health care infrastructure and workforce, and effective monitoring and evaluation to assess programmatic success. This legislation also required the President to establish the Office of the Global AIDS Coordinator (OGAC) within the U.S. Department of State, which would have primary responsibility for oversight and coordination of all U.S. international activities to combat the HIV/AIDS pandemic. On October 6, 2003, Randall Tobias 10

OCR for page 1
was sworn in as the first Global AIDS Coordinator, with the rank of Ambassador. On February 23, 2004, Ambassador Tobias presented the U.S. 5-year Global HIV/AIDS Strategy to Congress. President's Emergency Plan for AIDS Relief (PEPFAR) The initiative is commonly known by the title given to the U.S. 5-year Global HIV/AIDS Strategy: "The President's Emergency Plan for AIDS Relief", or PEPFAR. In order to measure the progress of the initiative, the PEPFAR strategy establishes three overarching goals: To encourage bold leadership at every level to fight HIV/AIDS; To apply best practices within bilateral HIV/AIDS prevention, treatment, and care programs, in concert with the objectives and policies of host governments' national HIV/AIDS strategies; and To encourage partners, including multilateral organizations and other host governments, to coordinate at all levels to strengthen response efforts, to embrace best practices, to adhere to principles of sound management, and to harmonize monitoring and evaluation efforts to ensure the most effective and efficient use of resources (OGAC, 2004). The PEPFAR strategy also describes the principles according to which it will achieve its mission and goals, including responding with urgency to the crisis; seeking new approaches; coordinating the U.S. Government oversight and direction of PEPFAR activities; drawing on the scientific evidence base in developing interventions; establishing measurable goals for programs; harmonizing program development and implementation with the host countries; integrating prevention, treatment and care programs; building national capacity; encouraging national leadership; and coordinating with other partners (OGAC, 2004). PEPFAR, while encompassing activities in over 120 countries, is focused on the devel- opment of comprehensive and integrated prevention, treatment, and care programs in 15 coun- tries selected largely because they are heavily affected by HIV/AIDS (OGAC, 2005b). With re- gard to funding, $10 billion of the $15 billion authorized under the Act is to be allocated to efforts in these 15 countries over five years (OGAC, 2005b). Currently, the 15 PEPFAR focus countries are Botswana, Cote d'Ivoire, Ethiopia, Guyana, Haiti, Kenya, Mozambique, Namibia, Nigeria, Rwanda, South Africa, Tanzania, Uganda, Zambia, and Vietnam. PEPFAR has estab- lished additional goals for its prevention, treatment and care programs in these countries, specifi- cally: to support prevention of 7 million new HIV infections, treatment of 2 million HIV-infected 11

OCR for page 1
people with antiretroviral therapy (ART), and care of 10 million people infected and affected by HIV/AIDS.1 OGAC is responsible for maintaining the focus of PEPFAR by leading policy develop- ment, program oversight, and coordination both among U.S. government departments and agen- cies and with other donors and governments. Additionally, OGAC is responsible for the alloca- tion of funds which are distributed through a number of U.S. government departments and agencies including the Departments of State, Defense (DoD), Commerce, Labor (DoL), and Health and Human Services (HHS); the U.S. Agency for International Development (USAID); and the Peace Corps (OGAC, 2005a). The two largest implementing entities are USAID and HHS which includes the Centers for Disease Control and Prevention (CDC), Food and Drug Administration, National Institute of Health (NIH), and Health Resources and Services Admini- stration (HRSA). Within OGAC, staffing is organized into several groups, all of which include OGAC staff as well as representatives from the other U.S. government departments and agencies coordinated by OGAC. These groups include the Policy Group, incorporating representation from USAID, HHS, the White House and the National Security Council; the Deputy Principals Group, handling program management and logistics with representation from agencies including the Peace Corps, HHS, USAID, DoD and DoL; and a Scientific Steering Committee, consisting of representatives from CDC, NIH, USAID, and DoD (Moloney-Kitts, 2005). Finally, Core and Technical Teams, whose members are drawn from a wide range of U.S. government agencies, are responsible for supporting programs in PEPFAR countries with specific technical and im- plementation issues. Each of the focus countries has a U.S. Government Mission team that is intended to provide a unified strategy and voice and is responsible for coordination of PEPFAR-sponsored programs in the country. The team is led by the U.S. ambassador and includes representatives from all of the implementing departments and agencies. It is supported by a core team based in OGAC which serves as a liaison between the field and headquarters. In many countries, high-level PEPFAR advisory committees have been formed to ensure a close working relationship between the U.S. government and host-country counterparts (OGAC, 2005a). In an effort to create self-sustaining, lasting systems to address HIV/AIDS, PEPFAR emphasizes the development of national 1For purposes of this goal, PEPFAR defines "treatment" as antiretroviral therapy (ART) and categorizes other types of treatment such as ther- apy for opportunistic infections or for pain management under "care". 12

OCR for page 1
leadership, human resources, and other capacities through collaboration of Mission teams with existing country partners, as well as with other donors. Study Goals and Approach The Act mandates that the Institute of Medicine (IOM) evaluate PEPFAR and directs the President to consider IOM's findings. Specifically Sec. 101 (c) (1) of the Act states: "Not later than 3 years after the date of the enactment of this Act, the Institute of Medicine shall publish findings comparing the success rates of the various programs and methods used under the [PEPFAR] strategy. In prioritizing the distribution of resources under the [PEPFAR] strategy, the President shall consider the findings published by the Institute of Medicine under this subsection." Thus, the current task of the IOM is to be responsive to this mandate and provide to Congress in time for reauthorization discussions an assessment of PEPFAR implementation at the three- year mark. The IOM has undertaken an independent, expert consensus committee process to plan, conduct, and report on this short-term evaluation, the scope of which is limited to the im- plementation of PEPFAR in the focus countries and does not include the U.S. contribution to the Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund) which is also coordinated by OGAC. The IOM Committee for the Evaluation of PEPFAR Implementation (the Commit- tee) has three advisory Subcommittees focused on prevention, treatment, and care (see Appendix 1). This letter report outlines the Committee's plan for the short-term evaluation. In addition to the size and complexity of PEPFAR, two additional features make the Com- mittee's task an especially challenging one: PEPFAR is of necessity a dynamic, evolving pro- gram and it is still relatively early in its implementation. As its name implies, OGAC has implemented PEPFAR on an emergency basis and, as the Act acknowledged, has had to "maintain sufficient flexibility and remain responsive to the ever- changing nature of the HIV/AIDS pandemic." Thus PEPFAR is a rapidly-moving, continuously- evolving target for evaluation. The Committee is prepared to find considerable changes in PEPFAR throughout its evaluation, and has developed an approach to the evaluation that will allow it to adapt not only to changes in PEPFAR implementation, but also to what the Commit- tee learns as the evaluation proceeds. The Committee's evaluation plan should be viewed as a 13

OCR for page 1
work-in-progress that will be modified to reflect both the dynamic nature of PEPFAR and what the Committee learns--particularly as it visits the PEPFAR focus countries to directly observe implementation activities. Ultimately the "success" of PEPFAR will be judged by whether it has achieved its near-term goals of effectively supporting the prevention of 7 million HIV infections, treatment for 2 mil- lion people with HIV/AIDS with ART, and care of 10 million people infected with and affected by HIV/AIDS, as well as its longer-term goal of sustainable gains against the HIV/AIDS epi- demics in the focus countries. However, although the Act was passed in May 2003, funds were not appropriated until January 2004, and the majority of the first year's funding was not obli- gated until September 2004 (see Appendix 2). Thus, at the time of the Committee's evaluation, PEPFAR will have been supporting the implementation of prevention, treatment and care pro- grams in the focus countries for less than two years--less time than perhaps Congress had envi- sioned when they wrote the mandate for the IOM study. It would not be reasonable or feasible to evaluate PEPFAR solely against these goals so early in implementation, and therefore in this short-term evaluation the Committee plans instead to evaluate how PEPFAR is progressing to- ward 1) achieving these goals and 2) building the monitoring and evaluation capacity to demon- strate that it has achieved them. The short-term evaluation can provide insights into whether PEPFAR is making reasonable progress toward its goals and can suggest ways in which the program can be improved to ensure that it ultimately meets its goals. However, it cannot adequately measure what matters most the impact on the lives of the people the legislation seeks to serve. In recognition of this, in addi- tion to providing the short-term evaluation that will be responsive to the legislative mandate, the IOM Committee is planning a long-term evaluation to determine whether PEPFAR has ulti- mately succeeded in improving the lives of the people in the focus countries by preventing infec- tions, treating patients, and caring for people. We plan to publish the plan for a long-term study shortly after publication of the mandated report. This long-term evaluation plan will be informed by the Committee's fuller understanding of PEPFAR implementation and the challenging ex- perience of evaluating it. In the development of its plan, the Committee has consulted widely and remains open to re- ceiving input from the broad range of parties interested in and affected by PEPFAR. To develop the plan, the Committee (and the additional subcommittee members) convened first in February 14

OCR for page 1
2005 and then again in April 2005 when the Committee was fully-formed to review documenta- tion of and hear testimony about PEPFAR. The questions highlighted in the plan reflect the mul- tiple data sources that the committee believes capture the primary components of PEPFAR im- plementation. The Committee has considered peer-reviewed literature on global efforts to combat HIV/AIDS in resource-limited settings, documentation provided by OGAC, and other program-related materials. The Committee has heard testimony from OGAC officials and PEPFAR Mission staff; global organizations such as the World Health Organization, UNAIDS, Global Fund, and World Bank; and numerous PEPFAR grantees and partners. In addition, one Committee member and one staff attended the PEPFAR Annual Meeting in Addis Ababa, Ethio- pia in May 2005. Following these initial informationgathering activities and several phone con- ferences to further explore information specific to prevention, treatment, and care, the Commit- tee met in July 2005 to outline the short-term evaluation plan (see Figure 1). 15

OCR for page 1
Figure 1: Short-term Evaluation Plan Schematic Legislation PEPFAR Strategy Global Consensus Is the strategy consistent with the legislation? Is the strategy consistent with Resources global consensus? How are resources allocated according to the priorities outlined in the strategy? PEPFAR Management Is PEPFAR coordinated within US government, between OGAC & Mission, with other donors? Is PEPFAR functioning as a Learning Organization How is PEPFAR evaluating its own performance? Focus Countries Harmonization To what degree is PEPFAR harmonized with the National Plan, National Strategy Coordinating Entity, and National Monitoring & Evaluation Framework? Science Are programs Are programs and activities and activities aligned with Prevention, Treatment, Care evidence-based? the strategy? If no, is research What prevention, treatment, and care programs and activities is PEPFAR supported to supporting and how are they progressing? determine Is PEPFAR on track to reach its goals of supporting the: effectiveness? Prevention of 7 Treatment of 2 Care of 10 million million HIV million HIV-infected people infected & infections? people with ART? affected by HIV/AIDS? Capacity Building Integration How is PEPFAR enhancing national capacity with How is PEPFAR integrating programs and services within respect to Workforce, Laboratories, Supply Chain, and among each of the domains of prevention, treatment, and Clinics, Information Systems, Monitoring & Evaluation, care; into communities; into national health systems? Communities? 16

OCR for page 1
SHORT-TERM EVALUATION QUESTIONS Strategy Is the PEPFAR Strategy consistent with the Act and relevant global consensus? The same section of the Act that mandates the IOM study also specifies the major elements that should be contained in the PEPFAR Strategy. In addition, the legislation calls for the U.S. to take a strong leadership position with respect to the global effort to combat HIV/AIDS and also to coordinate its efforts with the international donor community. The global HIV/AIDS community, led by organizations such as those of UNAIDS, has published guidelines outlining international consensus strategies for the development and implementation of prevention, treat- ment, and care programs. Does the PEPFAR strategy address the major elements of the Act such as: o Developing specific objectives, approaches and strategies for activities related to HIV/AIDS prevention, treatment and care? (Sec. 101.a.1 & Sec.101.b.3.C) o Focusing prevention funding, educational messages and activities on the reduction of HIV/AIDS behavioral risks? (Sec. 101.a.4 & Sec.101.b.3.C) o Linking resources to program objectives and establishing of priorities for the distribu- tion of resources based on factors such as population characteristics and needs as well as the existing infrastructure or funding levels? (Sec. 101.a.6 & 9 & Sec.101.b.3.M) o Maximizing United States capabilities in the areas of technical assistance and train- ing? (Sec. 101.a.8 & Sec.101.b.3.H) o Improving coordination and reducing duplication among relevant executive branch agencies, foreign governments, and international organizations? (Sec. 101.a.3 & Sec.101.b.3.K & L) o Focusing on strategies developed to meet the needs of women, orphans and families? (Sec. 101.b.3.E, G, R, & S) 17

OCR for page 1
o Promoting the development and implementation of strategies and programs designed to enhance the development of health care infrastructure, delivery systems and leader- ship capacity? (Sec.101.a.2 & Sec.101.b.3.C & D) o Establishing monitoring and evaluating programs in order to measure success of the strategies, promoting successful models, and terminating unsuccessful programs? (Sec.101.b.3.B & N) Is the PEPFAR strategy consistent with best practices outlined in global consensus docu- ments related to prevention, treatment and care of HIV/AIDS including: o The Global Strategy Framework on HIV/AIDS (UNAIDS, 2001) o Intensifying HIV Prevention (UNAIDS, 2005) o A Public Health Approach for Scaling Up Antiretroviral (ARV) Treatment: A Toolkit for Programme Managers (WHO, 2003) o The Framework for the Protection, Care and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS (UNAIDS and UNICEF, 2004) o Joint ILO/WHO guidelines on health services and HIV/AIDS (ILO/WHO, 2005) Has OGAC modified the PEPFAR strategy to reflect insights gained and lessons learned during implementation? Resources How is PEPFAR allocating resources and what is the effect on implementation? Both the Act and the PEPFAR strategy include guidance on the allocation of resources under the program. The legislation authorizes an annual appropriation for PEPFAR and outlines priori- tization for resources in terms of program type (e.g., 20% for prevention, 55% for treatment, 15% for care, 10% for orphans and vulnerable children), and population and/or country charac- teristics (e.g., size and demographics of the HIV positive population, existing infrastructure). The PEPFAR strategy outlines allocation mechanisms including country allocations based on each country's five-year strategic plan and performance assessments related to reaching annual prevention, treatment, and care targets, as well as a central funding mechanism for regional ac- tivities. Although there are a number of constraints on resource allocation, OGAC leadership 18

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 INTRODUCTION HIV/AIDS has evolved into the world’s greatest public health crisis. Forty million people are estimated to be living with HIV/AIDS, 64 percent of them in sub-Saharan Africa (UNAIDS, 2004a). HIV prevalence among adults 15–49 years of age now exceeds 15 percent in many countries and has approached nearly 30 percent in Botswana, Lesotho, Swaziland, and Zimbabwe. In 2004 alone, almost 5 million people are thought to have become infected with HIV, including 2 million women and 800,000 children. UNAIDS estimated that 3.1 million people died of AIDS worldwide in 2004, and that AIDS has reversed the gains in life expectancy that had been achieved by Africa over the past 50 years (UNAIDS, 2004a). By 2003, an estimated 12 million children had been orphaned in sub-Saharan Africa as a result of HIV/AIDS, half of whom are between the ages of 10 and 14 (UNAIDS, 2004a). Girls and women are especially vulnerable to HIV, and now account for 50 percent of people living with HIV worldwide and 57 percent of those in sub-Saharan Africa (UNAIDS, 2004a). In addition, HIV/AIDS has severely strained national economies and contributed to political instability in many of the countries experiencing an epidemic. The U.S. Leadership against HIV/AIDS, Tuberculosis, and Malaria Act Global funding in response to HIV/AIDS has increased dramatically since 2001. On May 27, 2003, the U.S. Congress passed Public Law 108–25: the United States Leadership against HIV/AIDS, Tuberculosis, and Malaria Act of 2003 (the Act). Provisions of this legislation 1) required the President to establish a comprehensive, integrated 5-year strategy to combat global HIV/AIDS, including specific objectives, strategies, and approaches related to HIV/AIDS prevention, treatment, and care; 2) assigned priorities for relevant executive branch agencies; 3) required improved coordination among such agencies; and 4) projected general levels of resources needed to achieve the stated goals. The legislation emphasized the establishment of programs that focus on national HIV/AIDS strategies of recipient countries, women and children, strengthening of health care infrastructure and workforce, and effective monitoring and evaluation to assess programmatic success. This legislation also required the President to establish the Office of the Global AIDS Coordinator (OGAC) within the U.S. Department of State, which would have primary responsibility for oversight and coordination of all U.S. international activities to combat the HIV/AIDS pandemic. On October 6, 2003, Randall Tobias

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 was sworn in as the first Global AIDS Coordinator, with the rank of Ambassador. On February 23, 2004, Ambassador Tobias presented the U.S. 5-year Global HIV/AIDS Strategy to Congress. President’s Emergency Plan for AIDS Relief (PEPFAR) The initiative is commonly known by the title given to the U.S. 5-year Global HIV/AIDS Strategy: “The President’s Emergency Plan for AIDS Relief” or PEPFAR. In order to measure the progress of the initiative, the PEPFAR strategy establishes three overarching goals: To encourage bold leadership at every level to fight HIV/AIDS; To apply best practices within bilateral HIV/AIDS prevention, treatment, and care programs, in concert with the objectives and policies of host governments’ national HIV/AIDS strategies; and To encourage partners, including multilateral organization and other host governments, to coodinate at all levels to strengthen response efforts, to embrace best practices, to adhere to principles of sound management, and to harmonize monitoring and evaluation efforts to ensure the most effective and efficient use of resources (OGAC, 2004). The PEPFAR strategy also describes the principles according to which it will achieve its mission and goals, including responding with urgency to the crisis; seeking new approaches; coordinating the U.S. Government oversight and direction of PEPFAR activities; drawing on the scientific evidence base in developing interventions; establishing measurable goals for programs; harmonizing program development and implementation with the host countries; integrating prevention, treatment and care programs; building national capacity; encouraging national leadership; and coordinating with other partners (OGAC, 2004). PEPFAR, while encompassing activities in over 120 countries, is focused on the development of comprehensive and integrated prevention, treatment, and care programs in 15 countries selected largely because they are heavily affected by HIV/AIDS (OGAC, 2005b). With regard to funding, $10 billion of the $15 billion authorized under the Act is to be allocated to efforts in these 15 countries over five years (OGAC, 2005b). Currently, the 15 PEPFAR focus countries are Botswana, Cote d’Ivoire, Ethiopia, Guyana, Haiti, Kenya, Mozambique, Namibia, Nigeria, Rwanda, South Africa, Tanzania, Uganda, Zambia, and Vietnam. PEPFAR has established additional goals for its prevention, treatment and care programs in these countries, specifically: to support prevention of 7 million new HIV infections, treatment of 2 million HIV-infected

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 leadership, human resources, and other capacities through collaboration of Mission teams with existing country partners, as well as with other donors. Study Goals and Approach The Act mandates that the Institute of Medicine (IOM) evaluate PEPFAR and directs the President to consider IOM’s findings. Specifically Sec. 101 (c) (1) of the Act states: “Not later than 3 years after the date of the enactment of this Act, the Institute of Medicine shall publish findings comparing the success rates of the various programs and methods used under the [PEPFAR] strategy. In prioritizing the distribution of resources under the [PEPFAR] strategy, the President shall consider the findings published by the Institute of Medicine under this subsection.” Thus, the current task of the IOM is to be responsive to this mandate and provide to Congress in time for reauthorization discussions an assessment of PEPFAR implementation at the three-year mark. The IOM has undertaken an independent, expert consensus committee process to plan, conduct, and report on this short-term evaluation, the scope of which is limited to the implementation of PEPFAR in the focus countries and does not include the U.S. contribution to the Global Fund to Fight AIDS, Tuberculosis, and Malaria (Global Fund) which is also coordinated by OGAC. The IOM Committee for the Evaluation of PEPFAR Implementation (the Committee) has three advisory Subcommittees focused on prevention, treatment, and care (see Appendix 1). This letter report outlines the Committee’s plan for the short-term evaluation. In addition to the size and complexity of PEPFAR, two additional features make the Committee’s task an especially challenging one: PEPFAR is of necessity a dynamic, evolving program and it is still relatively early in its implementation. As its name implies, OGAC has implemented PEPFAR on an emergency basis and, as the Act acknowledged, has had to “maintain sufficient flexibility and remain responsive to the ever-changing nature of the HIV/AIDS pandemic.” Thus PEPFAR is a rapidly-moving, continuously-evolving target for evaluation. The Committee is prepared to find considerable changes in PEPFAR throughout its evaluation, and has developed an approach to the evaluation that will allow it to adapt not only to changes in PEPFAR implementation, but also to what the Committee learns as the evaluation proceeds. The Committee’s evaluation plan should be viewed as a

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 work-in-progress that will be modified to reflect both the dynamic nature of PEPFAR and what the Committee learns—particularly as it visits the PEPFAR focus countries to directly observe implementation activities. Ultimately the “success” of PEPFAR will be judged by whether it has achieved its near-term goals of effectively supporting the prevention of 7 million HIV infections, treatment for 2 million people with HIV/AIDS with ART, and care of 10 million people infected with and affected by HIV/AIDS, as well as its longer-term goal of sustainable gains against the HIV/AIDS epidemics in the focus countries. However, although the Act was passed in May 2003, funds were not appropriated until January 2004, and the majority of the first year’s funding was not obligated until September 2004 (see Appendix 2). Thus, at the time of the Committee’s evaluation, PEPFAR will have been supporting the implementation of prevention, treatment and care programs in the focus countries for less than two years—less time than perhaps Congress had envisioned when they wrote the mandate for the IOM study. It would not be reasonable or feasible to evaluate PEPFAR solely against these goals so early in implementation, and therefore in this short-term evaluation the Committee plans instead to evaluate how PEPFAR is progressing toward 1) achieving these goals and 2) building the monitoring and evaluation capacity to demonstrate that it has achieved them. The short-term evaluation can provide insights into whether PEPFAR is making reasonable progress toward its goals and can suggest ways in which the program can be improved to ensure that it ultimately meets its goals. However, it cannot adequately measure what matters most—the impact on the lives of the people the legislation seeks to serve. In recognition of this, in addition to providing the short-term evaluation that will be responsive to the legislative mandate, the IOM Committee is planning a long-term evaluation to determine whether PEPFAR has ultimately succeeded in improving the lives of the people in the focus countries by preventing infections, treating patients, and caring for people. We plan to publish the plan for a long-term study shortly after publication of the mandated report. This long-term evaluation plan will be informed by the Committee’s fuller understanding of PEPFAR implementation and the challenging experience of evaluating it. In the development of its plana, the Committee has consulted widely and remains open to receiving input from the broad range of parties interested in and affected by PEPFAR. To develop the plan, the Committee (and the additional subcommittee members) convened first in February

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 2005 and then again in April 2005 when the Committee was fully-formed to review documentation of and hear testimony about PEPFAR. The questions highlighted in the plan reflect the multiple data sources that the committee believes capture the primary components of PEPFAR implementation. The Committee has considered peer-reviewed literature on global efforts to combat HIV/AIDS in resource-limited settings, documentation provided by OGAC, and other program-related materials. The Committee has heard testimony from OGAC officials and PEPFAR Mission staff; global organizations such as the World Health Organization, UNAIDS, Global Fund, and World Bank; and numerous PEPFAR grantees and partners. In addition, one Committee member and one staff attended the PEPFAR Annual Meeting in Addis Ababa, Ethiopia in May 2005. Following these initial information-gathering activities and several phone conferences to further explore information specific to prevention, treatment, and care, the Committee met in July 2005 to outline the short-term evaluation plan (see Figure 1).

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 Figure 1: Short-term Evaluation Plan Schematic

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 SHORT-TERM EVALUATION QUESTIONS Strategy Is the PEPFAR Strategy consistent with the Act and relevant global consensus? The same section of the Act that mandates the IOM study also specifies the major elements that should be contained in the PEPFAR Strategy. In addition, the legislation calls for the U.S. to take a strong leadership position with respect to the global effort to combat HIV/AIDS and also to coordinate its efforts with the international donor community. The global HIV/AIDS community, led by organizations such as those of UNAIDS, has published guidelines outlining international consensus strategies for the development and implementation of prevention, treatment, and care programs. Does the PEPFAR strategy address the major elements of the Act such as: Developing specific objectives, approaches and strategies for activities related to HIV/AIDS prevention, treatment and care? (Sec. 101.a. 1 & Sec.101.b.S.C) Focusing prevention funding, educational messages and activities on the reduction of HIV/AIDS behavioral risks? (Sec. 101.a.4 & Sec.101.b.S.C) Linking resources to program objectives and establishing of priorities for the distribution of resources based on factors such as population characteristics and needs as well as the existing infrastructure or funding levels? (Sec. 101.a.6 & 9 & Sec.101.b.S.M) Maximizing United States capabilities in the areas of technical assistance and training? (Sec. 101.a.8 & Sec. 101.b.S.H) Improving coordination and reducing duplication among relevant executive branch agencies, foreign governments, and international organizations? (Sec. 101.a.3 & Sec.101.b.3.K&L) Focusing on strategies developed to meet the needs of women, orphans and families? (Sec. 101.b.3.E, G, R, & S)

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 Promoting the development and implementation of strategies and programs designed to enhance the development of health care infrastructure, delivery systems and leadership capacity? (Sec. 101.a.2 & Sec. 101.b.S.C & D) Establishing monitoring and evaluating programs in order to measure success of the strategies, promoting successful models, and terminating unsuccessful programs? (Sec. 101.b.3.B&N) Is the PEPFAR strategy consistent with best practices outlined in global consensus documents related to prevention, treatment and care of HIV/AIDS including: The Global Strategy Framework on HIV/AIDS (UNAIDS, 2001) Intensifying HIV Prevention (UNAIDS, 2005) A Public Health Approach for Scaling Up Antiretroviral (ARV) Treatment: A Toolkit for Programme Managers (WHO, 2003) The Framework for the Protection, Care and Support of Orphans and Vulnerable Children Living in a World with HIV and AIDS (UNAIDS and UNICEF, 2004) Joint ILO/WHO guidelines on health services and HIV/AIDS (ILO/WHO, 2005) Has OGAC modified the PEPFAR strategy to reflect insights gained and lessons learned during implementation? Resources How is PEPFAR allocating resources and what is the effect on implementation? Both the Act and the PEPFAR strategy include guidance on the allocation of resources under the program. The legislation authorizes an annual appropriation for PEPFAR and outlines prioritization for resources in terms of program type (e.g., 20% for prevention, 55% for treatment, 15% for care, 10% for orphans and vulnerable children), and population and/or country characteristics (e.g., size and demographics of the HIV positive population, existing infrastructure). The PEPFAR strategy outlines allocation mechanisms including country allocations based on each country’s five-year strategic plan and performance assessments related to reaching annual prevention, treatment, and care targets, as well as a central funding mechanism for regional activities. Although there are a number of constraints on resource allocation, OGAC leadership

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 and country missions have discretion in the allocation of resources amongst agencies, countries, programs, and activities. How is PEPFAR allocating resources according to the priorities outlined in its strategy? How is PEPFAR managing congressional allocations in the context of harmonization? How do PEPFAR’s funding mechanisms affect operations of programs? How is PEPFAR addressing the coordination of regulations for procurement among and within cooperating agencies? How is the Central Procurement Contract functioning? What kinds of technical assistance is PEPFAR providing and how is it distributed? PEPFAR Management How is PEPFAR coordinating its efforts, evaluating it progress, and improving its programs? Coordination Coordination is one of the key elements of PEPFAR implementation identified by the Act. More specifically, the stated objectives within the legislation include strengthening coordination among U.S. government agencies “to ensure effective and efficient use of financial and technical resources” and to foster greater dialogue and synchronization of efforts with foreign governments and international organizations. OGAC has emphasized its “new way of doing business” and has focused a great deal of effort on its role as “Coordinator” across U.S. government agencies and with other donors. How well are U.S. government agencies coordinating under the auspices of the Office of the Global AIDS Coordinator? How well coordinated are PEPFAR operations between headquarters and missions? How well coordinated is PEPFAR with other HIV/AIDS donors at all levels?

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 Monitoring and Evaluation Measurement to assess progress, to improve programs, and for accountability is critical to the success of PEPFAR. With some unavoidable overlap, questions related to PEPFAR’s support of the focus countries’ monitoring and evaluation frameworks and capacity are emphasized in the harmonization and capacity building sections respectively. The questions emphasized in this section pertain primarily to PEPFAR’s efforts to monitor and evaluate itself and demonstrate that it has achieved its goals. How is PEPFAR monitoring progress on the major components of its program? Coordination Harmonization Intermediate Milestones for: Prevention of 7 million HIV infections Treatment of 2 million HIV-infected people with ART Care for 10 million people infected with and affected by HIV/AIDS Capacity Building Integration How is PEPFAR planning to evaluate its success in the long-term? Prevention of 7 Million HIV Infections Treatment of 2 Million HIV-infected People with ART Care for 10 Million People Infected with and Affected by HIV/AIDS Reduction of the HIV/AIDS Epidemic in the Focus Countries Improved Survival and Quality of Life for People in the Focus Countries Sustainable Programs and Systems Enhanced Knowledge Base How does PEPFAR manage difficult monitoring and evaluation issues such as measurement of infections prevented and attribution of people treated or cared for? How does PEPFAR assure the quality of programs and services? What steps is PEPFAR taking to ensure that the necessary epidemiologic data and other information are available from the focus countries to assess the initiative’s short-, as well as long-term impact?

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 Is PEPFAR supporting programs to generate the information they need for on-going program improvement? Learning Organization PEPFAR is an acknowledged “emergency” response. By design, the initiative moved rapidly into implementation before all of the arguably necessary pieces were in place, and many foundational elements remain to be developed—for example, OGAC has yet to issue the majority of guidance documents that it has said are under development. In recognition of this, however, PEPFAR has committed to “learning-by-doing”. Is PEPFAR functioning as a learning organization (Senge, 1990)? Does PEPFAR have an approach to learning by doing that allows them to generate and share evidence to improve programs? Harmonization How is PEPFAR harmonizing its efforts in the focus countries? In April 2004, UNAIDS, the United Kingdom, and the United States co-hosted a high-level meeting at which donors reaffirmed their commitment to strengthening national AIDS responses to be led by the affected countries themselves. They endorsed the “Three Ones” as guiding principles to improve the country-level response (UNAIDS, 2004b): One agreed upon HIV/AIDS Action Framework that provides the basis for coordinating the work of all partners One National AIDS Coordinating Authority, with a broad-based multi-sectoral mandate One agreed upon country-level Monitoring and Evaluation System. PEPFAR has expressed full commitment to the Three Ones, and although these principles were not formally in place at the time of the passage of the Act, the legislation calls for the U.S. to be coordinated with other donors, and thus PEPFAR’s commitment to the Three Ones is consistent with the legislation. All of the PEPFAR focus countries have national AIDS authorities, and thus with this commitment, “Harmonization” with the Three Ones of each focus country be-

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 came the centerpiece of the structure of PEPFAR. As such, it is central to the structure of the IOM evaluation. To what degree is PEPFAR harmonized with and what are the explanations for varying degrees of harmonization with the national plan, the national coordinating entity, and the national monitoring and evaluation framework? How do the goals that PEPFAR assigned to each country for number of infections prevented, number of people receiving treatment, and number of people cared for align with the country’s goals? How does PEPFAR manage differences between its plan and the national plan? How has PEPFAR affected the development of national plans? How does PEPFAR manage varying degrees of participation in the national coordinating entity? How does PEPFAR manage differences between its data collection and reporting requirements and those of the national monitoring and evaluation framework? Prevention, Treatment & Care Are PEPFAR prevention, treatment and care activities aligned with its strategy and the best available science and how is implementation progressing? Prevention, treatment and care activities are key components of an effective and comprehensive response to HIV/AIDS; helping to decrease risk, vulnerability, and impact of the epidemic (UNAIDS, 2001). The PEPFAR strategy identifies prevention, treatment and care programs as critical interventions, around which other PEPFAR strategies, such as strengthening health care systems, building capacity for long term sustainability of programs and the collection of strategic information, are structured (OGAC, 2004). The targets for PEPFAR-funded prevention, treatment and care programs—to support prevention of 7 million new HIV infection, treatment of 2 million HIV-infected people with ART, and care of 10 million people infected and affected by HIV/AIDS—were introduced in the State of the Union address given by President Bush in 2003, referenced in the Act, and later included in the PEPFAR strategy (OGAC, 2004).

OCR for page 1
Plan for a Short-term Evaluation of PEPFAR Implementation: Letter Report # 1 What programs and activities is PEPFAR supporting to address aspects of Prevention such as Behavioral Change, Blood Safety, Counseling and Testing, Post Exposure Prophylaxis, Prevention of Maternal to Child Transmission, and Safe Medical Practice? What programs and activities is PEPFAR supporting to address aspects of Treatment such as ARV Therapy, Clinical Laboratory Testing, and Prevention of Maternal to Child Transmission Plus? What programs and activities is PEPFAR supporting to address aspects of Care, particularly for Orphans and Vulnerable Children, such as Health Services and Social Support Services, Pain Management, and Prevention and Treatment of Opportunistic Infections? Are the programs and activities evidence-based? If evidence-base is incomplete or unclear, is PEPFAR supporting research to determine effectiveness? What programs and activities are focused on women and girls? How do programs and activities address stigmatization of and discrimination against people living with HIV/AIDS? How do programs and activities address issues of equity and human rights? How are prevention, treatment, and care programs and activities progressing? Has PEPFAR achieved the intermediate milestones established to measure progress in reaching its goals of supporting the prevention of 7 million HIV infections, treatment of 2 million HIV-infected people with ART, and care for 10 million people infected and affected by HIV/AIDS? Integration PEPFAR seeks to address prevention, treatment, and care in an integrated approach. The availability of these three interventions and effective linkages among them is expected to strengthen the effect of each intervention and have an impact that is greater than the sum of its parts. Integration of PEPFAR programs into the community and of HIV/AIDS programs into national health systems is intended to strengthen and enhance indigenous ownership and sustainability of these programs.