National Academies Press: OpenBook

Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program (2020)

Chapter: 3 Human Resources for Health Program Context, Vision, and Design

« Previous: 2 Evaluation Design, Methods, and Limitations
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 45
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 46
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 47
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 48
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 49
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 50
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 51
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 52
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 53
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 54
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 55
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 56
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 57
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 58
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 59
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 60
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 61
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 62
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 63
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 64
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 65
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 66
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 67
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 68
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 69
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 70
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 71
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 72
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 73
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 74
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 75
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 76
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 77
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 78
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 79
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 80
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 81
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 82
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 83
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 84
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 85
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 86
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 87
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 88
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 89
Suggested Citation:"3 Human Resources for Health Program Context, Vision, and Design." National Academies of Sciences, Engineering, and Medicine. 2020. Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program. Washington, DC: The National Academies Press. doi: 10.17226/25687.
×
Page 90

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

3 Human Resources for Health Program Context, Vision, and Design CONTEXT IN RWANDA LEADING UP TO THE PROGRAM The Costs of Conflict Rwanda’s recent history of conflict is essential context for the HRH Program’s origin and implementation progress. The 1994 genocide against the Tutsi in Rwanda resulted in an estimated 1 million deaths and 2 million displaced people. All social services were devastated, bringing to a halt all foreign assistance programs related to health, education, agriculture, and other economic development efforts. The toll on the health system included physical destruction of hospitals, laboratories, and equipment, not to mention the death or displacement of more than 80 percent of the country’s health professionals (USAID, 1996). By 1995, fewer than 10 pediatricians were reported to have been practicing, and there were no trained medical personnel, such as psychiatrists or trauma surgeons, to address issues emerging from the conflict (Binagwaho et al., 2014; Nsanzimana et al., 2015). The time and other investments required to counter the long-term public health effects of damaged social institutions were evident in the fact that it took 6 years for the first class of medical students to graduate (Willis and Levy, 2000). During and immediately following the genocide against the Tutsi, international organizations prioritized restoring primary health care services and water and sanitation systems (USAID, 1996). The International Committee of the Red Cross, United Nations Children’s Fund, and nongovernmental organizations (NGOs) such as Médecins Sans Frontières focused on repairing and operationalizing clinics with emergency health kits, medicines, supplies, and staff PREPUBLICATION COPY: UNCORRECTED PROOFS 3-1

3-2 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM to provide on-the-job training to auxiliary health workers (USAID, 1996). Immunization service delivery was disrupted, but the Ministry of Health (MOH) received international support to restore vaccine stocks and function to its immunization supply chain and logistics system (USAID, 1996). However, other capacities to detect and control infectious diseases, particularly the spread of HIV and cholera from migration and refugee camps at Rwanda’s border, would remain a challenge for years (Binagwaho et al., 2014; Nsanzimana et al., 2015). Although donors and NGOs were essential for providing necessary emergency relief, these agencies did not fully coordinate with the Rwandan government in the transition from emergency operations to rehabilitation of the health system. Consequently, health care services were fragmented and the MOH was slower to build institutional capacity to manage the country’s health priorities and sustain a health care delivery system (USAID, 1996). Major Developments in Rwanda’s Health-Sector Planning During the recovery period following the genocide against the Tutsi, the Government of Rwanda has consistently planned for how the health sector could better the health status of the population and support national poverty reduction goals. The release of the Health Sector Policy in 2005 outlined an overhaul of the sector, in light of decentralization efforts underway and in pursuit of more significant health gains (MOH, 2005a). Seven intervention priorities were established in seven areas: 1. Availability of human resources; 2. Availability of high-quality drugs, vaccines, and consumables; 3. Geographical access to health service; 4. Financial access to health services; 5. Quality of and demand for services in disease control; 6. Strengthening national referral hospitals and treatment centers; and 7. Strengthening the health sector’s institutional capacity. In particular, the Government of Rwanda would support financial access to health services by increasing public funding of health services (MOH, 2005a). The government strategy around this policy has evolved over time. The first Health Sector Strategic Plan (2005–2009) laid out plans for achieving the health-related Millennium Development Goals (MDGs), with attention to getting maternal and child mortality on track (MOH, 2005b). The following iteration of the Health Sector Strategic Plan (2009–2012) highlighted health systems strengthening in each of the seven intervention areas and continued to emphasize increased financial access to health services, noting the scale-up of community-based health insurance, as well as increased quality of care promoted by performance-based financing (MOH, 2009a). The World Health Organization (WHO) framework of health systems building blocks served as a foundation for the third Health Sector Strategic Plan (2012–2018), in which the MOH started to orient toward objectives in the post-MDG era and increasingly considered health resource management and governance mechanisms (MOH, 2012d). The resulting Health Sector Policy of 2015 touted key health achievements, including improved maternal and child health, increased community health worker coverage, and antiretroviral therapy (ART) and malaria program successes since 2004 (MOH, 2015a). However, it also outlined policies to address challenges in support for vulnerable and marginalized populations, sustainable health PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-3 system financing, and noncommunicable disease prevention and control, taking into account Rwanda’s epidemiologic transition and socioeconomic progress (MOH, 2015a). The current Health Sector Strategic Plan (2018–2024) centers on the Sustainable Development Goals and fully acknowledges these challenges and the need to reorganize the health system and involve other sectors of development to ensure universal health coverage for all needed services at all stages of life (MOH, 2018a). HIV in Rwanda The first case of HIV in Rwanda was documented in 1983; the Government of Rwanda initiated its response in 1985 with a blood donor screening program and has since sustained efforts to address the HIV epidemic (Kayirangwa et al., 2006). In 2009, Rwanda published its first National Strategic Plan on HIV/AIDS (MOH, 2009b). The development of national strategic plans on HIV/AIDS, the decentralization of the Rwandan health system, and movement toward community-based health insurance and performance-based financing facilitated its key achievements and remarkable progress toward achieving HIV epidemic control (MOH, 2009a,b, 2018c). Rwanda has also made steady improvements in increasing access to and coverage of ART over the past decade. The government’s commitment to confronting its HIV epidemic has accelerated progress toward the UNAIDS 90-90-90 targets prior to the Human Resources for Health (HRH) Program, as discussed in more detail in Chapter 7. HRH in Rwanda Despite improvements since the genocide against the Tutsi nearly destroyed the health infrastructure and resulted in acute health workforce shortages, which hindered health service delivery and served as a major barrier to HIV care and treatment, Rwanda continues to fall far below WHO’s recommended critical minimum threshold of 4.45 doctors, nurses, and midwives per 1,000 people (WHO, 2016). This shortfall comes from an insufficient number of trained health professionals relative to the need. Addressing HRH capacity had been featured in prior health-sector planning, but specific direction to develop a “competent, dedicated, productive, and accessible workforce” in support of the MOH’s mission of “providing quality preventive, curative, rehabilitative, and promotional services” was not articulated until the National Human Resources for Health Policy in 2014 (MOH, 2014c). These guidelines were developed for the planning, management, use, and monitoring of health-sector resources to operationalize the National HRH Strategic Plan (2011–2016) (MOH, 2011a), building on Rwanda’s Health Systems Strengthening Framework and Consolidated Plan 2009–2012 (MOH, 2009a). Notably, the National HRH Strategic Plan called for the development of a clear health service delivery plan, delineating specific competencies for each cadre providing services at each level of care, more emphasis on the quality of trained professionals and their distribution, and demand for reliable data to inform health resource management and evaluation of health system effectiveness (MOH, 2011a). Economic and Financial Context for Health in Rwanda As Rwanda recovered and rebuilt from the genocide against the Tutsi, its economy eventually also started to experience considerable growth. Figure 3-1 shows the trajectory of PREPUBLICATION COPY: UNCORRECTED PROOFS

3-4 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM growth from the early 2000s, as illustrated by gross domestic product (GDP), the total national expenditure, and the portion of that national expenditure that is government spending. The HRH Program was situated in this period of economic growth. FIGURE 3-1 The size of the Rwandan economy from 2000 to 2018 in current U.S. dollars. NOTES: Gross national expenditure is the sum of household final consumption expenditure, general government final consumption expenditure, and gross capital formation. General government final consumption expenditure includes all government current expenditures for purchases of goods and services (including compensation of employees). On average, inflation was about 6.5 percent in the 2000 to 2018 period (World Bank, 2019f), and the average growth of the official exchange rate in the same period was approximately 4.6 percent. SOURCES: World Bank World Development Indicators; World Bank, 2019c,e,g. Concurrently, there was growth in spending on health in Rwanda. Figure 3-2 shows the trajectories of growth in both per capita total health expenditures and per capita government spending on health, alongside the growth in per capita GDP. PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-5 FIGURE 3-2 Health expenditures in Rwanda from 2000 to 2016 (in purchasing power parity per capita). NOTES: Current health expenditure is the estimate of all health care goods and services consumed each year. Domestic general government health expenditure is public expenditure on health from domestic sources. On average, inflation was about 6.5 percent in the 2000–2018 period (World Bank, 2019f). The purchasing power parity conversion factor, for the 2000 to 2018 period, presented an average growth of about 4.4 percent (World Bank, 2019h). SOURCES: World Bank World Development Indicators; World Bank, 2019a,b,d. In this same time period, as Figure 3-3 shows, close to half of the health expenditure in Rwanda has consistently come from external aid, ranging from 41 percent to 54 percent between 2000 and 2016. Spending specifically for Rwanda’s response to HIV increased drastically during this period, and as of 2014 it has been consistently estimated at about $200 million annually. Of this, 80 percent comes from external funders, a larger proportion than for health overall (Nsanzimana et al., 2015). PREPUBLICATION COPY: UNCORRECTED PROOFS

3-6 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM FIGURE 3-3 Proportion of health expenditure in Rwanda by revenue source. NOTE: Domestic “all sources” comprises of out-of-pocket voluntary health insurance and domestic public expenditures. SOURCES: WHO Global Health Expenditure Database; WHO, 2019a. PEPFAR Funding Context Before the HRH Program was launched in 2012, the President’s Emergency Plan for AIDS Relief (PEPFAR) had an established funding history in Rwanda going back to 2003 (Binagwaho et al., 2016). Figure 3-4 shows PEPFAR total annual planned funding for programs in Rwanda from fiscal year (FY) 2009 to FY 2019, including the years surrounding the HRH Program. 1 Amounts shown in the bar graph were PEPFAR allocations to treatment, testing, prevention, health systems strengthening (HSS), government and systems, care, and applied pipeline. Percentages indicate proportion of annual funding allocated for HSS over time (PEPFAR, 2019c). PEPFAR’s total planned funding for its Rwanda program portfolio declined overall from 2009 to 2019. The proportion of funding allocated to HSS increased during the period between FY 2013 and FY 2016 relative to surrounding years, with a peak in FY 2013. This period is concurrent with the timing of financial support for the HRH Program. Before FY 2013, PEPFAR planned funding in Rwanda for HSS had not exceeded $9 million and had remained under 10 percent of total planned funding. Between FY 2013 and FY 2016, PEPFAR funding for HSS activities ranged from $24.7 million in FY 2013 (27 percent of its total portfolio) to $11.8 million (17 percent of its portfolio) in FY 2016. Since FY 2016, funding for HSS activities in the portfolio has not exceeded $1.9 million, or 2 percent of total planned funding. In the same period between FY 2013 and FY 2016, there was a decrease in the proportion of funding allocated to other program areas. In contrast, the proportion of PEPFAR planned funding devoted to HSS 1 This graph was generated from publicly available PEPFAR planned funding data, downloaded June 16, 2019, from PEPFAR’s Panorama Spotlight website. Planned funding information was used because publicly available expenditure information was less granular and was not available from the start of the HRH Program. PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-7 activities between FY 2009 and FY 2019 in other East African Community countries has been less variable and has not exceeded 13 percent of the total portfolio. 2 FIGURE 3-4 PEPFAR Rwanda planned funding by program area and percent allocation for HSS. NOTES: The bar graph shows amounts allocated to each program area indicated and the percentage of funding dedicated to health systems strengthening (HSS). These activities are classified under the OHSS code by PEPFAR and defined as “contribut[ing] to improvements in national-, regional- or district-level health systems,” and notes that these “[a]ctivities may be focused on health systems building blocks themselves as well as on institutions and processes that strengthen the building blocks and their interactions.” The Treatment category includes funding for programs classified under Adult Treatment, Antiretroviral Drugs, and Pediatric Treatment. Testing includes funding for programs under HIV Testing and Counseling. Prevention includes funding for programs classified under Blood Safety, Injection Safety, Injecting and Non-Injecting Drug Use, Prevention of Mother-to-Child Transmission, Sexual Prevention: Abstinence/Be Faithful, Sexual Prevention: Other Sexual Prevention, and Voluntary Medical Male Circumcision. Government & Systems includes funding for programs classified under Laboratory and Strategic Information. Care includes funding for programs classified under Adult Care and Support, Pediatric Care and Support, Tuberculosis/HIV, and Orphans and Vulnerable Children. Applied Pipeline includes appropriated but unspent funds carried over from prior years. The U.S. government FY runs from October 1 through September 30. SOURCES: PEPFAR, 2019c,d. PEPFAR Investments in the HRH Program At the inception and during the first 3 years of its funding, PEPFAR considered the HRH Program to be a strategic or key priority within its overall portfolio (PEPFAR Rwanda, 2012, 2013, 2014). Initially, PEPFAR shifted resources from other activities to “transfer significant 2 The other East African Community countries include Kenya, Tanzania, and Uganda. In Kenya, PEPFAR planned funding for HSS activities has ranged from 1 percent to 3 percent of total planned funding between FY 2009 and FY 2019. In Uganda, funding for these activities has constituted between 2 percent and 5 percent of the PEPFAR portfolio, and in Tanzania, between 3 percent and 13 percent of the PEPFAR portfolio, during this time. South Sudan and Burundi have smaller total PEPFAR portfolios, averaging less than $16 million per year. PREPUBLICATION COPY: UNCORRECTED PROOFS

3-8 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM support to the area of HRH” (PEPFAR Rwanda, 2012) and thus fund the Program (PEPFAR Rwanda, 2012, 2013). Subsequently, the Program’s placement relative to other PEPFAR programmatic priorities decreased over time (PEPFAR Rwanda, 2013, 2014, 2015). The HRH Program was being funded during the window of peak PEPFAR funding for HSS in Rwanda, shown in Figure 3-4. Between FY 2011 and FY 2013, the Program was funded as a component of an existing U.S. Centers for Disease Control and Prevention (CDC) cooperative agreement with the MOH called “Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under PEPFAR” (also known as the Clinical Services Cooperative Agreement) (Mtiro, 2018). Preceding the HRH Program, CDC had already been providing direct support to the MOH for its HIV/AIDS-related programs (CDC, 2009c, 2012): • The integration of HIV services and programs into the health system at all levels; • Capacity building for infection control; • Injection safety; • Medical waste management; • Epidemiological investigation, lab management, pharmaceutical management, health communication, electronic medical record use, and data collection and analysis; • Use of terminology standards and registries; and • Training for physicians, community health workers, and service providers in clinical handling and management of sexual and gender-based violence. This funding also provided salary support and benefits for physicians, technicians, and data managers, as well as supporting quality improvement initiatives, performance-based financing evaluations, technical meetings with stakeholders, and management and audits of MOH/PEPFAR-supported facilities and sites (MOH, 2014a). The United States Agency for International Development (USAID) also provided some funds for the Program, although the amounts could not be confirmed (CDC, 2012; PEPFAR Rwanda, 2013). In addition to PEPFAR funding, the Program had a diversified funding base, evidenced by the engagement of the Global Fund to Fight AIDS, Tuberculosis and Malaria. 3 Starting in FY 2014, CDC used a separate cooperative agreement with the MOH, “Strengthening Human Resources for Health Capacity in the Republic of Rwanda under PEPFAR,” to fund the HRH Program (Mtiro, 2018). In 2015, PEPFAR’s priorities shifted, and the HRH Program was classified as a “noncore” investment as part of an analysis in alignment with PEPFAR’s new 3.0 strategy. The decision was made to end funding for the Program on March 30, 2017, instead of continuing PEPFAR support through June 30, 2019, the Program’s official closing date (Mtiro, 2018; PEPFAR Rwanda, 2015). Public reporting systems provided limited information about the actual amount invested by PEPFAR in the HRH Program. Table 3-1 shows available information regarding the mechanisms PEPFAR used to provide funding for the Program. 3 The Global Fund specifically sponsored the launch of Rwanda’s first dental school and development of dental curricula and partnerships (Seymour et al., 2013), as PEPFAR did not include dentistry in the list of specialties it would support. Examination of the dentistry school and program was not included in this evaluation. PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-9 TABLE 3-1 PEPFAR Funding Sources That Contributed to the HRH Program FY 2012 FY 2013 FY 2014 FY 2015 FY 2016 FY 2017 FY 2018 (Oct 2011–Sep (Oct 2012–Sep (Oct 2013–Sep (Oct 2014–Sep (Oct 2015–Sep (Oct 2016–Sep (Oct 2017–Sep 2012) 2013) 2014) 2015) 2016) 2017) 2018) U.S. Centers for Disease Control and Prevention (CDC)* Strengthening the Capacity of the $17,556,432 $22,168,552 $23,507,981 $0 $0 N/A N/A Ministry of Health to Respond to the (total (total (total cooperative HIV/AIDS Epidemic in the Republic cooperative cooperative agreement) of Rwanda Under PEPFAR agreement) agreement) (Award No. U2GPS002091; Mechanism No. 10825) Strengthening Human Resources for N/A N/A N/A $11,000,000 $10,500,000 Unknown** Unknown** Health Capacity in the Republic of Rwanda Under PEPFAR (Award No. U2GGH001614; Mechanism No. 17621) United States Agency for International Development (USAID)*** Unknown Award(s)/Mechanism(s) Unknown Unknown Unknown Unknown Unknown Unknown Unknown NOTES: N/A = not applicable (funding mechanism not used to contribute to the HRH Program that year). All amounts are in U.S. dollars. * Annual total disbursed funding by award/mechanism as reported through the U.S. HHS Tracking Accountability in Government Grants System (TAGGS) (accessed July 7, 2019). TAGGS reports the total amount disbursed through such agreements, but does not provide any further breakdown by program or activity. Therefore, the proportion of the amount that was specific to the HRH Program is not known in FY 2012 to FY 2014, when the HRH Program was funded as a component of the cooperative agreement, “Strengthening Capacity to Respond to the HIV/AIDS Epidemic,” that started in FY 2009. Starting in FY 2015, the HRH Program was funded through its own cooperative agreement, "Strengthening Human Resources for Health Capacity in the Republic of Rwanda under PEPFAR,” and the amounts reported in TAGGS are reflective of the amounts disbursed for the HRH Program: $11 million in FY 2015 and $10.5 million in FY 2016. Any amounts disbursed through CDC after FY 2016 are also not known. ** Information is not publicly available on whether amounts were disbursed through this or any other CDC award or mechanism after FY 2016. *** Although there is documentation that some PEPFAR investments in the HRH Program came from other implementing agencies, those amounts and sources are not reported publicly. Approval of USAID maternal and child health funds to be used for implementing the HRH Program is referenced in a 2012 CDC memo requesting a multiyear expansion supplement for the “Strengthening the Ministry of Health’s Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under PEPFAR” cooperative agreement. However, amounts were redacted and data on the portion of the award/mechanism allocated to the HRH Program were not publicly available (correspondence provided by CDC Rwanda in August 2018). In addition, Rwanda's Country Operational Plan FY 2013 referenced a planned shift of $2 million from USAID’s Family Health Project to the HRH Program, but the actual amount disbursed and allocated for the HRH Program is not publicly available. SOURCES: CDC, 2012; PEPFAR Rwanda, 2014; TAGGS, 2019. PREPUBLICATION COPY: UNCORRECTED PROOFS

3-10 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM HRH Program Budget and Ministry of Health Expenditures on Health The HRH Program budget proposed in 2011 was $151.8 million over 8 years (MOH, 2011b). The Program received external funding from the U.S. government, through PEPFAR, and from the Global Fund. The total funding amount from these external sources, as provided by the MOH, was just under $100 million, with approximately 60 percent coming from the U.S. government (see Table 3-2). This comprised 12 percent to 16 percent of the total annual PEPFAR investment in Rwanda. TABLE 3-2 HRH Program Budget by Year (U.S. Dollars) Fiscal Year U.S. Government Global Fund Total Annual Budget 2012–2013 12,300,163 6,775,325 19,075,488 2013–2014 14,971,013 6,775,325 21,746,338 2014–2015 12,976,798 6,775,325 19,752,123 2015–2016 11,000,000 5,729,026 16,729,026 2016–2017 10,500,000 5,847,111 16,347,111 2017–2018 0 5,519,856 5,519,856 2018–2019 0 362,246 362,246 Total HRH Program 61,747,974 37,784,214 99,532,188 Budget NOTES: Amounts and totals are reported as provided by the MOH in current U.S. dollars. On average, inflation was about 5.2 percent in the same period, and the average growth of the exchange rate was about 5.8 percent between 2012 and 2018 (World Bank, 2019f,g). U.S. government and Rwandan FYs are not the same: U.S. FY runs October 1 through September 30; Rwandan FY runs July 1 through June 30. SOURCE: Financial data provided by the MOH. The amounts disbursed from CDC to the MOH, however, were slightly lower than the budgeted amounts (see Table 3-3). Where there was a balance, CDC approved carrying it over to the next year, with the exception of 2015 to 2016, after PEPFAR had deemed the Program noncore. The total amount budgeted for the HRH Program comprised about four percent of the total annual health budget for the MOH as reported in the Ministry’s Health Resource Output Tracking Report for available concurrent years (see Table 3-4). Given that government expenditure comprises less than half of the total expenditure on health in Rwanda (as described in Figure 3-3), the HRH Program likely represented less than 3 percent of total health spending in Rwanda at the time it was implemented. PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-11 TABLE 3-3 CDC Disbursements for the HRH Program by Year (U.S. Dollars) Disbursement from Year CDC Approved Carryover Expenses Balance 2012–2013 12,300,163 6,630,040 5,670,123 2013–2014 12,577,279 5,670,123 4,898,183 13,349,219 2014–2015 12,769,798 13,349,219 26,119,017 — 2015–2016 11,000,000 10,558,378 441,622 2016–2017 10,058,378 10,500,000 Total 58,705,618 SOURCE: Financial data provided by the MOH. TABLE 3-4 Ministry of Health Budget and Expenditures Reported Preceding and During the HRH Program (U.S. Dollars, Rounded to Millions) FY 2010/2011 FY 2011/2012 FY 2012/2013 FY 2013/2014 FY 2014/2015 Total Budget 514 milliona 539 millionb 533 millionc * 499 milliond Total Expenditure 420 milliona 514 millionc * 481 milliond 480 milliond NOTES: Amounts shown are the current amounts reported at the time each source report was published. Amounts were provided in Rwandan francs (RWF) after FY 2011/2012 and therefore converted to U.S. dollars (USD) using the following World Bank historical exchange rates, rounded to the nearest dollar: in 2013, 1 USD = 647 RWF; in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g). *Expenditure data for FY 2012/13 and budget data for FY 2013/14 were not reported. SOURCES: a MOH, 2012c; b MOH, 2012c, 2013b; c MOH, 2013b; d MOH, 2018b. Figure 3-5 provides the context of the MOH’s broader concurrent workforce-related expenditures before and during the HRH Program. From FY 2010/2011 to FY 2014/2015, as the amount of MOH expenditure going toward health workforce increased there was a small decline in workforce expenditure as a proportion of the total MOH expenditure, from 42 percent to 39 percent. Concurrently, there was an increase in the proportion of investments for drugs and other consumables, and materials and equipment. However, trends in these data are difficult to interpret because of changes from year to year in how expenditures were reported by the MOH. It is also not possible to clearly interpret how the overall expenditures of the MOH intersected with the HRH Program expenditures because these reports do not indicate whether these expenditures include or exclude the funds expended as part of the HRH Program. In addition, there may be other sources of expenditures on HRH in Rwanda. PREPUBLICATION COPY: UNCORRECTED PROOFS

3-12 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM FIGURE 3-5 MOH expenditures FY 2010/2011 to FY 2014/2015 (U.S. dollars). NOTES: USD = U.S. dollars. Amounts shown are the current amounts reported at the time each source report was published. Different input categories were used each year and aggregated according to the four major categories shown in the figure. Expenditures after FY 2011/2013 were provided in Rwandan francs (RWF) and converted to USD using the World Bank conversion for each year and rounded to the nearest whole number: in 2011, 1 USD = 600 RWF; in 2012, 1 USD = 614 RWF; in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g). * Other comprises public relations and awareness such as advertising, campaigns, and communications for health; domestic and international travel and transport costs; indirect costs; infrastructure; overhead and general administrations costs; and vehicles and maintenance. ** Workforce comprises in-service training and workshops, incentives for community health workers, preservice training and workshops, salaries for contracted government personnel, salaries for government personnel, salaries for nongovernmental personnel, salaries for short-term consultants, service provision, technical assistance, and performance-based financing. *** Materials & Equipment comprises medical and nonmedical equipment. **** Drugs & Health-Related Consumables comprises commodities, consumables, and drugs. SOURCES: MOH, 2012c, 2013b, 2016a, 2018b. The categories of expenditures aggregated into the broad category of “Workforce,” and how they were reported, varied from year to year (see Table 3-5). PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-13 TABLE 3-5 MOH Health Workforce Expenditures with Category Breakdowns by Year (U.S. Dollars) FY 2010/2011 FY 2011/2012 FY 2013/2014 FY 2014/2015 Total Expenditures 419,636,014 514,014,611 480,938,416 479,889,043 Workforce (percent of 40% 39% Expenditures) Workforce 176,132,460 212,751,636 Expenditure Amount Workforce Category Breakdowns (as Reported) Direct Salaries/Labor 109,328,045 28% (of total 30% (of total Costs/Remuneration expenditures) expenditures) Capacity Building 42,369,081 7% (of total 5% (of total expenditures) expenditures) Incentives 24,435,334 In-Service Training 40,200,881 and Workshops Incentives for 29,381,321 Community Health Workers Preservice Training 8,064,948 and Workshops Salaries (Contracted 35,049,224 Government Personnel) Salaries (Government 27,784,464 Personnel) Salaries 1,475,639 (Nongovernment Personnel) Salaries (Short-Term 3,998,622 Consultants) Service Provision 18,688,571 Technical Assistance 21,554,278 Performance-Based 26,553,688 Financing Social Benefits 5% (of total 4% (of total expenditures) expenditures) NOTES: Amounts shown are the current amounts reported at the time each source report was published. In FY 2013/2014 and FY 2014/2015 expenditure categories were reported as percentages of the total expenditure. Amounts reported in Rwandan francs (RWF) were converted to U.S. dollars (USD) using the World Bank’s historical exchange rates, rounded to the nearest dollar: in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g). SOURCES: MOH, 2012c, 2013b, 2016a, 2018b. PREPUBLICATION COPY: UNCORRECTED PROOFS

3-14 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM A consistent pattern was that salaries and labor costs represented the largest proportion of expenditures. When reported, categories such as capacity building, preservice training, and workshops represent a very small amount of the total investment. HUMAN RESOURCES FOR HEALTH PROGRAM VISION AND DESIGN Political, Economic, and Social Context During the Design of the HRH Program By 2012, when the HRH Program was launched, Rwanda was viewed as an ideal environment, not least because it was perceived as being a “peaceful and stable country,” which was seen as an important factor for successful program implementation. Health-sector achievements in the intervening years (see Chapter 7) made the landscape especially attractive for international donors: Everyone wants to work with a winning team. Even before HRH, there [was] evidence on the ground the Rwandan health system was performing. We, in the last 15 years, have reduced maternal mortality tremendously, neonatal mortality, achieved the [Millennium Development Goals], and we are on the track with the Abuja Declaration for Health, which is the amount allocated for health in general…one of the highest in Africa. So, all these health indicators were improving, [and] I think it becomes much easier working with someone who is already busy working for himself and help them achieve results, than trying to invest where you don’t see results anyway. (09, University of Rwanda Administrator in Obstetrics and Gynecology) These achievements built on years of previous engagements by and with the United States and other partners in Rwanda, which created energy and interest in continuing to develop the health sector: From the beginning, USAID and CDC were on board…. Other partners were interested, WHO and UN agencies, other bilaterals … also I think a lot of U.S. institutions had an interest here … family medicine and Tulane University, but also Yale University was present. They already had an MOU [Memorandum of Understanding] with the Ministry of Health, which was then integrated into the HRH [Program]. So there was definitely a strong presence to create a core group of people, to create a momentum on that. (22, Non-Government of Rwanda HRH Program Administrator from an International NGO) The perceived political will, on the part of the United States at the time and the Government of Rwanda, also created enabling conditions to develop and fund a holistic HSS program such as the HRH Program: [T]he government of Rwanda and even the side of the U.S. government—of CDC, even the Global Fund … [were] very committed and convinced that this was the right program to face the situation, and so there were a lot of high-level advocacy PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-15 and enthusiasm on both sides. (20, Government of Rwanda HRH Program Administrator) Across all respondent groups, the perception was that the Rwandan government’s leadership in the context of a strong health system were keys to successful programming: The first factor is political will. The Ministry of Health put so much effort in HRH programming for it to succeed … the environment was good enough for the HRH Program to operate without any challenge as the system was already well established. (30, Former Government of Rwanda Program Administrator and PLHIV Representative) There is a huge leadership commitment in Rwanda. The government…wants to be a partner in everything that is happening, working hand in hand with partners and being in the driver’s seat…. That’s a huge success factor. (11, International NGO Representative) If you brought a program in a country where the leadership is not very strong in delivering results, we wouldn’t be where we are. (09, University of Rwanda Administrator in Obstetrics and Gynecology) The facilitating environment in Rwanda, from the perspective of U.S. institution (USI) faculty participating in the HRH Program, had three parts. First, Rwanda was viewed as having a “flexible” health system that could change with an evolving health workforce. Second, the government was confident in its successes in fighting HIV. Finally, existing infrastructure, such as roads and Internet access, was seen as facilitating successful implementation. HRH Program Vision At the time of the HRH Program’s design and funding, Rwanda was implementing the Health Sector Strategic Plan 2009–2012 and the HIV/AIDS National Strategic Plan 2009–2012, both of which had goals around the availability, quality, and rational use of HRH in service of improved health outcomes, in alignment with Vision 2020 goals of health equity, universal health coverage, increased access to and delivery of quality health care (CDC, 2014; MOH, 2011b; Uwizeye et al., 2018). However, there were several critical obstacles within the Rwandan health care system, including the shortage of skilled health workers, poor quality of health worker education, and inadequate infrastructure, equipment, and management in health facilities (MOH, 2011b, 2014b, 2016b). According to program documents, the HRH Program was designed to address those challenges as a large, systems-based, health sectorwide initiative to scale up institutional and training capacity and create a high-quality, sustainable health system in Rwanda capable of providing “world-class care” (CDC, 2012; MOH, 2011b, 2012b, 2014b, 2016b). Upgrading the health professional workforce to be of “sufficient quantity and quality to meet the national need,” would “contribute to the distal goal of improved availability and quality of care in Rwanda” (MOH, 2014b). Strategies would focus on increasing skill levels and specialization of health professionals and educators, establishing high-quality training sites, accelerating recruitment and PREPUBLICATION COPY: UNCORRECTED PROOFS

3-16 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM retention of students, creating a culture and career ladder for health professional teaching, and expanding research partnerships and academic exchange (MOH, 2011b). According to the Government of Rwanda, sustainable effect of the HRH Program would be the country’s ability to produce a supply of new skilled, specialized health workers and Rwandan health educators and to ensure adequate infrastructure, equipment, and supplies without external financial support (MOH, 2014b). This country vision aligned with PEPFAR 2.0’s strategic vision and focus on developing a sustainable HIV response that supported HSS activities with partner governments (PEPFAR, 2019a,b). With the inception of PEPFAR 3.0, PEPFAR’s strategic vision pivoted away from this horizontal approach and toward targeted epidemic control (PEPFAR, 2014). This shift in priorities does not fully align with Rwanda’s strategic goals, as the Government of Rwanda has consistently planned for developing the capacity of its health system. Iterations of Rwanda’s Health Sector Strategic Plans for 2009 to 2024 have emphasized HSS in key intervention areas, community-based health insurance, and quality of care through performance-based financing, while outlining policies to address challenges for vulnerable and marginalized populations, sustainable health system financing, and prevention and control of noncommunicable diseases (MOH, 2009a, 2012d, 2018a). Against this backdrop, the HRH Program was conceived with the goal of ultimately increasing the quality of health care delivery and the overall health care system in Rwanda, contributing to the government’s “mandate in terms of developing capacity of the health staff” (45, Government of Rwanda HRH Program Administrator). Interview respondents for this evaluation shared the same understanding of the HRH Program’s rationale, which they saw as being borne out of a need to rebuild the health system following the genocide against the Tutsi and in the face of an HIV epidemic and an aging population of people living with HIV (PLHIV): [O]ur discussion in 2010 to 2012 was how can we develop this capacity, rebuild this health system by working on the areas which are highly affected, which is medical personnel, nurses, specialists, doctors. The demand was also high not only for a single disease, but for everything. For HIV/AIDS is not seen as a single disease as such; sometimes it starts as a small virus and it ends by being a whole medicine. (01, Government of Rwanda HRH Program Administrator) Thus, the HRH Program “was fully integrated into the health system” with potential effects beyond “just one disease” such as HIV (87, Government of Rwanda Program Administrator). This is in keeping with the evolution of Rwanda’s health sector strategic planning at the time, which focused on health-related MDGs, and specifically maternal and child mortality in the first phase (2005–2009), and transitioning to a focus on HSS and financial access to health services in the second phase (2009–2012). The third phase (2012–2018) emphasized health resource management and governance mechanisms (MOH, 2012d). HIV Achievements in Rwanda By 2011, when the HRH Program was being designed, Rwanda had made notable achievements in addressing the HIV epidemic relative to other countries in the region, as discussed in Chapter 1. Concurrent to the evolution in the health-sector strategic planning was a process of decentralization that facilitated increased access to HIV services. The first phase PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-17 (2000–2005) facilitated the expansion of HIV services to lower-level facilities (Binagwaho et al., 2016). This effort helped to distribute and expand HIV care by integrating community health workers who were providing services at the village level to the more advanced care offered at health centers, and district and referral hospitals (Binagwaho et al., 2016). In 2008, during the second phase of decentralization (2006–2010), the Rwandan health system moved authority to the district level to launch new HIV treatment sites. However, coordination of services between HIV and other care remained poor. While districts received funding for health, they received funding for essential HIV/AIDS services much more slowly as PEPFAR and the Global Fund followed different paths to decentralization (Nsanzimana et al., 2015). The second Health Sector Strategic Plan in 2009 called for the need to better integrate HIV/AIDS care into routine health services. To support this effort, the Government of Rwanda entered into a cooperative agreement with CDC to increase staffing (CDC, 2009b). It also started transferring management of HIV patients from international partners to government-run programs (PEPFAR, 2010; PEPFAR Rwanda, 2011). This transition resulted in a sustainable and successful HIV program that now has the internal expertise to manage HIV care at all levels (Binagwaho et al., 2016). During the third phase of decentralization, starting in 2011, the Rwanda Biomedical Center restructured HIV coordinating mechanisms to facilitate better integration with other disease-specific programs (Nsanzimana et al., 2015). The previous HIV programs were dissolved, including the National AIDS Control Commission, so other disease-focused programs could be leveraged to create operational efficiencies, especially given dwindling resources. The third Health Sector Strategic Plan (2012–2018) called for the integration of HIV services at a decentralized level, the need to improve quality, and the need to maintain trained and adequate numbers of staff at all facilities (MOH, 2012d). The result of this decentralization has been a rapid increase in the number of facilities offering ART services, from four in 2002 to 552 in 2016, as reported in the Rwanda Integrated Health Management Information System. Respondents related the success of Rwanda’s health sector to the window of opportunity provided to the HRH Program. Respondents perceived Rwanda as having “achieved all the expressed outcomes that we entered with the intent of achieving,” providing an opportunity to explore what could be done With a vertically funded program that has achieved outcomes that were desired but now is turning to look at the larger needs of those already infected HIV- positive individuals for their broader health care needs as we move forward in their care. (25, Non-Government of Rwanda HRH Program Administrator and U.S. Government Donor) Rwanda was seen as an interesting test case for a different model of development that leveraged the gains achieved through a vertical program to strengthen an entire health system. To the extent that HIV outcomes were considered in the design of the HRH Program, the MOH viewed building a specialized health workforce as a priority in addressing long-term care for PLHIV and infectious disease control, which was considered a moral, epidemiologic, and economic necessity—a “triple imperative” (Binagwaho et al., 2013). Respondents working with NGOs in the HRH and HIV space in Rwanda felt that an HRH Program with the goals of strengthening the health system could improve HIV outcomes: PREPUBLICATION COPY: UNCORRECTED PROOFS

3-18 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM [T]he program was not designed specifically to improve the lives of people with HIV/AIDS. It is a health system strengthening intervention, and it will help the system; obviously, it will support the people who have HIV/AIDS because of their frequent contact with the health system. (05, Other International NGO Representative) HRH Program Goals, Objectives, and Strategy While there was general agreement on the overarching vision of the HRH Program, there was less congruence between program documentation and interview respondents on the intended mechanisms for achieving this vision. Program documents illustrated that developing the Rwandan health education system remained a principal component of the HRH Program. Throughout the Program, the MOH noted that focusing on health education was a mechanism for filling the gap between the supply and demand of highly qualified health professionals in Rwanda (MOH, 2011b, 2015b, 2016b). By the midterm review, however, key HRH Program activities such as twinning through participating USIs that formed the Academic Consortium were deemed necessary to “gap-fill” and “simultaneously build the long-term capacity of their counterparts and host institutions” (MOH, 2016b). This evolved from the initial premise that the Program was designed to simply fund the mechanism for dramatically increasing production of health workers in a short time frame (MOH, 2011b). As highlighted in the 2011 HRH Program proposal, the original objectives were to increase the number of physicians and physician specialists, continue advancing the skill levels of nurses and midwives, introduce the role of health manager into the Rwandan health system, launch the Rwanda School of Dentistry (the first class began in the fall of 2013 at the University of Rwanda), and build the institutional capacity of their health professional schools and clinical teaching hospitals to sustain high-quality health education (MOH, 2011b). Most of these objectives continued to be referenced as core program expectations informing the development of core indicators in their eventual monitoring and evaluation (M&E) plan and as objectives in a results framework for the midterm review (see Table 3-6). Many outlined areas of work, such as expanding research partnerships and academic exchange, were also generally consistent through the years though with varying specificity. For example, the 2015 programmatic/technical work plan emphasized “internationally benchmarked curricula” in one of the areas of work, whereas other program documents stated that the HRH Program would “implement quality, competency-based instruction” (MOH, 2011b, 2015b, 2016b). Notably, it was not until the MOH’s performance and measurement plan development process, which began after program implementation, that three overarching strategic outcomes emerged. Two of these outcomes highlighted the role of the University of Rwanda’s College of Medicine and Health Sciences (CMHS) in providing high-quality health professional education programs and accompanying training environments, alongside the third outcome of establishing a sustainable, skilled, and specialized health professional workforce (MOH, 2014b, 2015b). In addition, M&E had not been designated as a specific area of work until this point; the 2014 M&E plan acknowledged the challenge of establishing a baseline for all program areas, because M&E efforts were being articulated after implementation (MOH, 2014b). This was reflected in inconsistencies in baseline and target goals for the number of health professionals in several PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-19 cadres, particularly in the number of nurses and midwives to be produced by the HRH Program once it began to deemphasize the upgrading of A2 to A1 nurses (MOH, 2011b, 2014b). 4 TABLE 3-6 Evolution of the HRH Program’s Goals and Approaches 2016 2011 Rwanda HRH Rwanda HRH 2014 2015 Program Midterm Program, 2011– HRH Monitoring & Project Narrative and Review Report 2019, Funding Evaluation Plan, Programmatic/Technical (October 2015–June Proposal March 2014 Work Plan 2016) Aim Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Identified Critical shortage of Reduce the critical Critical shortage of Challenges skilled health shortage of skilled skilled health to workers health professionals workers Achieving Aim Poor quality of Improve the quality of Poor quality of health health worker health professional worker education education education Sustainability of Inadequate Increase and diversify health education infrastructure and health care worker system equipment in specialties health facilities Enhance infrastructure Inadequate and equipment in management of health facilities and health facilities educational sites Improve health facilities and educational site management Goals Increase the Increase the number of Increase the total (2011) number of general practitioners number of physicians physicians from practicing in Rwanda Core 633 to 1,182, and from 625 to 1,182 Increase the number Program the number of (disaggregate by of physician Expectations physician cadre) specialists in priority (2014) specialists in areas clinical areas such as internal Increase the number of Complete medicine, family physician specialists Results and community (subspecialists) from Framework medicine, 128 to 551 obstetrics and 4 A2 nurses have completed secondary school education; A1 nurses receive a diploma after 3 years of training at a higher education institute; A0 nurses are graduates of a 4-year Bachelor’s program and may go on to enroll in a Master’s program (Uwizeye et al., 2018). PREPUBLICATION COPY: UNCORRECTED PROOFS

3-20 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM 2016 2011 Rwanda HRH Rwanda HRH 2014 2015 Program Midterm Program, 2011– HRH Monitoring & Project Narrative and Review Report 2019, Funding Evaluation Plan, Programmatic/Technical (October 2015–June Proposal March 2014 Work Plan 2016) Objectives gynecology, (2016) pediatrics, surgery, and anesthesiology from 150 to 551 Dramatically Increase the number of Increase the total advance the skill nurses and midwives number of nurses and level of from 9,670 to 10,200 midwives nurses/midwives by increasing the Increase the skill number of level of nurses and nurses/midwives midwives with A0 credentials from 104 to 1,011 and the number of nurses/midwives with A1 credentials from 797 to 5,095. These actions will increase the overall number of nurses/midwives from 6,970 to 9,178 Introduce the role Introduce the role of Introduce the role of of health manager health manager and health manager at into the Rwandan increase their number district hospital level health system and from 7 to 157 increase the (introduce trained number of trained health manager health managers position in district from 7 to 157 hospital and develop their job description) Launch the Launch the school of Increase the number Rwanda School of dentistry and increase of oral health Dentistry, and the number of health professionals increase the professionals from 122 number of oral to 424 health professionals from 122 to 424 PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-21 2016 2011 Rwanda HRH Rwanda HRH 2014 2015 Program Midterm Program, 2011– HRH Monitoring & Project Narrative and Review Report 2019, Funding Evaluation Plan, Programmatic/Technical (October 2015–June Proposal March 2014 Work Plan 2016) Build the Create teaching institutional hospitals and medical capacity of the schools that have the medical, nursing, infrastructure, oral health, health equipment, and management institutional capacity schools, and to sustain high-quality clinical teaching education hospitals to sustain high-quality health education Strategic To improve the Improved capacity of the Outcomes capacity of the University of Rwanda’s University of CMHS to implement Rwanda’s College of quality, competency- Medicine and Health based health professional Sciences (CMHS) to education programs implement quality, competency-based health professional education programs To establish an Established environment enabling environment in CMHS schools and in CMHS schools and training sites (e.g., training sites (e.g., referral hospitals) referral hospitals) to conducive to facilitating facilitate improved improved health health professional professional education education By 2019, have a Established a sustainable, skilled, sustainable, skilled and and specialized health specialized health professional workforce professional workforce in Rwanda in Rwanda Areas of Increase skill levels Increase skills levels and Increase skill levels Work and specialization specialization of health and specialization of of health care care professionals and health care professionals and educators professionals and educators educators PREPUBLICATION COPY: UNCORRECTED PROOFS

3-22 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM 2016 2011 Rwanda HRH Rwanda HRH 2014 2015 Program Midterm Program, 2011– HRH Monitoring & Project Narrative and Review Report 2019, Funding Evaluation Plan, Programmatic/Technical (October 2015–June Proposal March 2014 Work Plan 2016) Establish high- Establish high-quality Establish high- quality clinical clinical training sites and quality clinical training sites and schools training sites and schools schools through procurement of infrastructure and equipment and improvement of health management capacity Accelerate Support recruitment and Accelerate recruitment and retention of trainees and recruitment and support student students in nursing and support student retention midwifery, biomedical retention laboratory sciences, medicine and surgery, and health management programs Implement Develop and deliver Implement integrated, internationally integrated, competency-based benchmarked curricula competency-based curricula curricula Increase the Increase the engagement Increase the importance of of health professionals in importance of teaching and learning, teaching, and teaching and careers careers in health scholarships in health professions professions Build institutional Recruit to and retain Build institutional capacity for health faculty in Rwanda by capacity for health education working with USIs education Increase collaboration between health professional education stakeholders Enhance the recruitment and retention of graduates in the health care and health education sectors PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-23 2016 2011 Rwanda HRH Rwanda HRH 2014 2015 Program Midterm Program, 2011– HRH Monitoring & Project Narrative and Review Report 2019, Funding Evaluation Plan, Programmatic/Technical (October 2015–June Proposal March 2014 Work Plan 2016) Development of faculty for health professional education Expand research Expand scientific Expand research partnerships and partnerships and partnerships and academic exchange academic exchange academic exchange Monitoring and evaluation NOTE: CMHS = College of Medicine and Health Sciences; HRH = human resources for health; USI = U.S. institution. SOURCES: MOH, 2011b, 2014b, 2015b, 2016b. Qualitative data reveal a lack of congruence in the pathways to reach the Program vision; respondents reported a range of strategies including improving quality of care, producing high- quality health workers, building primary care, building specialty care, strengthening the medical education system (including faculty), and improving the availability of equipment and infrastructure. Among Government of Rwanda respondents, the Program’s main objective was unequivocally to build a larger cadre of health care workers across specialties: “The vision of the MOH was to improve the shortage of HRH, improve their quality in terms of skills and knowledge, and how to deal with some of the major issues that we had here in Rwanda. (48, Government of Rwanda HRH Program Administrator) To do this, as one HRH Program trainee articulated, it was necessary to build postgraduate training programs (32, University of Rwanda Non-Twinned Faculty and Former University of Rwanda Student in Obstetrics and Gynecology). In contrast, the perceived objectives of the HRH Program among most University of Rwanda and USI respondents was to upgrade the number and skills of Rwandan health educators. When probed, one Government of Rwanda respondent who had been involved in the Program’s design vehemently disagreed with the assertion that building capacity within the University of Rwanda was a programmatic objective: Creating people who want to teach—it’s something else. Don’t forget that we [the HRH Program] create people to give care. That was the objective. And this has to be well understood. We needed people to provide care. It’s good that we PREPUBLICATION COPY: UNCORRECTED PROOFS

3-24 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM reinforced the University, but our program was not to reinforce the University. (18, Former Government of Rwanda HRH Program Administrator) Some USI faculty reported that the objectives and design of specific specialties changed significantly over the course of the Program. For example, the initial objective of the nursing and midwifery activity was to contribute to the skills upgrading of A2 nurses, but this shifted to focusing on building an A0 nursing cadre through the development of the Master of Science in Nursing (MSN) program: The HRH Program started with increasing the level of A2 nurses to A1, to advanced diploma. Because the majority of health care providers in Rwanda were A1.... Then they started upgrading the A2 to advanced diploma. (31, University of Rwanda Administrator in Nursing and Midwifery) Similarly, the objectives around the Master of Hospital and Healthcare Administration (MHA) program evolved as its developers gained an understanding of the context and needs in hospital administration: Initially, the program is for the [USI] faculty to come to Rwanda and work in a hospital and pair with the hospital administrator. So, instead of doing a formal education program … we pair with them and help them side by side to do the day- to-day operations…. Later on, they discovered they don’t have an official curriculum for hospital management so they started the development of a hospital management program that was based in the University of Rwanda School of Public Health. (06, USI Faculty in Pediatrics) The emphasis on specialized care over primary care evolved throughout the Program and was not clearly understood among different stakeholder groups. The Program comprised both pre- and in-service training activities, but the former were prioritized, in accordance with national HRH policy because preservice education “is less costly and gives immediate hand[s] on skills to the health professional” (MOH, 2014c). The strategy to focus on immediate hands-on skills was a central tenet of train-the-trainer models and the twinning program, which planned for Rwandan faculty and new graduates to phase out Academic Consortium faculty or to train colleagues in district hospitals, building capacity for local ownership and sustainability. Efforts to strengthen the medical and nursing programs and the medical education system were underway before the HRH Program. In FY 2008 and FY 2009, USAID funded the University of Colorado to second “a family medicine faculty member … to provide extensive practical teaching, postgraduate supervision, and assistance with [the] development of the Family Medicine program” and to assist in integrating HIV/AIDS into postgraduate medical program curricula (PEPFAR Rwanda, 2008, 2009). By FY 2010, the University of Colorado’s efforts had been subsumed under a large CDC-funded capacity-building program with Tulane University and had expanded to include support for a 4-year postgraduate medical program aimed at preparing “physicians to function with a broad clinical scope … to better address the burden of disease existing in Rwanda’s rural communities.” Twenty-three physicians, “including seven in Family and Community Medicine,” were enrolled (PEPFAR Rwanda, 2010). PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-25 The in-country postgraduate medical program was initiated under the former Minister of Health, Jean Damascene Ntawukuriryayo, and continued by his successor, Richard Sezibera (Flinkenflögel et al., 2015). Although some of the work performed under Tulane’s cooperative agreement with CDC was transitioned to the University of Rwanda, the agreement was not renewed at the close of its 5-year term under Agnes Binagwaho’s tenure as Minister of Health (PEPFAR Rwanda, 2012). One respondent who was involved in the design and early implementation phases noted that primary care was central to the HRH Program at the start, but was excluded from the list of clinical areas during implementation: The emphasis on the primary care thing was one of the cornerstones of the HRH Program. The U.S. government came back to us when we submitted the proposal with lots of questions and concerns. They wanted to make sure that we really had an emphasis on primary care. So, we brought memos and answers to … certify that primary care was going to continue to be a key emphasis of the program. But then, as soon as the program was approved and funded, shortly [thereafter] the family medicine residency was discontinued and there was a shift from primary care to specialty care. (22, Non-Government of Rwanda HRH Program Administrator) According to one respondent representing an international NGO, the reasoning behind supporting specialized care over primary care was not well understood, because it Flew in the face of the primary health care focus that people were having, particularly USAID, at the time [and there was] fear of donors that are putting a lot of resources in the specialized services is going to be done at the detriment of primary health care. (05, International NGO Representative) In contrast, a senior MOH official commented that there was no need to build primary care in Rwanda when the HRH Program started, because there were—and continue to be—other investments and efforts to strengthen primary care, but no efforts focused on specialized care. Notably, few respondents spoke about the objectives of the HRH Program in relation to HIV. Two University of Rwanda administrators reported that the Program’s goal should have been positioned in the context of “transition from acute burdens of HIV as a signature illness to much more chronic disease management” (02, University of Rwanda Administrator). A former HRH Program administrator expanded on this idea: [T]he main problem was, we have a huge HIV population who have been on treatment for a very long period of time, and they started to develop other kinds of diseases—internal disease, where they might need surgery. They were in need of more specialized type of care that [could] be provided by very few specialists. (45, Government of Rwanda HRH Program Administrator) The literature echoes the HRH Program–HIV connection, in which it was imperative for the MOH to build a specialized health workforce to address long-term care for PLHIV and infectious disease control (Binagwaho et al., 2013). PREPUBLICATION COPY: UNCORRECTED PROOFS

3-26 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM DESIGN PROCESS Figure 3-6 depicts the overall timeline of the HRH Program. In 2011, the Government of Rwanda submitted an unsolicited proposal, which was funded in 2012 under the expanded Clinical Services Cooperative Agreement (CDC, 2012). Prior to funding, 18 USIs submitted letters of intent to join the Academic Consortium, although one withdrew before the Program was launched (MOH, 2011c). USI participation in the Program increased until 2015, when institutions began withdrawing. An MOU was established between the MOH and a new USI in 2018, after PEPFAR’s investment had ended. Membership in the Academic Consortium provided USIs with a mechanism through which to establish annual MOUs with the MOH. The Consortium was also a mechanism for determining the clinical purview of each USI. For example, Yale University was an obstetrics and gynecology partner, and New York University and Emory University provided support to the nursing programs, both upgrading nurses and the establishment of the MSN program. USIs under the Academic Consortium contracted with individuals to work at the University of Rwanda to “twin” with and mentor existing Rwandan faculty, aid in strengthening existing residency programs and establish new programs, provide direct teaching services to Rwandan students, and, in some cases, provide direct care to patients. With few exceptions, the USIs contracted with U.S. citizens and did not engage regional faculty who could lend practical experience and knowledge from a more locally relevant context to the Program (see Chapter 4 for a more detailed discussion). PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-27 FIGURE 3-6 HRH Program time line. NOTE: HRH = human resources for health; MSN = Master of Science in Nursing; PEPFAR = President’s Emergency Plan for AIDS Relief; USI = U.S. institution. Data-Informed Design It was clear from interview respondents, including those who were engaged in the design of the HRH Program, that no specific baseline assessment was done prior to launching the Program. Six Government of Rwanda current program administrators referenced supporting documents, such as reports or an assessment, that informed the Program; three of these respondents specifically cited the Third Health Sector Strategic Plan and the Human Resources for Health Strategic Plan 2011–2016: [F]rom the HRH Strategic Plan, that’s why they had all the information: How many specialists do we have? What are the problems? What are the number of physicians per population? And then from the strategy that was developed in 2010, if my memory is serving well, that’s what informed that we need the PREPUBLICATION COPY: UNCORRECTED PROOFS

3-28 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM program. We need trainees to provide care to the population. (45, Government of Rwanda HRH Program Administrator) The Third Health Sector Strategic Plan makes reference to an extensive situation analysis and comprehensive midterm review conducted in 2011, which informed the plan’s priority to “improve quantity and quality of human resources for health (planning, quantity, quality, management)” (MOH, 2012d). Other HRH Strategic Plan objectives include increasing the number of trained and equitably distributed staff and improving health worker productivity and performance in part by improving the quality of preservice training (MOH, 2011a). Similarly, PEPFAR’s Rwanda Country Operational Plans reference Tulane University’s collaboration with other donors and the “National Technical Working Group for Human Resources in an in-depth needs assessment … designed to determine the health workforce needs in Rwanda, taking into account the disease burden, existing cadres and ongoing trainings (pre- and in-service)” (PEPFAR Rwanda, 2010, 2011). However, respondents from the Clinton Health Access Initiative (CHAI), who collaborated with the MOH in the design of the HRH Program, felt the MOH was not in a position to examine or plan for their HRH needs: I think at the time, the reason the Ministry asked for our help was that they didn’t have, you know, a very strong or well-established vision for where they should go in HRH…. So, even though there were some documents put together, people didn’t really take it seriously. I don’t think there was any kind of plan for longer term.... I haven’t seen needs assessments that were done in an objective way [or] any kind of transparent process. (22, Non-Government of Rwanda HRH Program Administrator) Collaboration During Design The HRH Program design process appeared to have involved members of the MOH and CHAI, without involving other key partners, such as the Ministry of Education (MOE), Ministry of Finance and Economic Planning (MINECOFIN), or implementing partners working in HIV and HRH. CHAI’s role, with financial support from the ELMA Foundation, was one facilitator. At the request of then-Minister of Health Richard Sezibera, CHAI convened an HRH working group to prepare a road map and strategy. These documents formed the basis of the HRH Program proposal, which CHAI “unilaterally wrote … and submitted to the U.S. Embassy” (20, Government of Rwanda HRH Program Administrator). As part of this process, CHAI worked with the MOH to determine programs within the PEPFAR portfolio whose funding could be reduced or terminated to cover the costs of the Program: [CHAI] helped to figure out where the money was going to come from … basically, it’s a budget and expenditure reporting from partners. We analyzed that, compared that to the Country Operating Plan of the U.S. government. And then we sat down with the Ministry of Health, discussed the programs they wanted to keep, what they wanted to cut. We went through the list, suggested some cuts to find them funding. And then the Ministry went back to the U.S. government and said, “cut this, cut that” and that’s how they found the funding. (20, Government of Rwanda HRH Program Administrator) PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-29 This was corroborated by PEPFAR during one of the committee’s open sessions. Once the Program was funded, CHAI was asked to stay on, and managed the HRH Program through the first year of implementation while the MOH set up internal management structures and onboarded staff. Although the MOH–MOE relationship faced challenges at the beginning of the HRH Program, it strengthened over time. Not involving the MOE during Program design and early implementation was perceived as a mistake: We actually started the HRH Program without the Ministry of Education involvement. Only after we launched it and it was a reality, I think, that’s when we realized that we had made a mistake by not involving education. To be fair, I think things were also in flux; even in the government, it was not totally clear where postgraduate training was supposed to be, whether in the School of Medicine, it was under Ministry of Health, or Education and where the respective roles started and stopped. But at some point, they [MOH officials] had to go to see the Minister of Education, Dr. Vincent Biruta, and basically apologize and explain what happened … from then on, things started to work well with the Ministry of Education. (22, Non-Government of Rwanda HRH Program Administrator) One MOH respondent involved in the early stages of the Program described the MOE as “a contractor” (18, Former Government of Rwanda HRH Program Administrator). A non- Government of Rwanda HRH Program Administration respondent expressed his view that there was no relationship between the MOH and MOE before the Program, and that a relationship had been built through MOH–MOE steering committees formed to set standards for health care professional education: We used to sit together—Minister of Health, Minister of Education, University [of Rwanda]—to try to understand the challenges and provide solutions together. I think from the HRH Program we strengthened this collaboration between the Ministry of Health and Education. (48, Government of Rwanda HRH Program Administrator) FINANCIAL MANAGEMENT The 2011 HRH Program proposal indicated that a Single Project Implementation Unit would be established “to centralize programmatic, financial, and administrative management functions” (MOH, 2011b). PEPFAR investments in the HRH Program flowed directly to the MOH, although according to one respondent, a “parastatal” had been set up to manage the money, where funds were disbursed by MINECOFIN. One respondent involved in the early phases of the Program noted that this design was intended “to try and eliminate the high cost of an international NGO go-between … and having a government that had demonstrated their ability to manage and oversee, monitor, and evaluate programs” (25, Non-Government of Rwanda HRH Program Administrator). PREPUBLICATION COPY: UNCORRECTED PROOFS

3-30 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM The Permanent Secretary of the MOH had oversight over the HRH Program budget. The Government of Rwanda employed a Public Financial Management system, in which funds were held in an account separate from the National Treasury. The system facilitated budget oversight, because the University of Rwanda used the same system. According to MOH data, the bulk of the expenditures from PEPFAR investments in the HRH Program went to USIs, followed by equipment procured for health professional education and clinical training (see Table 3-7). TABLE 3-7 HRH Program Expenditures of PEPFAR Investments (U.S. Dollars) Recipient Budgeted Amount Disbursed Amount USIs 31,180,833 29,804,744 University of Rwanda 5,830,730 5,830,730 Equipment Procurement 16,083,568 17,901,279 Travel 429,600 429,600 Supplies 3,092,778 3,092,778 Program Management 2,529,771 1,646,487 Total HRH Program Expenditures 59,147,280 58,705,618 NOTES: The fluctuating exchange rate was set by the Rwandan National Bank. The original Program proposal had a specific line item for equipment maintenance, amounting to $1.5 million. A similar line item is not reflected in these expenditure data. SOURCE: Financial data provided by the MOH. Table 3-7 includes only funds provided by CDC and does not reflect other investments from sources such as the Global Fund and the MOH. According to the MOH, the $441,661 difference between the budgeted amount and the disbursed amount was not funded by CDC. The MOH articulated the importance of unmeasurable inputs that were critical for the running of the HRH Program, including overhead costs expended by the University of Rwanda for managing its facilities, unallowable expenses incurred by USIs in recruiting and managing USI faculty, and other essential financial inputs by the MOH, such as annual travel allowances for all residents to move across sites for their rotations and visit their families when they were rotating at a site away from their primary residence. Financial audits of the Global Fund’s investments during periods concurrent with the HRH Program (2014, 2016 to 2018), and therefore presumably inclusive of funds applied toward the HRH Program, yielded no major concerns. Systems of internal control were typically found to be “generally” or “partially” effective. The 2014 audit identified three areas for improvement: data quality assurance, supporting documentation, and low absorption of funds (Global Fund, 2014). The 2016–2018 Global Fund audit found, however, that grant oversight and assurance mechanisms needed significant improvement (Global Fund, 2019). Audits of other investments in the MOH had similar findings. An audit of the MOH’s management of Gavi funds rated the MOH as partially satisfactory (Gavi, 2018), and an assessment of a Swedish International Development Cooperation Agency–supported program that centered on building research and PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-31 higher education capacity identified three key issues in the MOH’s financial management: lack of consolidated financial reporting; delays in disbursements to Swedish partner universities; and slow budget execution that affected implementation (Tvedten et al., 2018). Additionally, a thorough search for a CDC audit of the HRH Program was conducted, and none was located. Beyond the financial management processes, program administrators expressed some frustration about PEPFAR’s processes and restrictions on how funding could be used, especially in the light of the HRH Program’s holistic and integrated design: The second [challenge] was how surprisingly hard this was using PEPFAR money … it was a pain in the ass, I’m sorry for my language..... For anything, any comment, any point, you need to ask for authorization…. People from Atlanta will have to sign something and Rwandans as well and … you need to write 200 pages of document. The equipment that was supposed to come in year 1 came in year 3 or 4. It was really not easy to use the funds. (45, Government of Rwanda HRH Program Administrator) There are too many specialties that these fund were not really able to support and for us, I can see how [HIV] is related to that. A radiologist doesn’t just treat a non-HIV patient, it treats both and the most complicated cases come from the people who have HIV. Same with dentistry. (03, Government of Rwanda HRH Program Administrator) Institutional Financial Arrangements The 2011 HRH Program proposal outlined several key principles guiding the relationship between the MOH and USIs. The MOH would receive funds directly from the U.S. government and issue contracts to USIs. MOUs were established between USIs, the MOH, and MINECOFIN. Each USI had separate coordinators for the academic aspects of the MOU and for finances. The mismatch of FYs between USIs, the Government of Rwanda, and CDC, compounded by yet another difference in the University of Rwanda’s academic year, presented an additional contractual challenge that affected payment scheduling: CDC came with a specific CDC fiscal year for some reason which was March- April [and] that was even more difficult because it pushes the quarter to another one, so from October you will see that it comes to the other year and all this payment to universities was done quarterly. (20, Government of Rwanda HRH Program Administrator) The 2016 midterm review also noted this challenge. Upfront stipulations around financial management included that USIs’ overhead to the programmatic and administrative functions could not exceed 7 percent of total direct costs for recruiting faculty, predicated on the assumption that the Government of Rwanda would be responsible for some of these administrative activities (MOH, 2011b,c). The HRH Program proposal also stipulated that USI faculty were expected to commit to residing in Rwanda for at least 11 months and would receive: PREPUBLICATION COPY: UNCORRECTED PROOFS

3-32 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM Salary and benefits … lower than they could obtain at their universities in the U.S. This reflects the fact that the HRH Program is a development project meant to serve poor people in a resource-poor country and also reflecting the fact that this is an educational opportunity for the U.S. faculty (MOH, 2011b). USI faculty were mostly contract hires, not existing faculty receiving time to dedicate to the HRH Program. PROGRAMMATIC MANAGEMENT APPROACHES AND CHALLENGES The HRH Program experienced challenges in its design, launch, and execution that were both internal to Rwanda (between the MOH and MOE and stakeholders outside the Government of Rwanda) and external to Rwanda (between the MOH and USIs). Interinstitutional Relations MOH–MOE Relations As indicated above, the design phase did not actively engage the MOE, a gap that had implications in the early implementation period. Some faculty at the University of Rwanda were aware of the HRH Program before it was launched, and reported being excited that former U.S. President Bill Clinton and Rwandan President Paul Kagame had participated (37, University of Rwanda Administrator), although respondents knew of other faculty members who were not aware of the Program: I was a little bit ahead of my colleagues who had those challenges of not being informed ahead, but for me, I knew that the program was there. (80, USI Faculty in Nursing) This absence of communication about the Program included incoming students who were anticipating joining training programs abroad, but were told they would be enrolled in programs in Rwanda: Because we were not informed of the Program … yeah it was a surprise! We did exams, competing for scholarships [in other East African countries]. We were responsible for getting a mission letter from in-country ourselves…. When we presented them…. They said, “No, no, we are not going” and they didn’t inform [us] that we have a program locally. We kept asking, “What happened?… What is next?” Then it was 6 months later they say, “All those who have scholarship are going to have your program locally.” (47, University of Rwanda Faculty and Former Student in Nursing) The MOH received and managed external HRH Program funding and was responsible for signing MOUs with USIs. The MOE had little input into the allocation of funds. The roles and the responsibilities of the MOH and MOE, which required a collaborative relationship between the two ministries, was articulated by a respondent representing a professional association: PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-33 The Ministry of Health had nothing to do with the assessment of the students or their teaching, not very much, except that the Ministry of Health was the one that was creating the environment in which people were trained because, in the Ministry of Health they were responsible for the good running of the hospital, and without a hospital that is capable of providing good care you cannot talk about good teaching of postgraduates.… The academic head of medicine [at the University of Rwanda] was the one who was looking at the planning of the rotations, he would be the one who decides who comes to King Faisal, who goes to CHUK [Centre Hospitalier Universitaire de Kigali/University Teaching Hospital, Kigali], who goes to Kanombe … how long that person will stay and will also come for a visit to see if people are there to get reports of what is happening. (35, University of Rwanda Non-Twinned Faculty in Internal Medicine and Professional Association Representative) During implementation, University of Rwanda administrators reported that the communication between the MOE and the MOH was clear and managed through a steering committee chaired by the principal of CMHS. They reported that he had oversight over all of the operational and programmatic issues with HRH trainees, while the MOH dealt with contractual issues and accreditations: We have a steering committee … chaired by the principal of the [College of Medicine and Health Sciences] and is composed [of] all deans of the college where we have the HRH Program,… and the team of HRH from the Ministry of Health. We sometimes invite also HoDs [heads of department] and if it’s necessary, we invite the HRH faculty. (37, University of Rwanda Administrator) MOH respondents also mentioned the value of the steering committee as the entity “in charge of giving the guidance and making the policies” (20, Government of Rwanda HRH Program Administrator), which strengthened the ministries’ relationship. MOH respondents also articulated the roles and responsibilities, while reinforcing the idea that “it is one government” and neither ministry could unilaterally determine the priorities of both sectors (03, Government of Rwanda HRH Program Administrator). The steering committee was also responsible for ensuring no duplication of efforts between the HRH Program and other investments in strengthening health workers, such as the Capacity Development Pooled Fund. USI faculty described the relationship between the MOH and the MOE as “painful” (23, USI Twinned Faculty in Nursing), affecting their ability to support the review and approval of new or updated curricula and accreditation of new programs: I would just think that the Ministry of Health would want the best teachers but that’s not what they have been doing. (16, USI Faculty in Obstetrics and Gynecology) PREPUBLICATION COPY: UNCORRECTED PROOFS

3-34 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM MOH–International NGO Relations As with the design, respondents from international NGOs felt the MOH was not inclusive of other organizations in the program’s management and implementation: My view is that at least the Ministry of Health doesn’t try to involve other partners…. My feeling is that the Government of Rwanda, the Ministry of Health, has taken an attitude of saying: “You know, this is our business and we are not….” (05, International NGO Representative) Coordination [between the HRH Program and other programs] wasn’t great … activities tended to occur in silos. (26, International NGO Representative) MOH–USI Relations The 2016 midterm review highlighted challenges in contracting processes: lack of clarity around leave and other policies, timeline and processes for obtaining work permits and licenses for USI faculty, and funding and reporting requirements that resulted in delays in drafting MOUs; delays in annual revising and reissuing MOUs; late award notification, which delayed program funding and USI reimbursements; and a need to accommodate USI and HRH Program processes delaying submission of invoices and payment (MOH, 2016b). Interview respondents for this evaluation echoed many of these challenges. From the USI perspective, challenges with contracting included contracts being limited to 1 year, requiring annual renewal, differences in malpractice standards, and the exclusion of overhead from contracts (described in the previous section), which was viewed as an “extra horror” that was “not realistic for a long-term project with a U.S. institution” (15, USI Faculty in Hospital and Health Administration). MOH delays in issuing contracts, raised in the midterm review and in interviews, had notable implications on the presence of USI faculty at the University of Rwanda and their personal and professional futures: We did not start immediately in year 1…. We had no contract. There were people from here who were told that there would be contracts in July. We got our contract in November…They don’t get it. People stay here without contracts for periods of time…. [The MOH coordinator] was like, “Oh, we’ll get the money,” but when? My faculty is the sole provider of her family. She can’t work for free. (17, USI Faculty in Obstetrics and Gynecology) We would stay on a few months to finish up or to make a transition. But it’s very unsettling for the people [Rwandan faculty and residents] that you are working with when you say, “I hope I’ll be back.” (16, USI Faculty in Obstetrics and Gynecology) Program administrators who worked outside of the Government of Rwanda shared that some USI faculty “pulled out” of the HRH Program because the financial gap (due to these contract conditions) “was putting enormous pressure … so it was too much risk” (22, Non- PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-35 Government of Rwanda HRH Program Administrator). There was also fluctuation in the number of USIs participating in the Program. In 2011, 18 programs had submitted letters of intent to join the Academic Consortium (11 in medicine, 6 in nursing and midwifery, and 1 in health management and global health delivery). In 2012, one of those institutions withdrew prior to program launch, though an additional 10 joined the program (9 in medicine, 1 in nursing and midwifery). USIs began exiting the Academic Consortium in 2015 with 3 exiting in 2015 (2 in medicine, 1 in nursing and midwifery); 3 in 2016 (2 in medicine, 1 in nursing and midwifery); 1 in 2017 (from nursing and midwifery); 5 in 2018 (4 in medicine and 1 in health management and global health delivery); and 5 in 2019 (2 in medicine and 3 in nursing and midwifery). Two USIs did join during that period: 1 in 2015 (in health management and global health delivery) and 1 in 2018 (in medicine). That USIs were expected to spend no more than 7 percent on administrative activities related to recruitment reinforced a feeling that these institutions were nothing but a “recruitment firm, rather than true partners” (06, USI Twinned Faculty in Pediatrics) in building the institutional capacity to provide high-quality health professional education. This feeling was exacerbated in situations when the MOH told USIs its needs, sometimes with little warning, and with USIs conducting the screening and proposing individuals and the MOH making final decisions. Additionally, differing benefits between the two countries created obstacles, especially around maternity leave, which was part of the contracts USIs issued but was not a Government of Rwanda benefit. A similar obstacle was perceived with sick leave: In Rwanda, we know that sick leave is something that you take when you are sick. In the U.S., sick leave are accrued and you have day per month as a right and when you do not take them you will add those accrued sick leave into your annual leave to sometimes we could have faculty doing 60 days or 40 days of leave and you know for an academic year where you have to work 1 year, it is not easy … it had implications on the budget, meaning you are paying someone who did not work for 60 days, did not deliver. (20, Government of Rwanda HRH Program Administrator) The 2011 proposal indicated that “basic benefits [would] be provided” to USI faculty, naming only health insurance, emergency evacuation insurance, adequate housing, and roundtrip airfare (MOH, 2011b). Government of Rwanda staff who managed the HRH Program also sometimes felt disappointed by USIs, which was seen as affecting programmatic success: There are also some institutions that disappointed us in the middle just by withdrawing from the program or not sending the required qualified staff. All those already are jeopardizing the good implementation of the program. (03, Government of Rwanda HRH Program Administrator) Communication from USIs to the MOH about the physician specialists, nurses, and midwives selected to travel to Rwanda, specifically their experience and seniority, also affected the relationship, although this was seen as having improved “over time as we went on mentioning this challenge the profiles changed and I think they would send even better people” (48, Government of Rwanda HRH Program Administrator). PREPUBLICATION COPY: UNCORRECTED PROOFS

3-36 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM MONITORING AND EVALUATION Reporting requirements for the HRH Program evolved over the years, in part because of the changing funding mechanisms as well as the broader shift in standards for M&E. In April 2012, before the Program was integrated into the MOH’s periodic program reports for the Clinical Services Cooperative Agreement, the MOH submitted to CDC a detailed work plan and budget for the Program. The work plan was similar to a logical framework and included goals, objectives, activities, a quarterly timeline, and program indicators that were consistent, if not more detailed, than CDC reporting requirements at the time (CDC, 2009a; MOH, 2010, 2012a). Program indicators for the HRH Program and other components of the Clinical Services Cooperative Agreement, however, remained largely output measures (CDC, 2009a; MOH, 2010, 2012e). In 2013, the work plan for the Clinical Services Cooperative Agreement Continuing Application called for the development of an “M&E framework and indicators” as one of the HRH Program’s “major programmatic/technical activities” (MOH, 2013a). An open invitation for Program participants to volunteer to join one of two stakeholder groups—an in-country M&E Working Group (comprising HRH Program members and Rwandan faculty) and a USI M&E advisory group—was also sent in 2013 to support the M&E plan development. As a result, the MOH submitted to CDC the Program’s “Human Resources for Health Monitoring & Evaluation Plan” in March 2014 (MOH, 2014b). In addition to the three main strategic outcomes described previously, the M&E plan provided outputs, detailed process and outcome measures, data sources, annual performance targets, reporting frequency, and baseline and midterm evaluation components; the rigor of this plan reflected PEPFAR’s increasing emphasis on M&E (PEPFAR, 2011, 2012) and use of more sophisticated metrics, which had begun in 2009 with the development of PEPFAR’s Next Generation Indicators. In contrast to PEPFAR’s earlier metrics, which had been largely output measures, the Next Generation Indicators sought to measure program coverage and quality using both process and outcome indicators. In October 2014, when CDC issued a noncompetitive funding opportunity announcement to continue the Program’s funding at a level of $14 million per year for 5 years under its own funding mechanism, PEPFAR’s M&E requirements for the Program were both comprehensive and rigorous: CDC will work with the awardee to implement, using PEPFAR and national indicators as a base, a robust monitoring and evaluation system designed to track the implementation of HRH efforts, preservice training, and maintenance of critical health care equipment that have already begun and will continue after [U.S. government] support for these activities concludes. This system will provide evidence needed to drive a programmatic process for decision making. This will be built on a three-tiered approach that captures data through routine program monitoring, program evaluation, and specific surveys (CDC, 2014). To fulfill these requirements, the MOH submitted a project narrative and evaluation and performance measurement plan in its award application (MOH, 2014d). Under the new cooperative agreement, CDC required the MOH to revise and submit a more detailed evaluation PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-37 and performance measurement plan within the first 6 months of the award (CDC, 2014). In 2015, the MOH submitted a revised programmatic/technical work plan that documented “SMART (specific, measurable, achievable, relevant, and time-based) Objectives,” activities, a timeline, responsible parties, and “process measures” (MOH, 2015b). According to the 2014 M&E plan, a point person, trained in the use of data collection tools for routine reporting, would be available for every teaching hospital and health facility (MOH, 2014b). Three main evaluations at three time points would also be planned to measure the program’s outcomes and effect, recognizing that a baseline might only be available for some areas because the Program had already started (MOH, 2014b). Most indicator data for program monitoring would be collected semiannually from the following sources: the Health Management Information System, Human Resources Information System, health education institution records, and program records. A Health Education Institution Survey and Hospital Survey were also planned, although no evidence was found to show that either survey occurred (MOH, 2014b). The HRH Program midterm review reported that quarterly surveys were administered to individuals involved in the Program, although response rates were low, especially among Rwandan faculty (MOH, 2016b). An organizational case study of the Program concluded that its prioritization of implementation costs and underestimate of administrative, monitoring, and evaluation costs—coupled with the substantial size and scope of the Program—created a “mismatch between needs and availability of resources and expertise” and a lack of resources for M&E (Cancedda et al., 2018). From the perspective of interview respondents, little investment was made in M&E processes, which had also been noted in the Program’s midterm review. A USI respondent commented that having a single M&E officer for the Program was insufficient: One M&E officer, like 100 U.S. faculty on the ground from 21 different institutions across 5 or 6 physical locations in Rwanda and in 4 major disciplines, and an officer barely had the capacity to figure out what kind of questionnaire to write, much less disseminate to collect it, analyze it, and report on it in a way that was receivable by funders, partners, government, and university. Impossible. Totally impossible. Total fail. (83, USI Faculty in Surgery) Another respondent stated that there was “a high-level M&E document [referring to the 2014 M&E plan], but it’s not specific to actual activities on a yearly basis” (22, Non-Government of Rwanda HRH Program administrator from an international NGO). SUSTAINABILITY PLANNING Under PEPFAR’s partnership frameworks, an important component of PEPFAR’s 2.0 strategy (PEPFAR, 2014, 2019b), sustainability was defined as “supporting the partner government in growing its capacity to lead, manage, and ultimately finance its health system with indigenous resources (including its civil society sector), rather than external resources, to the greatest extent possible” (PEPFAR, 2009). PEPFAR’s role was to move from supporting the expansion and provision of direct services to offering technical assistance to partner countries to work towards sustainable programming (PEPFAR, 2019b). This was operationalized through a 5-year Partnership Framework, which aimed to produce results in HIV prevention, care, and PREPUBLICATION COPY: UNCORRECTED PROOFS

3-38 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM treatment, and to position countries “to assume primary responsibility for the national responses to HIV/AIDS in terms of management, strategic direction, performance monitoring, decision making, coordination, and, where possible, financial support and service delivery” (PEPFAR, 2009). For the HRH Program, the 2011 proposal referred to sustainability in the context of the intention that, after 8 years, the Government of Rwanda would operate the Program under its own budget. The proposal included a rationale for the availability of the financial resources that would be needed to absorb the newly trained health workers and maintain equipment purchased under the HRH Program. It argued that the government’s commitment to allocating 15 percent of the total national budget to health, as agreed to in the Abuja Declaration, combined with the projected 5 percent annual increase in the total Rwandan budget due to GDP growth, would generate enough funds to cover the estimated $43 million per year for Rwandan health professionals produced by the HRH Program, $9 million per year for tuition support, $13 million (over several years) for equipment replacement, and $1.5 million per year for equipment maintenance (MOH, 2011b). The MOH’s 2014 funding application program narrative in response to the funding opportunity announcement articulated sustainability as the programmatic aim of building “sufficient Rwandan educators, infrastructure and equipment, and domestic financing to support Rwanda’s health care and health sciences education without external support” (MOH, 2014d). Although it appears that some planning for sustainability took place during the proposal phase, interview respondents expressed variation in how to define and measure HRH Program sustainability, which was linked to how they conceptualized the Program goal and objectives, as articulated by an administrator at the University of Rwanda: It depends on what you mean by sustainability. If you mean a program that goes on forevermore, having a hundred people coming every year … if that’s your version of sustainability we need to sit down and talk about it. If it’s about building on the gains, that’s another issue and that’s something that the university takes very seriously.… In terms of sustainability, do we need the same thing forevermore for the next 20 years? Then, no. We would have failed. Do we want input of experts for the expertise that we don’t have? Absolutely, you know! Sustainability is about us, being able to do by ourselves, better than it is done anywhere in the world, and that’s what we are striving to do at the University of Rwanda all the time. (02, University of Rwanda Administrator) Among respondents who believed the intent was to build capacity within the University of Rwanda to continually produce high-quality health workers, sustainability was seen as built into the design: [T]he goal was to mentor teachers and train faculty. So, in the design, the program is already sustainable. (43, Government of Rwanda HRH Program Administrator) One respondent explained that the design included a gradual phasing out of USI faculty and a simultaneous increase in Rwandan faculty: PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-39 The program was implemented with the target that after 7 years the program will end.… [T]he sustainability plan was kind of [a] mechanism of decreasing gradually the U.S. faculty, while increasing retained and available local faculty. That is one thing. On the other side, the [University of Rwanda] had in their planning to have to increase the college faculty so the local Rwandan faculty in the university funded for 2 years, I think that was the initial plan for the 2 last years of the program, with the program funding while negotiating to be integrated in the structure of the university. I think that is how the sustainability plan was thought. (20, Government of Rwanda HRH Program Administrator) In contrast, for another respondent, who had expressed that the objective of the HRH Program was to build capacity of Rwandan health education institutions, most directly the University of Rwanda, the Program failed to clearly understand and build on the capacity of local institutions: [L]ocal institutions were … involved, but there was nothing really done to understand the local institutions’ capacity, strength, readiness, levels, interest, priorities, process. And nothing was really done to understand how we could integrate successfully this faculty and effectively in the local institutions, you know, how they were going to support the leadership, how they were going to complement and not replace local faculty in the way that built on synergies and not mix with redundancies which happened a lot, I think. (22, Non-Government of Rwanda HRH Program Administrator) Respondents external to the government were skeptical of the sustainability and long- term planning that was done in the design phase: I think the attitude of a lot of Rwandan institutions, departments, was that “Ok, we will take the support and enjoy it while it’s there. It is really helpful; it is filling a gap.” And then one day it ends and there’s something else coming in. And that’s fine, but not really long-term planning. (22, Non-Government of Rwanda HRH Program Administrator) CONCLUSIONS The HRH Program represented a confluence of unusual circumstances and an opportunity in foreign assistance. Its design endorsed Rwanda’s larger vision of strengthening the country’s workforce, including in the health sector, although there were missed opportunities to learn systematically. PEPFAR 2.0 marked “a new era of collaborative planning and health systems strengthening activities with … partner governments” (PEPFAR, 2014). PEPFAR’s investments in the HRH Program were a critical contribution to launching an Academic Consortium that leveraged (1) existing and new partnerships with individual USIs, (2) the political commitment of a government with record of successfully managing health systems programs, and (3) a holistic PEPFAR strategy that emphasized HSS. PEPFAR’s investments in the HRH Program in Rwanda were relatively unique, departing from U.S government and many other big donors’ models for “business as usual,” but aligned PREPUBLICATION COPY: UNCORRECTED PROOFS

3-40 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM with prevailing principles established by the Busan Partnership for Effective Development Co- operation, in which funding partners would support shared goals according to specific country situations (Fourth High-Level Forum on Aid Effectiveness, 2011). PEPFAR investments have continuously supported programs and activities implemented in partner countries and, consistent with PEPFAR’s strategy to move toward sustainability, have directed their funding over time to local prime partners, although the proportion of funding to partner country governments has been relatively stable (IOM, 2013). From the outset, Rwanda expressed a strong commitment to national ownership, as articulated in Vision 2020, and sought to increase the direct management of externally funded health and HIV programs. To move toward this vision, the Government of Rwanda and its PEPFAR-funded partners promoted a transition plan that shifted HIV program leadership from partners to the host country, including direct PEPFAR financing to the government (Government of Rwanda, 2012). The management transition took place by February 2012, when the MOH became a direct recipient of PEPFAR funds and gained responsibility for coordinating critical service delivery (Binagwaho et al., 2016). These investments challenged the conventional model, and used country ownership and shared responsibility as a new paradigm when the HRH Program commenced in 2012 (Goosby et al., 2012a,b). The investments in the HRH Program also were an opportunity to add to the evidence around twinning because the Program’s Academic Consortium was designed not only as a departure from short-term faculty stints in Rwanda, but also to completely phase out by the end of the Program, as the “twins” and new graduates gained skills to serve as local faculty. The twinning arrangements were between individual USI faculty and University of Rwanda faculty, not between the two institutions. The implications of this arrangement, discussed more fully in subsequent chapters, were felt in terms of sustained capacity at the institutional level to provide ongoing health professional education. There was concurrence among HRH Program participants on a high-level vision and intent, which aligned with broader health-sector goals. However, there was a lack of clarity around the mechanisms and pathway for achieving a world-class health care system, the consequences of which were felt during the implementation of the Program, as discussed throughout subsequent chapters. The relative importance of building health education capacity, including enhancing the skills of Rwandan faculty and the learning environment at the University of Rwanda, was inconsistent during the life of the Program. In particular, it was unclear whether activities for building institutional capacity for health professional education were prioritized simply as a mechanism to produce the desired number of health professionals across cadres, or viewed as complementary goals to having a sustainable, skilled, and specialized workforce, as expressed in the later strategic outcomes (MOH, 2011b, 2014b, 2015b). The downstream effects of the shifting language in MOH documents was a lack of clarity across stakeholders around the Program’s goals and objectives, which had implications for the design and for sustainability planning and were further compounded by differing (and sometimes conflicting or restrictive) administrative practices across participating institutions (Government of Rwanda, USIs, and PEPFAR). Planning for sustainability evolved from accounting for funding of the established training activities under the HRH Program after 2019 to more holistically ensuring the quality of health professional education and service delivery. Although the Government of Rwanda recognized the unprecedented scale and nature of the HRH Program’s strategies, there was insufficient planning to exploit the opportunity to learn systematically from this endeavor by establishing rigorous M&E processes and supportive PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-41 mechanisms at the outset (MOH, 2011b, 2014b, 2016b). The original program objectives were incorporated into various indicators for the M&E plan and midterm review; however, the development of an M&E plan after implementation had begun resulted in a lack of baseline data for some cadres and program areas, as well as unexplained differences in baseline and target values (MOH, 2011b, 2014b, 2016b). Even as the post hoc M&E plan articulated an intention to “provide a reliable data-driven approach … to ensure accountability, program improvement, impact, and learning,” there was lack of follow through by the Government of Rwanda in providing dedicated resources to establish an M&E platform (for example, M&E working groups were composed of volunteer HRH Program participants, as opposed to designated roles), to carry out planned assessments, and to build overall capacity for monitoring, evaluation, and learning (MOH, 2014b). Overall management of the HRH Program was challenged by the lack of time for operational management, both at the outset of implementation and continuously, as unexpected circumstances arose. More time between the design, launch, and execution phases would have supported stakeholders’ ability to better anticipate and develop contingent strategies for issues such as PEPFAR funding processes and restrictions. REFERENCES Binagwaho, A., P. Kyamanywa, P. E. Farmer, T. Nuthulaganti, B. Umubyeyi, J. P. Nyemazi, S. D. Mugeni, A. Asiimwe, U. Ndagijimana, H. Lamphere McPherson, D. Ngirabega Jde, A. Sliney, A. Uwayezu, V. Rusanganwa, C. M. Wagner, C. T. Nutt, M. Eldon-Edington, C. Cancedda, I. C. Magaziner, and E. Goosby. 2013. The Human Resources for Health Program in Rwanda--new partnership. New England Journal of Medicine 369(21):2054-2059. Binagwaho, A., P. E. Farmer, S. Nsanzimana, C. Karema, M. Gasana, J. De Dieu Ngirabega, F. Ngabo, C. M. Wagner, C. T. Nutt, T. Nyatanyi, M. Gatera, Y. Kayiteshonga, C. Mugeni, P. Mugwaneza, J. Shema, P. Uwaliraye, E. Gaju, M. A. Muhimpundu, T. Dushime, F. Senyana, J. B. Mazarati, C. M. Gaju, L. Tuyisenge, V. Mutabazi, P. Kyamanywa, V. Rusanganwa, J. P. Nyemazi, A. Umutoni, I. Kankindi, C. Ntizimira, H. Ruton, N. Mugume, D. Nkunda, E. Ndenga, J. M. Mubiligi, J. B. Kakoma, E. Karita, C. Sekabaraga, E. Rusingiza, M. L. Rich, J. S. Mukherjee, J. Rhatigan, C. Cancedda, D. Bertrand-Farmer, G. Bukhman, S. N. Stulac, N. M. Tapela, C. Van Der Hoof Holstein, L. N. Shulman, A. Habinshuti, M. H. Bonds, M. S. Wilkes, C. Lu, M. C. Smith-Fawzi, J. D. Swain, M. P. Murphy, A. Ricks, V. B. Kerry, B. P. Bush, R. W. Siegler, C. S. Stern, A. Sliney, T. Nuthulaganti, I. Karangwa, E. Pegurri, O. Dahl, and P. C. Drobac. 2014. Rwanda 20 years on: Investing in life. Lancet 384(9940):371-375. Binagwaho, A., I. Kankindi, E. Kayirangwa, J. P. Nyemazi, S. Nsanzimana, F. Morales, R. Kadende- Kaiser, K. W. Scott, V. Mugisha, R. Sahabo, C. Baribwira, L. Isanhart, A. Asiimwe, W. M. El- Sadr, and P. L. Raghunathan. 2016. Transitioning to country ownership of HIV programs in Rwanda. PLoS Medicine 13(8):e1002075. Cancedda, C., P. Cotton, J. Shema, S. Rulisa, R. Riviello, L. V. Adams, P. E. Farmer, J. N. Kagwiza, P. Kyamanywa, D. Mukamana, C. Mumena, D. K. Tumusiime, L. Mukashyaka, E. Ndenga, T. Twagirumugabe, K. B. Mukara, V. Dusabejambo, T. D. Walker, E. Nkusi, L. Bazzett-Matabele, A. Butera, B. Rugwizangoga, J. C. Kabayiza, S. Kanyandekwe, L. Kalisa, F. Ntirenganya, J. Dixson, T. Rogo, N. McCall, M. Corden, R. Wong, M. Mukeshimana, A. Gatarayiha, E. K. Ntagungira, A. Yaman, J. Musabeyezu, A. Sliney, T. Nuthulaganti, M. Kernan, P. Okwi, J. Rhatigan, J. Barrow, K. Wilson, A. C. Levine, R. Reece, M. Koster, R. T. Moresky, J. E. O'Flaherty, P. E. Palumbo, R. Ginwalla, C. A. Binanay, N. Thielman, M. Relf, R. Wright, M. Hill, D. Chyun, R. T. Klar, L. L. McCreary, T. L. Hughes, M. Moen, V. Meeks, B. Barrows, M. E. Durieux, C. D. McClain, A. Bunts, F. J. Calland, B. Hedt-Gauthier, D. Milner, G. Raviola, S. PREPUBLICATION COPY: UNCORRECTED PROOFS

3-42 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM E. Smith, M. Tuteja, U. Magriples, A. Rastegar, L. Arnold, I. Magaziner, and A. Binagwaho. 2018. Health professional training and capacity strengthening through international academic partnerships: The first five years of the Human Resources for Health Program in Rwanda. International Journal of Health Policy and Management 7(11):1024-1039. CDC (U.S. Centers for Disease Control and Prevention). 2009a. Funding opportunity announcement clinical services 5-year proposed work plan. Atlanta, GA: U.S. Centers for Disease Control and Prevention. CDC. 2009b. Funding opportunity announcement: Strengthening the Ministry of Health's Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under PEPFAR. Atlanta, GA: U.S. Centers for Disease Control and Prevention. CDC. 2009c. U.S. Department of Health and Human Services Centers for Disease Control and Prevention Strengthening the Ministry of Health's Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR) funding opportunity. Atlanta, GA: U.S. Centers for Disease Control and Prevention. CDC. 2012. Request approval to award a multi-year expansion supplement exceeding 25% for the cooperative agreement PS09-9118, 3U2GPS2091 “Strengthening the Ministry of Health’s Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR)” and approval to waive the requirement of an objective review for the single awardee to this cooperative agreement. Atlanta, GA: U.S. Centers for Disease Control and Prevention. CDC. 2014. Funding opportunity announcement: Strengthening Human Resources for Health Capacity in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR) CDC- RFA-GH15-1568. Atlanta, GA: U.S. Centers for Disease Control and Prevention. Flinkenflögel, M., P. Kyamanywa, B. Asiimwe-Kateera, S. Musafiri, P. Kayumba, M. Irakoze, P. Cotton, and A. Hibble. 2015. Umuganda for improved health professions education in Rwanda: Past, present and future in the training of health professionals at the University of Rwanda. Rwanda Journal 2:96. Fourth High-Level Forum on Aid Effectiveness. 2011. Busan partnership for effective development co- operation: Fourth high-level forum on aid effectiveness, Busan, Republic of Korea, 29 November- 1 December 2011. Paris, France: Organisation for Economic Co-operation and Development. Gavi (Gavi, the Vaccine Alliance). 2018. Republic of Rwanda: Programme audit of Gavi support to the ministry of health. Geneva, Switzerland: Gavi, the Vaccine Alliance. Global Fund (Global Fund to Fight AIDS, Tuberculosis and Malaria). 2014. Audit report: Audit of global fund grants to Rwanda. Geneva, Switzerland: Global Fund. Global Fund. 2019. Audit follow up report: Global fund grants to Rwanda. Geneva, Switzerland: Global Fund. Goosby, E., M. Dybul, A. S. Fauci, J. Fu, T. Walsh, R. Needle, and P. Bouey. 2012a. The United States President's Emergency Plan For AIDS Relief: A story of partnerships and smart investments to turn the tide of the global AIDS pandemic. Journal of Acquired Immune Deficiency Syndrome 60(Suppl 3):S51-S56. Goosby, E., D. Von Zinkernagel, C. Holmes, D. Haroz, and T. Walsh. 2012b. Raising the bar: PEPFAR and new paradigms for global health. Journal of Acquired Immune Deficiency Syndrome 60(Suppl 3):S158-S162. Government of Rwanda. 2012. Vision 2020, revised 2012. Kigali, Rwanda: Government of Rwanda. IOM (Institute of Medicine). 2013. Evaluation of PEPFAR. Washington, DC: The National Academies Press. Kayirangwa, E., J. Hanson, L. Munyakazi, and A. Kabeja. 2006. Current trends in Rwanda's HIV/AIDS epidemic. Sexually Transmitted Infection 82(Suppl 1):i27-i31. MOH (Ministry of Health). 2005a. Health sector policy, February 2005. Kigali, Rwanda: Ministry of Health. MOH. 2005b. Health sector strategic plan, 2005-2009. Kigali, Rwanda: Ministry of Health. PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-43 MOH. 2009a. Health sector strategic plan, July 2009-June 2012. Kigali, Rwanda: Ministry of Health. MOH. 2009b. National strategic plan on HIV and AIDS 2009-2012. Kigali, Rwanda: Ministry of Health. MOH. 2010. Clinical services work plan. Kigali, Rwanda: Ministry of Health. MOH. 2011a. Human resources for health strategic plan, 2011-2016. Kigali, Rwanda: Ministry of Health. MOH. 2011b. Rwanda Human Resources for Health Program, 2011-2019: Funding proposal. Kigali, Rwanda: Ministry of Health. MOH. 2011c. U.S. Academic institutions—selection rationale. Kigali, Rwanda: Ministry of Health. MOH. 2012a. HRH application brief cabinet paper, May 2012-30 Sept 2013. Kigali, Rwanda: Ministry of Health. MOH. 2012b. HRH U.S. Academic Consortium budget years 1 & 2 Kigali, Rwanda: Ministry of Health. MOH. 2012c. Rwanda health resource tracker output report for FY 2009-10 exp. & FY 2010-11 budget & FY 2010-11 exp. & FY 2011-12 budget. Kigali, Rwanda: Ministry of Health. MOH. 2012d. Third health sector strategic plan, July 2012-June 2018. Kigali, Rwanda: Ministry of Health. MOH. 2012e. Work plan. Kigali, Rwanda: Ministry of Health. MOH. 2013a. CDC-RFA-PS09-9118-05CONT13: Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President's Emergency Plan for AIDS Relief (PEPFAR). Kigali, Rwanda: Ministry of Health. MOH. 2013b. Rwanda health resource tracker output report December 2013: Expenditures FY 2011/12 - budget FY 2012/13. Kigali, Rwanda: Ministry of Health. MOH. 2014a. Annual report on the implementation of CDC-funded projects, period: October 2012- September 2013. Kigali, Rwanda: Ministry of Health. MOH. 2014b. Human Resources for Health monitoring & evaluation plan, March 2014. Kigali, Rwanda: Ministry of Health. MOH. 2014c. National human resources for health policy, October 2014. Kigali, Rwanda: Ministry of Health. MOH. 2014d. Project narrative. Kigali, Rwanda: Ministry of Health. MOH. 2015a. Health sector policy, January 2015. Kigali, Rwanda: Ministry of Health. MOH. 2015b. Programmatic/technical work plan. Kigali, Rwanda: Ministry of Health. MOH. 2016a. Rwanda health resource tracking output report: Expenditure FY 2013/14 and budget FY 2014/15. Kigali, Rwanda: Ministry of Health. MOH. 2016b. Rwanda Human Resources for Health Program midterm review report (2012-2016). Kigali, Rwanda: Ministry of Health. MOH. 2018a. Fourth health sector strategic plan, July 2018-June 2024. Kigali, Rwanda: Ministry of Health. MOH. 2018b. Rwanda health resource tracking output report: Expenditure FY 2014/15 and budget FY 2015/16. Kigali, Rwanda: Ministry of Health. MOH. 2018c. Rwanda HIV and AIDS national strategic plan 2013-2018: Extension: 2018-2020. Kigali, Rwanda: Ministry of Health. Mtiro, E. 2018. Termination of Rwanda cooperative agreement. Kigali, Rwanda. Nsanzimana, S., K. Prabhu, H. McDermott, E. Karita, J. I. Forrest, P. Drobac, P. Farmer, E. J. Mills, and A. Binagwaho. 2015. Improving health outcomes through concurrent HIV program scale-up and health system development in Rwanda: 20 years of experience. BMC Medicine 13(1):216. PEPFAR (President’s Emergency Plan for AIDS Relief). 2009. Guidance for PEPFAR partnership frameworks and partnership framework implementation plans. Version 2.0, September 14, 2009. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEFPAR. 2010. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2011. FY 2012 country operational plan (COP) guidance. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PREPUBLICATION COPY: UNCORRECTED PROOFS

3-44 EVALUATION OF PEPFAR’S CONTRIBUTION TO RWANDA’S HRH PROGRAM PEPFAR. 2012. FY 2013 country operational plan (COP) guidance. Version 2. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2014. PEPFAR 3.0: Controlling the epidemic; delivering on the promise of an AIDS-free generation. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2019a. Executive summary of PEPFAR's strategy. https://www.pepfar.gov/about/strategy/document/133244.htm (accessed February 24, 2019). PEPFAR. 2019b. PEPFAR’s five-year strategy. https://www.pepfar.gov/about/strategy/document/133251.htm (accessed February 21, 2019). PEPFAR. 2019c. PEPFAR Panorama Spotlight. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2019d. PEPFAR Panorama Spotlight glossary. https://data.pepfar.gov/glossary (accessed October 11, 2019). PEPFAR Rwanda. 2008. Populated printable COP without TBD partners 2008 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2009. Populated printable COP 2009 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2010. Rwanda operational plan report FY 2010. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2011. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2012. Rwanda country operational plan FY 2012. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2013. Rwanda operational plan report FY 2013. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2014. Rwanda operational plan report FY 2014. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2015. Rwanda country operational plan COP 2015 strategic direction summary. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. Seymour, B., I. Muhumuza, C. Mumena, M. Isyagi, J. Barrow, and V. Meeks. 2013. Including oral health training in a health system strengthening program in Rwanda. Global Health Action 6:1-6. TAGGS (Tracking Accountability in Government Grants System). 2019. Ministry of Health Rwanda (accessed July 9, 2019). Tvedten, I., A. Byabagambi, J. Lindstrom, and M. Tedre. 2018. Evaluation of the SIDA-supported research capacity and higher education development program in Rwanda, 2013–2017. Stockholm, Sweden: Swedish International Development Cooperation Agency. USAID. (United States Agency for International Development). 1996. Rebuilding postwar Rwanda: The role of the international community. Washington, DC: United States Agency for International Development. Uwizeye, G., D. Mukamana, M. Relf, W. Rosa, M. J. Kim, P. Uwimana, H. Ewing, P. Munyiginya, R. Pyburn, N. Lubimbi, A. Collins, I. Soule, K. Burke, J. Niyokindi, and P. Moreland. 2018. Building nursing and midwifery capacity through Rwanda's Human Resources for Health Program. Journal of Transcultural Nursing 29(2):192-201. WHO (World Health Organization). 2016. Health workforce requirements for universal health coverage and the Sustainable Development Goals. Geneva, Switzerland: World Health Organization. WHO. 2019. Global health expenditure database. Geneva, Switzerland: World Health Organization. WHO and GHWA (Global Health Workforce Alliance). 2008. Scaling up, saving lives: Task Force for Scaling Up Education and Training for Health Workers, Global Health Workforce Alliance. Geneva, Switzerland: World Health Organization. Willis, B. M., and B. S. Levy. 2000. Recognizing the public health impact of genocide. JAMA 284(5):612-614. PREPUBLICATION COPY: UNCORRECTED PROOFS

HRH PROGRAM CONTEXT, VISION, AND DESIGN 3-45 World Bank. 2019a. Current health expenditure per capita, PPP (current international $)—Rwanda. https://data.worldbank.org/indicator/SH.XPD.CHEX.PP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019b. Domestic general government health expenditure per capita, PPP (current international $)—Rwanda. https://data.worldbank.org/indicator/SH.XPD.GHED.PP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019c. GDP (current US$)—Rwanda. https://data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019d. GDP per capita, PPP (current international $)—Rwanda. https://data.worldbank.org/indicator/NY.GDP.PCAP.PP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019e. General government final consumption expenditure (current US$). https://data.worldbank.org/indicator/NE.CON.GOVT.CD (accessed November 4, 2019). World Bank. 2019f. Inflation, consumer prices (annual %)—Rwanda. https://data.worldbank.org/indicator/FP.CPI.TOTL.ZG?end=2018&locations=RW&start=2000 (accessed November 4, 2019). World Bank. 2019g. Official exchange rate (LCU per US$, period average). https://data.worldbank.org/indicator/PA.NUS.FCRF?locations=RW (accessed November 4, 2019). World Bank. 2019h. PPP conversion factor, GDP (LCU per international $)—Rwanda. https://data.worldbank.org/indicator/PA.NUS.PPP?locations=RW (accessed November 4, 2019). PREPUBLICATION COPY: UNCORRECTED PROOFS

Next: 4 Faculty Twinning »
Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program Get This Book
×
Buy Paperback | $65.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Since 2004, the U.S. government has supported the global response to HIV/AIDS through the President’s Emergency Plan for AIDS Relief (PEPFAR). The Republic of Rwanda, a PEPFAR partner country since the initiative began, has made gains in its HIV response, including increased access to and coverage of antiretroviral therapy and decreased HIV prevalence. However, a persistent shortage in human resources for health (HRH) affects the health of people living with HIV and the entire Rwandan population.

Recognizing HRH capabilities as a foundational challenge for the health system and the response to HIV, the Government of Rwanda worked with PEPFAR and other partners to develop a program to strengthen institutional capacity in health professional education and thereby increase the production of high-quality health workers. The Program was fully managed by the Government of Rwanda and was designed to run from 2011 through 2019. PEPFAR initiated funding in 2012. In 2015, PEPFAR adopted a new strategy focused on high-burden geographic areas and key populations, resulting in a reconfiguration of its HIV portfolio in Rwanda and a decision to cease funding the Program, which was determined no longer core to its programming strategy. The last disbursement for the Program from PEPFAR was in 2017.

Evaluation of PEPFAR's Contribution (2012-2017) to Rwanda's Human Resources for Health Program evaluates the HRH Program to understand how PEPFAR's investment affected morbidity and mortality outcomes for people living with HIV. This study describes PEPFAR investments in HRH in Rwanda over time, including its support for Ministry of Health efforts to address HRH needs and the broader context in which these investments were made; describes PEPFAR-supported HRH activities in Rwanda in relation to programmatic priorities, outputs, and outcomes; examines the impact of PEPFAR funding for the HRH Program on HRH outcomes and on patient- or population-level HIV-related outcomes; and provides recommendations to inform future HRH investments that would support people living with HIV and advance PEPFAR's mission.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!