Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
3 Human Resources for Health Program Context, Vision, and Design Key Findings: + Successes and - Challenges 3 Vision and Design Ch â¢ Concurrence among â¢ Lack of clarity around the mechanisms and pathway interview participants for the vision and intent of achieving a world-class on a high-level vision health care system and intent that aligned â¢ Tension between the perceived needs for and with broader health prioritization of specialized versus primary care sector goals providers â¢ Program management â¢ Insufficient planning and funding to systematically led by the Government learn from the Program by establishing rigorous MEL of Rwanda, in line with processes and supportive mechanisms at the outset emerging global principles for donor â¢ Insufficient time for operational management, both at assistance the outset of implementation and continuously, as unexpected circumstances arose + - CONTEXT IN RWANDA LEADING UP TO THE PROGRAM The Costs of Conflict Rwandaâs recent history of conflict is essential context for the Human Resources for Health (HRH) Programâs origin and implementation prog- ress. The 1994 genocide against the Tutsi in Rwanda resulted in an esti- mated 1 million deaths and 2 million displaced people. All social services 55
56 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA were devastated, bringing to a halt all foreign assistance programs related to health, education, agriculture, and other economic development efforts. The toll on the health system included physical destruction of hospitals, laboratories, and equipment, not to mention the death or displacement of more than 80 percent of the countryâs health professionals (USAID, 1996). By 1995, fewer than 10 pediatricians were reported to have been practic- ing, and there were no trained medical personnel, such as psychiatrists or trauma surgeons, to address issues emerging from the conflict (Binagwaho et al., 2014; Nsanzimana et al., 2015). The time and other investments required to counter the long-term public health effects of damaged social institutions were evident in the fact that it took 6 years for the first class of medical students to graduate (Willis and Levy, 2000). During and immediately following the genocide against the Tutsi, inter- national organizations prioritized restoring primary health care services and water and sanitation systems (USAID, 1996). The International Committee of the Red Cross, United Nations Childrenâs Fund (UNICEF), and nongov- ernmental organizations (NGOs) such as MÃ©decins Sans FrontiÃ¨res focused on repairing and operationalizing clinics with emergency health kits, medi- cines, supplies, and staff to provide on-the-job training to auxiliary health workers (USAID, 1996). Immunization service delivery was disrupted, but the Ministry of Health (MOH) received international support to restore vaccine stocks and function to its immunization supply chain and logistics system (USAID, 1996). However, other capacities to detect and control infectious diseases, particularly the spread of HIV and cholera from migra- tion and refugee camps at Rwandaâs border, would remain a challenge for years (Binagwaho et al., 2014; Nsanzimana et al., 2015). Although donors and NGOs were essential for providing necessary emergency relief, these agencies did not fully coordinate with the Rwandan government in the transition from emergency operations to rehabilitation of the health system. Consequently, health care services were fragmented, and the MOH was slower to build institutional capacity to manage the countryâs health priorities and sustain a health care delivery system (USAID, 1996). Major Developments in Rwandaâs Health-Sector Planning During the recovery period following the genocide against the Tutsi, the Government of Rwanda has consistently planned for how the health sector could better the health status of the population and support national poverty reduction goals. The release of the Health Sector Policy in 2005 outlined an overhaul of the sector, in light of decentralization efforts under way and in pursuit of more significant health gains (MOH, 2005a). Seven intervention priorities were established in seven areas:
HRH PROGRAM CONTEXT, VISION, AND DESIGN 57 1. Availability of human resources; 2. Availability of high-quality drugs, vaccines, and consumables; 3. Geographical access to health services; 4. Financial access to health services; 5. Quality of and demand for services in disease control; 6. Strengthening national referral hospitals and treatment centers; and 7. Strengthening the health sectorâs institutional capacity. In particular, the Government of Rwanda would support financial access to health services by increasing public funding of health services (MOH, 2005a). The government strategy around this policy has evolved over time. The first Health Sector Strategic Plan (2005â2009) laid out plans for achiev- ing the health-related Millennium Development Goals (MDGs), with at- tention to getting maternal and child mortality on track (MOH, 2005b). The following iteration of the Health Sector Strategic Plan (2009â2012) highlighted health systems strengthening in each of the seven intervention areas and continued to emphasize increased financial access to health ser- vices, noting the scale-up of community-based health insurance, as well as increased quality of care promoted by performance-based financing (MOH, 2009a). The World Health Organization (WHO) framework of health systems building blocks served as a foundation for the third Health Sector Strategic Plan (2012â2018), in which the MOH started to orient toward objectives in the post-MDG era and increasingly considered health resource management and governance mechanisms (MOH, 2012d). The resulting Health Sector Policy of 2015 touted key health achievements, including improved maternal and child health, increased community health worker coverage, and antiretroviral therapy (ART) and malaria program successes since 2004 (MOH, 2015a). However, it also outlined policies to address challenges in support of vulnerable and marginalized populations, sustain- able health system financing, and noncommunicable disease prevention and control, taking into account Rwandaâs epidemiologic transition and socioeconomic progress (MOH, 2015a). The current Health Sector Strate- gic Plan (2018â2024) centers on the Sustainable Development Goals and fully acknowledges these challenges and the need to reorganize the health system and involve other sectors of development to ensure universal health coverage for all needed services at all stages of life (MOH, 2018a). HIV in Rwanda The first case of HIV in Rwanda was documented in 1983; the Govern- ment of Rwanda initiated its response in 1985 with a blood donor screen- ing program and has since sustained efforts to address the HIV epidemic
58 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA (Kayirangwa et al., 2006). In 2009, Rwanda published its first National Strategic Plan on HIV/AIDS (MOH, 2009b). The development of national strategic plans on HIV/AIDS, the decentralization of the Rwandan health system, and the movement toward community-based health insurance and performance-based financing facilitated its key achievements and remark- able progress toward achieving HIV epidemic control (MOH, 2009a,b, 2018c). Rwanda has also made steady improvements in increasing access to and coverage of ART over the past decade. The governmentâs commitment to confronting its HIV epidemic has accelerated progress toward the Joint United Nations Programme on HIV/AIDSâ (UNAIDSâ) 90-90-90 targets prior to the HRH Program, as discussed in more detail in Chapter 7. HRH in Rwanda Despite improvements since the genocide against the Tutsi nearly de- stroyed the health infrastructure and resulted in acute health workforce shortages, which hindered health service delivery and served as a major barrier to HIV care and treatment, Rwanda continues to fall far below WHOâs recommended critical minimum threshold of 4.45 doctors, nurses, and midwives per 1,000 people (WHO, 2016). This shortfall comes from an insufficient number of trained health professionals relative to the need. Addressing HRH capacity had been featured in prior health-sector plan- ning, but specific direction to develop a âcompetent, dedicated, productive, and accessible workforceâ in support of the MOHâs mission of âproviding quality preventive, curative, rehabilitative, and promotional servicesâ was not articulated until the National Human Resources for Health Policy in 2014 (MOH, 2014c). These guidelines were developed for the planning, management, use, and monitoring of health-sector resources to operationalize the National HRH Strategic Plan (2011â2016) (MOH, 2011a), building on Rwandaâs Health Systems Strengthening Framework and Consolidated Plan 2009â 2012 (MOH, 2009a). Notably, the National HRH Strategic Plan called for the development of a clear health service delivery plan, delineating specific competencies for each cadre providing services at each level of care, more emphasis on the quality of trained professionals and their distribution, and demand for reliable data to inform health resource management and evalu- ation of health system effectiveness (MOH, 2011a). Economic and Financial Context for Health in Rwanda As Rwanda recovered and rebuilt from the genocide against the Tutsi, its economy eventually also started to experience considerable growth. Fig- ure 3-1 shows the trajectory of growth from the early 2000s, as illustrated
HRH PROGRAM CONTEXT, VISION, AND DESIGN 59 Billions $12 $11.10 Gross national expenditure (current US$) $10.27 $10 GDP (gross domestic product) (current US$) $10.02 $8.73 $9.51 General government final $8 consumption expenditure (current $8.48 $8.28 US$) $7.33 $6 $4.19 $4 $3.81 $2.06 $2 $1.42 $1.21 $1.28 $1.81 $1.02 $0.59 $0.32 $0 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 FIGURE 3-1 The size of the Rwandan economy from 2000â2018 in current U.S. dollars. NOTES: Gross national expenditure is the sum of household final consumption expenditure, general government final consumption expenditure, and gross capital formation. General government final consumption expenditure includes all government current expenditures for purchases of goods and services (including compensation of employees). On average, infla- tion was about 6.5 percent in the 2000 to 2018 period (World Bank, 2019f), and the average growth of the official exchange rate in the same period was approximately 4.6 percent. SOURCES: World Bank World Development Indicators; World Bank, 2019c,e,g. by gross domestic product (GDP), the total national expenditure, and the portion of that national expenditure that is government spending. The HRH Program was situated in this period of economic growth. Concurrently, there was growth in spending on health in Rwanda. Figure 3-2 shows the trajectories of growth in both per capita total health expenditures and per capita government spending on health, alongside the growth in per capita GDP. In this same time period, as Figure 3-3 shows, close to half of the health expenditure in Rwanda has consistently come from external aid, ranging from 41 percent to 54 percent between 2000 and 2016. Spending specifically for Rwandaâs response to HIV increased drastically during this period, and as of 2014 it has been consistently estimated at about $200 million annually. Of this, 80 percent comes from external funders, a larger proportion than for health overall (Nsanzimana et al., 2015).
60 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA 2500 GDP per capita, PPP (purchasing power parity) (current international $) $2,254 Current health expenditure per capita, PPP (current international $) $1,978 2000 $1,895 Domestic general government health expenditure per capita, PPP (current international $) $1,589 1500 $1,124 1000 $639 500 $105 $130 $119 $130 $28 $33 $40 $20 $44 0 $5 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 FIGURE 3-2 Health expenditures in Rwanda from 2000â2016 (in purchasing power parity per capita). NOTES: Current health expenditure is the estimate of all health care goods and services con- sumed each year. Domestic general government health expenditure is public expenditure on health from domestic sources. On average, inflation was about 6.5 percent in the 2000â2018 period (World Bank, 2019f). The purchasing power parity conversion factor, for the 2000 to 2018 period, presented an average growth of about 4.4 percent (World Bank, 2019h). SOURCES: World Bank World Development Indicators; World Bank, 2019a,b,d. 6% 5% 4% 4% 5% 6% 6% 6% 5% 8% 9% 9% 8% 7% 7% 10% 12% Other 41% 49% 53% 52% 49% 45% 52% 53% 51% 50% 50% 54% 55% 49% 51% 54% 50% Aid 49% 46% 45% 44% 46% 44% 43% 43% 42% 42% 41% 43% 42% Domestic 40% 39% 38% 38% (public, out-of- pocket, voluntary health insurance) 2000 2002 2004 2006 2008 2010 2012 2014 2016 FIGURE 3-3 Proportion of health expenditure in Rwanda by revenue source. NOTE: Domestic âall sourcesâ comprises out-of-pocket voluntary health insurance and do- mestic public expenditures. SOURCES: WHO Global Health Expenditure Database; WHO, 2019.
HRH PROGRAM CONTEXT, VISION, AND DESIGN 61 PEPFAR Funding Context Before the HRH Program was launched in 2012, the Presidentâs Emer- gency Plan for AIDS Relief (PEPFAR) had an established funding history in Rwanda going back to 2003 (Binagwaho et al., 2016). Figure 3-4 shows PEPFAR total annual planned funding for programs in Rwanda from fiscal year (FY) 2009 to FY 2019, including the years surrounding the HRH Pro- gram.1 Amounts shown in the bar graph were PEPFAR allocations to treat- $119,041,924 $115,641,860 $115,641,860 Applied Pipeline $101,731,286 Care $92,366,603 $83,762,134 $77,874,802 Govt. & Systems* $68,389,437 $65,944,966 $65,024,359 $60,593,091 Prevention** Testing Treatment 5.8% 5.8% 7.5% 8.5% 26.8% 17.7% 16.4% 17.3% 3.0% 1.2% 1.9% HSS (Health $6,735,154 $6,735,154 $8,910,246 $8,670,017 $24,736,600 $14,820,370 $12,785,758 $11,800,000 $1,809,384 $777,719 $1,235,618 Systems Strengthening) 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 FIGURE 3-4 PEPFAR Rwanda planned funding by program area and percent allocation for HSS. NOTES: The bar graph shows amounts allocated to each program area indicated and the percentage of funding dedicated to health systems strengthening (HSS). These activities are classified under the PEPFAR OHSS (Health Systems Strengthening) budget code and defined as âcontribut[ing] to improvements in national-, regional- or district-level health systems;â and notes that these â[a]ctivities may be focused on health systems building blocks themselves as well as on institutions and processes that strengthen the building blocks and their interactions.â The Treatment category includes funding for programs classified under Adult Treatment, An- tiretroviral Drugs, and Pediatric Treatment. Testing includes funding for programs under HIV Testing and Counseling. Prevention includes funding for programs classified under Blood Safety, Injection Safety, Injecting and Non-Injecting Drug Use, Prevention of Mother-to-Child Transmis- sion, Sexual Prevention: Abstinence/Be Faithful, Sexual Prevention: Other Sexual Prevention, and Voluntary Medical Male Circumcision. Government & Systems includes funding for programs classified under Laboratory and Strategic Information. Care includes funding for programs classified under Adult Care and Support, Pediatric Care and Support, Tuberculosis/HIV, and Orphans and Vulnerable Children. Applied Pipeline includes appropriated but unspent funds car- ried over from prior years. The U.S. government FY runs from October 1 through September 30. SOURCES: PEPFAR, 2019c,d. 1â This graph was generated from publicly available PEPFAR planned funding data, down- loaded June 16, 2019, from PEPFARâs Panorama Spotlight website. Planned funding informa- tion was used because publicly available expenditure information was less granular and was not available from the start of the HRH Program.
62 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA ment, testing, prevention, health systems strengthening (HSS), government and systems, care, and applied pipeline. Percentages indicate proportion of annual funding allocated for HSS over time (PEPFAR, 2019c). PEPFARâs total planned funding for its Rwanda program portfolio de- clined overall from 2009 to 2019. The proportion of funding allocated to HSS increased during the period between FY 2013 and FY 2016 relative to surrounding years, with a peak in FY 2013. This period is concurrent with the timing of financial support for the HRH Program. Before FY 2013, PEPFAR planned funding in Rwanda for HSS had not exceeded $9 million and had remained under 10 percent of total planned funding. Between FY 2013 and FY 2016, PEPFAR funding for HSS activities ranged from $24.7 million in FY 2013 (27 percent of its total portfolio) to $11.8 million (17 percent of its portfolio) in FY 2016. Since FY 2016, funding for HSS activities in the portfolio has not exceeded $1.9 million, or 2 percent of total planned funding. In the same period between FY 2013 and FY 2016, there was a de- crease in the proportion of funding allocated to other program areas. In con- trast, the proportion of PEPFAR planned funding devoted to HSS activities between FY 2009 and FY 2019 in other East African Community countries has been less variable and has not exceeded 13 percent of the total portfolio.2 PEPFAR Investments in the HRH Program At the inception and during the first 3 years of its funding, PEPFAR considered the HRH Program to be a strategic or key priority within its overall portfolio (PEPFAR Rwanda, 2012, 2013, 2014). Initially, PEPFAR shifted resources from other activities to âtransfer significant support to the area of HRHâ (PEPFAR Rwanda, 2012) and thus fund the Program (PEPFAR Rwanda, 2012, 2013). Subsequently, the Programâs placement relative to other PEPFAR programmatic priorities decreased over time (PEPFAR Rwanda, 2013, 2014, 2015). The HRH Program was being funded during the window of peak PEPFAR funding for HSS in Rwanda, shown in Figure 3-4. Between FY 2011 and FY 2013, the Program was funded as a compo- nent of an existing U.S. Centers for Disease Control and Prevention (CDC) cooperative agreement with the MOH called âStrengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under PEPFARâ (also known as the Clinical Services 2â The other East African Community countries include Kenya, Tanzania, and Uganda. In Kenya, PEPFAR-planned funding for HSS activities has ranged from 1 percent to 3 percent of total planned funding between FY 2009 and FY 2019. In Uganda, funding for these activities has constituted between 2 percent and 5 percent of the PEPFAR portfolio, and in Tanzania, between 3 percent and 13 percent of the PEPFAR portfolio during this time. South Sudan and Burundi have smaller total PEPFAR portfolios, averaging less than $16 million per year.
HRH PROGRAM CONTEXT, VISION, AND DESIGN 63 Cooperative Agreement) (Mtiro, 2018). Preceding the HRH Program, CDC had already been providing direct support to the MOH for its HIV/AIDS- related programs (CDC, 2009c, 2012): â¢ The integration of HIV services and programs into the health sys- tem at all levels; â¢ Capacity building for infection control; â¢ Injection safety; â¢ Medical waste management; â¢ Epidemiological investigation, lab management, pharmaceutical management, health communication, electronic medical record use, and data collection and analysis; â¢ Use of terminology standards and registries; and â¢ Training for physicians, community health workers, and service providers in clinical handling and management of sexual and gender- based violence. This funding also provided salary support and benefits for physi- cians, technicians, and data managers, as well as supporting quality im- provement initiatives, performance-based financing evaluations, technical meetings with stakeholders, and management and audits of MOH/PEPFAR- supported facilities and sites (MOH, 2014a). The United States Agency for International Development (USAID) also provided some funds for the Program, although the amounts could not be confirmed (CDC, 2012; PEPFAR Rwanda, 2013). In addition to PEPFAR funding, the Program had a diversified funding base, evidenced by the en- gagement of the Global Fund to Fight AIDS, Tuberculosis and Malaria.3 Starting in FY 2014, CDC used a separate cooperative agreement with the MOH, âStrengthening Human Resources for Health Capacity in the Republic of Rwanda under PEPFAR,â to fund the HRH Program (Mtiro, 2018). In 2015, PEPFARâs priorities shifted, and the HRH Program was classified as a ânoncoreâ investment as part of an analysis in alignment with PEPFARâs new 3.0 strategy. The decision was made to end funding for the Program on March 30, 2017, instead of continuing PEPFAR support through June 30, 2019, the Programâs official closing date (Mtiro, 2018; PEPFAR Rwanda, 2015). Public reporting systems provided limited information about the ac- tual amount invested by PEPFAR in the HRH Program. Table 3-1 shows 3â The Global Fund specifically sponsored the launch of Rwandaâs first dental school and development of dental curricula and partnerships (Seymour et al., 2013), as PEPFAR did not include dentistry in the list of specialties it would support. Examination of the dentistry school and program was not included in this evaluation.
64 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA TABLE 3-1 PEPFAR Funding Sources That Contributed to the HRH Program FY 2012 FY 2013 (Oct 2011âSep (Oct 2012â 2012) Sep 2013) U.S. Centers for Disease Control and Prevention (CDC)* Strengthening the Capacity of the Ministry of Health to $17,556,432 $22,168,552 Respond to the HIV/AIDS Epidemic in the Republic of (total (total Rwanda Under PEPFAR cooperative cooperative agreement) agreement) (Award No. U2GPS002091; Mechanism No. 10825) Strengthening Human Resources for Health Capacity in N/A N/A the Republic of Rwanda Under PEPFAR (Award No. U2GGH001614; Mechanism No. 17621) United States Agency for International Development (USAID)*** Unknown Award(s)/ Unknown Unknown Mechanism(s) NOTES: N/A = not applicable (funding mechanism not used to contribute to the HRH Pro- gram that year). All amounts are in U.S. dollars. * Annual total disbursed funding by award/mechanism as reported through the U.S. HHS Tracking Accountability in Government Grants System (TAGGS) (accessed July 7, 2019). TAGGS reports the total amount disbursed through such agreements, but does not provide any further breakdown by program or activity. Therefore, the proportion of the amount that was specific to the HRH Program is not known in FY 2012 to FY 2014, when the HRH Program was funded as a component of the cooperative agreement, âStrengthening Capacity to Respond to the HIV/AIDS Epidemic,â that started in FY 2009. Starting in FY 2015, the HRH Program was funded through its own cooperative agreement, âStrengthening Human Resources for Health Capacity in the Republic of Rwanda Under PEPFAR,â and the amounts reported in TAGGS are reflective of the amounts disbursed for the HRH Program: $11 mil- lion in FY 2015 and $10.5 million in FY 2016. Any amounts disbursed through CDC after FY 2016 are also not known. ** Information is not publicly available on whether amounts were disbursed through this or any other CDC award or mechanism after FY 2016. *** Although there is documentation that some PEPFAR investments in the HRH Program came from other implementing agencies, those amounts and sources are not reported publicly. Approval of USAID maternal and child health funds to be used for implementing the HRH Program is referenced in a 2012 CDC memo requesting a multiyear expansion supplement for the âStrengthening the Ministry of Healthâs Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda Under PEPFARâ cooperative agreement. However, amounts were redacted, and data on the portion of the award/mechanism allocated to the HRH Program were not publicly available (correspondence provided by CDC Rwanda in August 2018). In addition, Rwandaâs Country Operational Plan FY 2013 referenced a planned shift of $2 million from USAIDâs Family Health Project to the HRH Program, but the actual amount disbursed and allocated for the HRH Program is not publicly available. SOURCES: CDC, 2012; PEPFAR Rwanda, 2014; TAGGS, 2019.
HRH PROGRAM CONTEXT, VISION, AND DESIGN 65 FY 2014 FY 2015 FY 2016 FY 2017 FY 2018 (Oct 2013âSep (Oct 2014â (Oct 2015âSep (Oct 2016âSep (Oct 2017âSep 2014) Sep 2015) 2016) 2017) 2018) $23,507,981 $0 $0 N/A N/A (total cooperative agreement) N/A $11,000,000 $10,500,000 Unknown** Unknown** Unknown Unknown Unknown Unknown Unknown available information regarding the mechanisms PEPFAR used to provide funding for the Program. HRH Program Budget and Ministry of Health Expenditures on Health The HRH Program budget proposed in 2011 was $151.8 million over 8 years (MOH, 2011b). The Program received external funding from the U.S. government, through PEPFAR, and from the Global Fund. The total funding amount from these external sources, as provided by the MOH, was just under $100 million, with approximately 60 percent coming from the U.S. government (see Table 3-2). This comprised 12 percent to 16 percent of the total annual PEPFAR investment in Rwanda. The amounts disbursed from CDC to the MOH, however, were slightly lower than the budgeted amounts (see Table 3-3). Where there was a balance, CDC approved car- rying it over to the next year, with the exception of 2015 to 2016, after PEPFAR had deemed the Program noncore. The total amount budgeted for the HRH Program comprised about 4 percent of the total annual health budget for the MOH as reported in the Ministryâs Health Resource Output Tracking Report for available concur- rent years (see Table 3-4). Given that government expenditure comprises less than half of the total expenditure on health in Rwanda (as described in Figure 3-3), the HRH Program likely represented less than 3 percent of total health spending in Rwanda at the time it was implemented.
66 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA TABLE 3-2 HRH Program Budget by Year (U.S. Dollars) Fiscal Year U.S. Government Global Fund Total Annual Budget 2012â2013 12,300,163 6,775,325 19,075,488 2013â2014 14,971,013 6,775,325 21,746,338 2014â2015 12,976,798 6,775,325 19,752,123 2015â2016 11,000,000 5,729,026 16,729,026 2016â2017 10,500,000 5,847,111 16,347,111 2017â2018 0 5,519,856 5,519,856 2018â2019 0 362,246 362,246 Total HRH 61,747,974 37,784,214 99,532,188 Program Budget NOTES: Amounts and totals are reported as provided by the MOH in current U.S. dollars. On average, inflation was about 5.2 percent in the same period, and the average growth of the exchange rate was about 5.8 percent between 2012 and 2018 (World Bank, 2019f,g). U.S. government and Rwandan FYs are not the same: U.S. FY runs October 1 through September 30; Rwandan FY runs July 1 through June 30. SOURCE: Financial data provided by the MOH. Figure 3-5 provides the context of the MOHâs broader concurrent workforce-related expenditures before and during the HRH Program. From FY 2010/2011 to FY 2014/2015, as the amount of MOH expenditure going toward health workforce increased there was a small decline in workforce ex- penditure as a proportion of the total MOH expenditure, from 42 percent to 39 percent. Concurrently, there was an increase in the proportion of investments for drugs and other consumables and for materials and equipment. However, trends in these data are difficult to interpret because of changes from year to year in how expenditures were reported by the MOH. It is also not possible to clearly interpret how the overall expenditures of the MOH intersected with the HRH Program expenditures because these reports do not indicate whether these ex- TABLE 3-3 CDC Disbursements for the HRH Program by Year (U.S. Dollars) Disbursement Approved Year from CDC Carryover Expenses Balance 2012â2013 12,300,163 6,630,040 5,670,123 2013â2014 12,577,279 5,670,123 4,898,183 13,349,219 2014â2015 12,769,798 13,349,219 26,119,017 â 2015â2016 11,000,000 10,558,378 441,622 2016â2017 10,058,378 10,500,000 Total 58,705,618 SOURCE: Financial data provided by the MOH.
HRH PROGRAM CONTEXT, VISION, AND DESIGN 67 TABLE 3-4 Ministry of Health Budget and Expenditures Reported Preceding and During the HRH Program (U.S. Dollars, Rounded to Millions) FY FY FY FY FY 2010/2011 2011/2012 2012/2013 2013/2014 2014/2015 Total Budget 514 539 533 * 499 milliona millionb millionc milliond Total Expenditure 420 514 * 481 480 milliona millionc milliond milliond NOTES: Amounts shown are the current amounts reported at the time each source report was published. Amounts were provided in Rwandan francs (RWF) after FY 2011/2012 and therefore converted to U.S. dollars (USD) using the following World Bank historical exchange rates, rounded to the nearest dollar: in 2013, 1 USD = 647 RWF; in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g). * Expenditure data for FY 2012/13 and budget data for FY 2013/14 were not reported. SOURCES: a MOH, 2012c; b MOH, 2012c, 2013b; c MOH, 2013b; d MOH, 2018b. 600 514 500 481 480 420 203 144 400 (40%) 163 (30%) Total (USD) Millions of U.S. Dollars (34%) 145 (35%) Other* (USD) 300 Workforce** (USD) 192 (40%) 187 213 (39%) Materials & Equipment*** (USD) 200 176 (41%) (42%) Drugs & Health-Related 38 (8%) Consumables**** (USD) 34 (7%) 100 20 (5%) 17 (3%) 106 96 79 81 (22%) (20%) (19%) (16%) - FY 2010/2011 FY 2011/2012 FY 2013/2014 FY 2014/2015 FIGURE 3-5 MOH expenditures FY 2010/2011âFY 2014/2015 (U.S. dollars). Fiscal Year NOTES: USD = U.S. dollars. Amounts shown are the current amounts reported at the time each source report was published. Different input categories were used each year and aggregated ac- cording to the four major categories shown in the figure. Expenditures after FY 2011/2013 were provided in Rwandan francs (RWF) and converted to USD using the World Bank conversion for each year and rounded to the nearest whole number: in 2011, 1 USD = 600 RWF; in 2012, 1 USD = 614 RWF; in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g). * Other comprises public relations and awareness such as advertising, campaigns, and com- munications for health; domestic and international travel and transport costs; indirect costs; infrastructure; overhead and general administrations costs; and vehicles and maintenance. ** Workforce comprises in-service training and workshops, incentives for community health workers, pre-service training and workshops, salaries for contracted government personnel, salaries for government personnel, salaries for nongovernmental personnel, salaries for short- term consultants, service provision, technical assistance, and performance-based financing. *** Materials & Equipment comprises medical and nonmedical equipment. **** Drugs & Health-Related Consumables comprises commodities, consumables, and drugs. SOURCES: MOH, 2012c, 2013b, 2016a, 2018b.
68 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA TABLE 3-5 MOH Health Workforce Expenditures with Category Breakdowns by Year (U.S. Dollars) FY FY FY FY 2010/2011 2011/2012 2013/2014 2014/2015 Total Expenditures 419,636,014 514,014,611 480,938,416 479,889,043 Workforce (Percent of 40% 39% Expenditures) Workforce Expenditure 176,132,460 212,751,636 Amount Workforce Category Breakdowns (as Reported) Direct Salaries/Labor Costs/ 109,328,045 28% 30% Remuneration (of total (of total expenditures) expenditures) Capacity Building 42,369,081 7% (of total 5% (of total expenditures) expenditures) Incentives 24,435,334 In-service Training and 40,200,881 Workshops Incentives for Community 29,381,321 Health Workers Pre-service Training and 8,064,948 Workshops Salaries (Contracted 35,049,224 Government Personnel) Salaries (Government 27,784,464 Personnel) Salaries (Nongovernment 1,475,639 Personnel) Salaries (Short-Term 3,998,622 Consultants) Service Provision 18,688,571 Technical Assistance 21,554,278 Performance-Based 26,553,688 Financing Social Benefits 5% (of total 4% (of total expenditures) expenditures) NOTES: Amounts shown are the current amounts reported at the time each source report was published. In FY 2013/2014 and FY 2014/2015 expenditure categories were reported as percentages of the total expenditure. Amounts reported in Rwandan francs (RWF) were con- verted to U.S. dollars (USD) using the World Bankâs historical exchange rates, rounded to the nearest dollar: in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g). SOURCES: MOH, 2012c, 2013b, 2016a, 2018b.
HRH PROGRAM CONTEXT, VISION, AND DESIGN 69 penditures include or exclude the funds expended as part of the HRH Program. In addition, there may be other sources of expenditures on HRH in Rwanda. The categories of expenditures aggregated into the broad category of âworkforce,â and how they were reported, varied from year to year (see Table 3-5). A consistent pattern was that salaries and labor costs represented the largest proportion of expenditures. When reported, categories such as ca- pacity building, pre-service training, and workshops represent a very small amount of the total investment. HUMAN RESOURCES FOR HEALTH PROGRAM VISION AND DESIGN Political, Economic, and Social Context During the Design of the HRH Program By 2012, when the HRH Program was launched, Rwanda was viewed as an ideal environment, not least because it was perceived as being a âpeaceful and stable country,â which was seen as an important factor for successful program implementation. Health-sector achievements in the in- tervening years (see Chapter 7) made the landscape especially attractive for international donors: Everyone wants to work with a winning team. Even before HRH, there [was] evidence on the ground the Rwandan health system was performing. We, in the last 15 years, have reduced maternal mortality tremendously, neonatal mortality, achieved the [Millen- nium Development Goals], and we are on the track with the Abuja Declaration for Health, which is the amount allocated for health in general â¦ one of the highest in Africa. So, all these health indica- tors were improving, [and] I think it becomes much easier work- ing with someone who is already busy working for himself and help them achieve results, than trying to invest where you donât see results anyway. (09, University of Rwanda Administrator in Obstetrics and Gynecology) These achievements built on years of previous engagements by and with the United States and other partners in Rwanda, which created energy and interest in continuing to develop the health sector: From the beginning, USAID and CDC were on boardâ¦. Other partners were interested, WHO and UN agencies, other bilaterals â¦ also I think a lot of U.S. institutions had an interest here â¦ fam-
70 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA ily medicine and Tulane University, but also Yale University was present. They already had an MOU [memorandum of understand- ing] with the Ministry of Health, which was then integrated into the HRH [Program]. So there was definitely a strong presence to create a core group of people, to create a momentum on that. (22, Non-Government of Rwanda HRH Program Administrator from an International NGO) The perceived political will, on the part of the United States at the time and the Government of Rwanda, also created enabling conditions to develop and fund a holistic HSS program such as the HRH Program: [T]he government of Rwanda and even the side of the U.S. govern- mentâof CDC, even the Global Fund â¦ [were] very committed and convinced that this was the right program to face the situa- tion, and so there were a lot of high-level advocacy and enthusi- asm on both sides. (20, Government of Rwanda HRH Program Administrator) Across all respondent groups, the perception was that the Rwandan governmentâs leadership in the context of a strong health system was key to successful programming: The first factor is political will. The Ministry of Health put so much effort in HRH programming for it to succeed â¦ the environment was good enough for the HRH Program to operate without any challenge as the system was already well established. (30, For- mer Government of Rwanda Program Administrator and PLHIV Representative) There is a huge leadership commitment in Rwanda. The government â¦ wants to be a partner in everything that is happening, working hand in hand with partners and being in the driverâs seatâ¦.ââThatâs a huge success factor. (11, International NGO Representative) If you brought a program in a country where the leadership is not very strong in delivering results, we wouldnât be where we are. (09, University of Rwanda Administrator in Obstetrics and Gynecology) The facilitating environment in Rwanda, from the perspective of U.S. institution (USI) faculty participating in the HRH Program, had three parts. First, Rwanda was viewed as having a âflexibleâ health system that could change with an evolving health workforce. Second, the government
HRH PROGRAM CONTEXT, VISION, AND DESIGN 71 was confident in its successes in fighting HIV. Finally, existing infrastruc- ture, such as roads and Internet access, was seen as facilitating successful implementation. HRH Program Vision At the time of the HRH Programâs design and funding, Rwanda was implementing the Health Sector Strategic Plan 2009â2012 and the HIV/ AIDS National Strategic Plan 2009â2012, both of which had goals around the availability, quality, and rational use of HRH in service of improved health outcomes, in alignment with Vision 2020 goals of health equity, universal health coverage, and increased access to and delivery of quality health care (CDC, 2014; MOH, 2011b; Uwizeye et al., 2018). However, there were several critical obstacles within the Rwandan health care system, including the shortage of skilled health workers, poor quality of health worker education, and inadequate infrastructure, equipment, and manage- ment in health facilities (MOH, 2011b, 2014b, 2016b). According to program documents, the HRH Program was designed to address those challenges as a large, systems-based, health-sector-wide initia- tive to scale up institutional and training capacity and create a high-quality, sustainable health system in Rwanda capable of providing âworld-class careâ (CDC, 2012; MOH, 2011b, 2012b, 2014b, 2016b). Upgrading the health professional workforce to be of âsufficient quantity and quality to meet the national needâ would âcontribute to the distal goal of improved availability and quality of care in Rwandaâ (MOH, 2014b). Strategies would focus on increasing skill levels and specialization of health profes- sionals and educators, establishing high-quality training sites, accelerating recruitment and retention of students, creating a culture and career ladder for health professional teaching, and expanding research partnerships and academic exchange (MOH, 2011b). According to the Government of Rwanda, sustainable impact of the HRH Program would be the countryâs ability to produce a supply of new skilled, specialized health workers and Rwandan health educators and to ensure adequate infrastructure, equipment, and supplies without external financial support (MOH, 2014b). This country vision aligned with the PEPFAR 2.0 (2008â2014) strategic vision and focus on developing a sus- tainable HIV response that supported HSS activities with partner govern- ments (PEPFAR, 2019a,b). With the inception of PEPFAR 3.0 in late 2014, PEPFARâs strategic vision pivoted away from this horizontal approach and toward targeted epidemic control (PEPFAR, 2014). This shift in priorities did not fully align with Rwandaâs strategic goals, as the Government of Rwanda has consis- tently planned for developing the capacity of its health system. Iterations of
72 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA Rwandaâs Health Sector Strategic Plans for 2009 to 2024 have emphasized HSS in key intervention areas, community-based health insurance, and qual- ity of care through performance-based financing, while outlining policies to address challenges for vulnerable and marginalized populations, sustainable health system financing, and prevention and control of noncommunicable diseases (MOH, 2009a, 2012d, 2018a). Against this backdrop, the HRH Program was conceived with the goal of ultimately increasing the quality of health care delivery and the overall health care system in Rwanda, contrib- uting to the governmentâs âmandate in terms of developing capacity of the health staffâ (45, Government of Rwanda HRH Program Administrator). Interview respondents for this evaluation shared the same understand- ing of the HRH Programâs rationale, which they saw as being borne out of a need to rebuild the health system following the genocide against the Tutsi and in the face of an HIV epidemic and an aging population of people living with HIV (PLHIV): [O]ur discussion in 2010 to 2012 was how can we develop this capacity, rebuild this health system by working on the areas which are highly affected, which is medical personnel, nurses, specialists, doctors. The demand was also high not only for a single disease, but for everything. For HIV/AIDS is not seen as a single disease as such; sometimes it starts as a small virus and it ends by being a whole medicine. (01, Government of Rwanda HRH Program Administrator) Thus, the HRH Program âwas fully integrated into the health systemâ with potential impacts beyond âjust one diseaseâ such as HIV (87, Gov- ernment of Rwanda Program Administrator). This is in keeping with the evolution of Rwandaâs health sector strategic planning at the time, which focused on health-related MDGs, and specifically maternal and child mor- tality in the first phase (2005â2009) and transitioning to a focus on HSS and financial access to health services in the second phase (2009â2012). The third phase (2012â2018) emphasized health resource management and governance mechanisms (MOH, 2012d). HIV Achievements in Rwanda By 2011, when the HRH Program was being designed, Rwanda had made notable achievements in addressing the HIV epidemic relative to other countries in the region, as discussed in Chapter 1. Concurrent to the evolution in the health-sector strategic planning was a process of decen- tralization that facilitated increased access to HIV services. The first phase (2000â2005) facilitated the expansion of HIV services to lower-level facili-
HRH PROGRAM CONTEXT, VISION, AND DESIGN 73 ties (Binagwaho et al., 2016). This effort helped to distribute and expand HIV care by integrating community health workers who were providing services at the village level with the more advanced care offered at health centers, and district and referral hospitals (Binagwaho et al., 2016). In 2008, during the second phase of decentralization (2006â2010), the Rwandan health system moved authority to the district level to launch new HIV treatment sites. However, coordination of services between HIV and other care remained poor. While districts received funding for health, they received funding for essential HIV/AIDS services much more slowly as PEPFAR and the Global Fund followed different paths to decentralization (Nsanzimana et al., 2015). The second Health Sector Strategic Plan in 2009 called for the need to better integrate HIV/AIDS care into routine health services. To support this effort, the Government of Rwanda entered into a cooperative agreement with CDC to increase staffing (CDC, 2009b). It also started transferring management of HIV patients from international partners to government- run programs (PEPFAR, 2010; PEPFAR Rwanda, 2011). This transition resulted in a sustainable and successful HIV program that now has the internal expertise to manage HIV care at all levels (Binagwaho et al., 2016). During the third phase of decentralization, starting in 2011, the Rwanda Biomedical Center restructured HIV coordinating mechanisms to facilitate better integration with other disease-specific programs (Nsanzimana et al., 2015). The previous HIV programs were dissolved, including the National AIDS Control Commission, so other disease-focused programs could be leveraged to create operational efficiencies, especially given dwindling re- sources. The third Health Sector Strategic Plan (2012â2018) called for the integration of HIV services at a decentralized level, the need to improve quality, and the need to maintain trained and adequate numbers of staff at all facilities (MOH, 2012d). The result of this decentralization has been a rapid increase in the number of facilities offering ART services, from 4 in 2002 to 552 in 2016, as reported in the Rwanda Integrated Health Man- agement Information System. Respondents related the success of Rwandaâs health sector to the win- dow of opportunity provided to the HRH Program. Respondents perceived Rwanda as having âachieved all the expressed outcomes that we entered with the intent of achieving,â providing an opportunity to explore what could be done With a vertically funded program that has achieved outcomes that were desired but now is turning to look at the larger needs of those already infected HIV-positive individuals for their broader health care needs as we move forward in their care. (25, Non-Government of Rwanda HRH Program Administrator and U.S. Government Donor)
74 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA Rwanda was seen as an interesting test case for a different model of de- velopment that leveraged the gains achieved through a vertical program to strengthen an entire health system. To the extent that HIV outcomes were considered in the design of the HRH Program, the MOH viewed building a specialized health workforce as a priority in addressing long-term care for PLHIV and infectious disease control, which was considered a moral, epidemio- logic, and economic necessityâa âtriple imperativeâ (Binagwaho et al., 2013). Respondents working with NGOs in the HRH and HIV space in Rwanda felt that an HRH Program with the goals of strengthening the health system could have an impact on HIV outcomes: [T]he program was not designed specifically to improve the lives of people with HIV/AIDS. It is a health system strengthening inter- vention, and it will help the system; obviously, it will support the people who have HIV/AIDS because of their frequent contact with the health system. (05, Other International NGO Representative) HRH Program Goals, Objectives, and Strategy While there was general agreement on the overarching vision of the HRH Program, there was less congruence between program documentation and interview respondents on the intended mechanisms for achieving this vision. Program documents illustrated that developing the Rwandan health education system remained a principal component of the HRH Program. Throughout the Program, the MOH noted that focusing on health education was a mechanism for filling the gap between the supply and demand of highly qualified health professionals in Rwanda (MOH, 2011b, 2015b, 2016b). By the midterm review, however, key HRH Program activities such as twin- ning through participating USIs that formed the Academic Consortium were deemed necessary to âgap-fillâ and âsimultaneously build the long-term capacity of their counterparts and host institutionsâ (MOH, 2016b). This evolved from the initial premise that the Program was designed to simply fund the mechanism for dramatically increasing production of health workers in a short time frame (MOH, 2011b). As highlighted in the 2011 HRH Program proposal, the original objectives were to increase the number of physicians and physician specialists, continue advancing the skill levels of nurses and midwives, introduce the role of health manager into the Rwandan health system, launch the Rwanda School of Dentistry (the first class began in fall 2013 at the University of Rwanda), and build the institutional capacity of Rwandaâs health professional schools and clinical teaching hospitals to sustain high-quality health education (MOH, 2011b). Most of these objectives continued to be referenced as core program expectations informing the development of core indicators in the Programâs
HRH PROGRAM CONTEXT, VISION, AND DESIGN 75 eventual monitoring and evaluation (M&E) plan and as objectives in a results framework for the midterm review (see Table 3-6). Many outlined areas of work, such as expanding research partnerships and academic ex- change, were also generally consistent through the years though with vary- ing specificity. For example, the 2015 programmatic/technical work plan emphasized âinternationally benchmarked curriculaâ in one of the areas TABLE 3-6 Evolution of the HRH Programâs Goals and Approaches 2015 Project 2016 Rwanda 2011 2014 Narrative and HRH Program Rwanda HRH HRH Monitoring Programmatic/ Midterm Review Program, 2011â2019, and Evaluation Plan, Technical Work Report (October Funding Proposal March 2014 Plan 2015âJune 2016) Aim: Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Identified Challenges to Achieving Aim Critical shortage of Reduce the critical Critical shortage skilled health workers shortage of skilled of skilled health health professionals workers Poor quality of health Improve the quality Poor quality of worker education of health professional health worker education education Inadequate Increase and diversify Sustainability of infrastructure and health care worker health education equipment in health specialties system facilities Inadequate Enhance management of health infrastructure facilities and equipment in health facilities and educational sites Improve health facilities and educational site management continued
76 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA TABLE 3-6 Continued 2011 2015 2016 Rwanda HRH Rwanda HRH 2014 Project Narrative Program Midterm Program, 2011â HRH Monitoring and Programmatic/ Review Report 2019, Funding and Evaluation Plan, Technical Work (October 2015âJune Proposal March 2014 Plan 2016) Aim: Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Goals (2011); Core Program Expectations (2014); or Complete Results Framework Objectives (2016) Increase the number Increase the Increase the total of physicians from number of general number of physicians 633 to 1,182, practitioners and the number practicing in Increase the number of physician Rwanda from 625 to of physician specialists in areas 1,182 (disaggregate specialists in priority such as internal by cadre clinical areas medicine, family and community Increase the medicine, obstetrics number of and gynecology, physician specialists pediatrics, surgery, (subspecialists) from and anesthesiology 128 to 551 from 150 to 551 Dramatically Increase the number Increase the total advance the skill of nurses and number of nurses level of nurses/ midwives from and midwives midwives by 9,670 to 10,200 increasing the Increase the skill number of nurses/ level of nurses and midwives with A0 midwives credentials from 104 to 1,011 and the number of nurses/ midwives with A1 credentials from 797 to 5,095. These actions will increase the overall number of nurses/midwives from 6,970 to 9,178
HRH PROGRAM CONTEXT, VISION, AND DESIGN 77 TABLE 3-6 Continued 2015 2011 Project 2016 Rwanda HRH Rwanda HRH 2014 Narrative and Program Midterm Program, 2011â HRH Monitoring Programmatic/ Review Report 2019, Funding and Evaluation Plan, Technical Work (October 2015âJune Proposal March 2014 Plan 2016) Aim: Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Goals (2011); Core Program Expectations (2014); or Complete Results Framework Objectives (2016) continued Introduce the role Introduce the role of Introduce the role of of health manager health manager and health manager at into the Rwandan increase their number district hospital level health system and from 7 to 157 increase the number (introduce trained of trained health health manager managers from 7 to position in district 157 hospital and develop a job description) Launch the Rwanda Launch the school of Increase the number School of Dentistry, dentistry and increase of oral health and increase the the number of health professionals number of oral professionals from health professionals 122 to 424 from 122 to 424 Build the institutional Create teaching capacity of the hospitals and medical medical, nursing, schools that have oral health, health the infrastructure, management schools, equipment, and and clinical teaching institutional capacity hospitals to sustain to sustain high- high-quality health quality education education continued
78 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA TABLE 3-6 Continued 2011 2015 2016 Rwanda HRH Rwanda HRH 2014 Project Narrative Program Midterm Program, 2011â HRH Monitoring and Programmatic/ Review Report 2019, Funding and Evaluation Plan, Technical Work (October 2015âJune Proposal March 2014 Plan 2016) Aim: Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Strategic Outcomes To improve the Improved capacity capacity of the of the University University of of Rwandaâs Rwandaâs College of CMHS to Medicine and Health implement quality, Sciences (CMHS) to competency-based implement quality, health professional competency-based education health professional programs education programs To establish Established an enabling environment in environment in CMHS schools and CMHS schools and training sites (e.g., training sites (e.g., referral hospitals) referral hospitals) to conducive to facilitate improved facilitating health professional improved health education professional education By 2019, have a Established a sustainable, skilled, sustainable, skilled and specialized and specialized health professional health professional workforce in workforce in Rwanda Rwanda
HRH PROGRAM CONTEXT, VISION, AND DESIGN 79 TABLE 3-6 Continued 2015 2011 Project 2016 Rwanda HRH Rwanda HRH 2014 Narrative and Program Midterm Program, 2011â HRH Monitoring Programmatic/ Review Report 2019, Funding and Evaluation Plan, Technical Work (October 2015âJune Proposal March 2014 Plan 2016) Aim: Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Areas of Work Increase skill levels Increase skills Increase skill levels and specialization levels and and specialization of health care specialization of health care professionals and of health care professionals and educators professionals and educators educators Establish high-quality Establish high- Establish high-quality clinical training sites quality clinical clinical training and schools training sites and sites and schools schools through procurement of infrastructure and equipment and improvement of health management capacity Accelerate Support Accelerate recruitment and recruitment recruitment and support student and retention support student retention of trainees retention and students in nursing and midwifery, biomedical laboratory sciences, medicine and surgery, and health management programs Implement Develop Implement integrated, and deliver integrated, competency-based internationally competency-based curricula benchmarked curricula curricula continued
80 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA TABLE 3-6 Continued 2011 2015 2016 Rwanda HRH Rwanda HRH 2014 Project Narrative Program Midterm Program, 2011â HRH Monitoring and Programmatic/ Review Report 2019, Funding and Evaluation Plan, Technical Work (October 2015âJune Proposal March 2014 Plan 2016) Aim: Build the health education infrastructure and health workforce necessary to create a high-quality, sustainable health care system in Rwanda Areas of Work continued Increase the Increase the Increase the importance of engagement importance of teaching and careers of health teaching and careers in health professions professionals in in health professions learning, teaching, and scholarships Build institutional Recruit to and Build institutional capacity for health retain faculty capacity for health education in Rwanda by education working with USIs Increase collaboration between health professional education stakeholders Enhance the recruitment and retention of graduates in the health care and health education sectors Development of faculty for health professional education Expand research Expand scientific Expand research partnerships and partnerships partnerships and academic exchange and academic academic exchange exchange Monitoring and evaluation
HRH PROGRAM CONTEXT, VISION, AND DESIGN 81 of work, whereas other program documents stated that the HRH Program would âimplement quality, competency-based instructionâ (MOH, 2011b, 2015b, 2016b). Notably, it was not until the MOHâs performance and measurement plan development process, which began after program implementation, that three overarching strategic outcomes emerged. Two of these outcomes highlighted the role of the University of Rwandaâs College of Medicine and Health Sciences (CMHS) in providing high-quality health professional edu- cation programs and accompanying training environments, alongside the third outcome of establishing a sustainable, skilled, and specialized health professional workforce (MOH, 2014b, 2015b). In addition, M&E had not been designated as a specific area of work until this point; the 2014 M&E plan acknowledged the challenge of establishing a baseline for all program areas, because M&E efforts were being articulated after implementation (MOH, 2014b). This was reflected in inconsistencies in baseline and target goals for the number of health professionals in several cadres, particularly in the number of nurses and midwives to be produced by the HRH Program once it began to deemphasize the upgrading of A2 to A1 nurses (MOH, 2011b, 2014b).4 Qualitative data reveal a lack of congruence in the pathways to reach the Program vision; respondents reported a range of strategies including improving quality of care, producing high-quality health workers, building primary care, building specialty care, strengthening the medical education system (including faculty), and improving the availability of equipment and infrastructure. Among Government of Rwanda respondents, the Programâs main objective was unequivocally to build a larger cadre of health care workers across specialties: The vision of the MOH was to improve the shortage of HRH, improve their quality in terms of skills and knowledge, and how to deal with some of the major issues that we had here in Rwanda. (48, Government of Rwanda HRH Program Administrator) To do this, as one HRH Program trainee articulated, it was necessary to build postgraduate training programs (32, University of Rwanda Non- Twinned Faculty and Former University of Rwanda Student in Obstetrics and Gynecology). In contrast, the perceived objectives of the HRH Program among most University of Rwanda and USI respondents were to upgrade the number 4â A2 nurses have completed secondary school education; A1 nurses receive a diploma after 3 years of training at a higher education institute; A0 nurses are graduates of a 4-year bachelorâs program and may go on to enroll in a masterâs program (Uwizeye et al., 2018).
82 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA and skills of Rwandan health educators. When probed, one Government of Rwanda respondent who had been involved in the Programâs design vehemently disagreed with the assertion that building capacity within the University of Rwanda was a programmatic objective: Creating people who want to teachâitâs something else. Donât forget that we [the HRH Program] create people to give care. That was the objective. And this has to be well understood. We needed people to provide care. Itâs good that we reinforced the University, but our program was not to reinforce the University. (18, Former Government of Rwanda HRH Program Administrator) Some USI faculty reported that the objectives and design of specific spe- cialties changed significantly over the course of the Program. For example, the initial objective of the nursing and midwifery activity was to contribute to the skills upgrading of A2 nurses, but this shifted to focusing on building an A0 nursing cadre through the development of the Master of Science in Nursing (MSN) program: The HRH Program started with increasing the level of A2 nurses to A1, to advanced diploma. Because the majority of health care providers in Rwanda were A1.... Then they started upgrading the A2 to advanced diploma. (31, University of Rwanda Administrator in Nursing and Midwifery) Similarly, the objectives around the Master of Hospital and Healthcare Administration program evolved as its developers gained an understanding of the context and needs in hospital administration: Initially, the program is for the [USI] faculty to come to Rwanda and work in a hospital and pair with the hospital administrator. So, instead of doing a formal education program â¦ we pair with them and help them side by side to do the day-to-day operationsâ¦. Later on, they discovered they donât have an official curriculum for hospital management so they started the development of a hospital management program that was based in the University of Rwanda School of Public Health. (06, USI Faculty in Pediatrics) The emphasis on specialized care over primary care evolved throughout the Program and was not clearly understood among different stakeholder groups. The Program comprised both pre- and in-service training activities, but the former were prioritized in accordance with national HRH policy because pre-service education âis less costly and gives immediate hand[s]
HRH PROGRAM CONTEXT, VISION, AND DESIGN 83 on skills to the health professionalâ (MOH, 2014c). The strategy to fo- cus on immediate hands-on skills was a central tenet of train-the-trainer models and the twinning program, which planned for Rwandan faculty and new graduates to phase out Academic Consortium faculty or to train colleagues in district hospitals, building capacity for local ownership and sustainability. Efforts to strengthen the medical and nursing programs and the medi- cal education system were under way before the HRH Program. In FY 2008 and FY 2009, USAID funded the University of Colorado to second âa family medicine faculty member â¦ to provide extensive practical teaching, postgraduate supervision, and assistance with [the] development of the Family Medicine programâ and to assist in integrating HIV/AIDS into postgraduate medical program curricula (PEPFAR Rwanda, 2008, 2009). By FY 2010, the University of Coloradoâs efforts had been sub- sumed under a large CDC-funded capacity-building program with Tulane University and had expanded to include support for a 4-year postgradu- ate medical program aimed at preparing âphysicians to function with a broad clinical scope â¦ to better address the burden of disease existing in Rwandaâs rural communities.â Twenty-three physicians, âincluding seven in Family and Community Medicine,â were enrolled (PEPFAR Rwanda, 2010). The in-country postgraduate medical program was initiated under the former Minister of Health, Jean Damascene Ntawukuriryayo, and contin- ued by his successor, Richard Sezibera (FlinkenflÃ¶gel et al., 2015). Although some of the work performed under Tulaneâs cooperative agreement with CDC was transitioned to the University of Rwanda, the agreement was not renewed at the close of its 5-year term under Agnes Binagwahoâs tenure as Minister of Health (PEPFAR Rwanda, 2012). One respondent who was involved in the design and early imple- mentation phases noted that primary care was central to the HRH Pro- gram at the start, but was excluded from the list of clinical areas during implementation: The emphasis on the primary care thing was one of the corner- stones of the HRH Program. The U.S. government came back to us when we submitted the proposal with lots of questions and con- cerns. They wanted to make sure that we really had an emphasis on primary care. So, we brought memos and answers to â¦ certify that primary care was going to continue to be a key emphasis of the program. But then, as soon as the program was approved and funded, shortly [thereafter] the family medicine residency was dis- continued and there was a shift from primary care to specialty care. (22, Non-Government of Rwanda HRH Program Administrator)
84 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA According to one respondent representing an international NGO, the reasoning behind supporting specialized care over primary care was not well understood, because it Flew in the face of the primary health care focus that people were having, particularly USAID, at the time [and there was] fear of donors that are putting a lot of resources in the specialized services is going to be done at the detriment of primary health care. (05, International NGO Representative) In contrast, a senior MOH official commented that there was no need to build primary care in Rwanda when the HRH Program started, be- cause there wereâand continue to beâother investments and efforts to strengthen primary care, but no efforts focused on specialized care. Notably, few respondents spoke about the objectives of the HRH Pro- gram in relation to HIV. Two University of Rwanda administrators reported that the Programâs goal should have been positioned in the context of âtransition from acute burdens of HIV as a signature illness to much more chronic disease managementâ (02, University of Rwanda Administrator). A former HRH Program administrator expanded on this idea: [T]he main problem was, we have a huge HIV population who have been on treatment for a very long period of time, and they started to develop other kinds of diseasesâinternal disease, where they might need surgery. They were in need of more specialized type of care that [could] be provided by very few specialists. (45, Government of Rwanda HRH Program Administrator) The literature echoes the HRH ProgramâHIV connection, in which it was imperative for the MOH to build a specialized health workforce to ad- dress long-term care for PLHIV and infectious disease control (Binagwaho et al., 2013). DESIGN PROCESS Figure 3-6 depicts the overall time line of the HRH Program. In 2011, the Government of Rwanda submitted an unsolicited proposal, which was funded in 2012 under the expanded Clinical Services Cooperative Agree- ment (CDC, 2012). Prior to funding, 18 USIs submitted letters of intent to join the Academic Consortium, although one withdrew before the Program was launched (MOH, 2011c). USI participation in the Program increased until 2015, when institutions began withdrawing. An MOU was established between the MOH and a new USI in 2018, after PEPFARâs
HRH PROGRAM CONTEXT, VISION, AND DESIGN 85 investment had ended. Membership in the Academic Consortium provided USIs with a mechanism through which to establish annual MOUs with the MOH. The Consortium was also a mechanism for determining the clini- cal purview of each USI. For example, Yale University was an obstetrics Contract/management U.S. institution involvement Academic programs â11 Government of Rwanda 18 USIs submit letters 1 USI partnership ends submits unsolicited of intent to join Academic before HRH Program proposal Consortium launch â12 1 USI exits Expansion of âStrengthening the HRH 10 USIs USI faculty Academic Ministry of Healthâs Capacity to Program join begin Consortium Respond to the HIV/AIDS Launch Academic arriving in Epidemic in the Republic of Consortium Rwanda Rwanda under PEPFARâ Cooperative Agreement â13 MHA program launched â14 âStrengthening Human Resources M&E plan Management of Clinical for Health Capacity in the Republic developed HRH Program Services of Rwanda under PEPFARâ transitioned to CoAg ends Cooperative Agreement issued MOH â15 1 USI joins 3 USIs exit MSN MGHD PEPFAR Academic Academic program program decides to Consortium Consortium launched launched cease funding â16 3 USIs exit Academic Consortium Midterm review conducted â17 1 USI exits Academic Consortium PEPFAR funding ends â18 5 USIs exit Academic Consortium 1 USI joins Academic Consortium â19 5 USIs exit Academic Consortium FIGURE 3-6 HRH Program time line. NOTE: HRH = human resources for health; MGHD = Master of Global Health Delivery; MSN = Master of Science in Nursing; PEPFAR = Presidentâs Emergency Plan for AIDS Relief; USI = U.S. institution.
86 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA and gynecology partner, and New York University and Emory University provided support to the nursing programs, both upgrading nurses and the establishment of the MSN program. USIs under the Academic Consortium contracted with individuals to work at the University of Rwanda to âtwinâ with and mentor existing Rwandan faculty, aid in strengthening existing residency programs and establish new programs, provide direct teaching services to Rwandan stu- dents, and, in some cases, provide direct care to patients. With few excep- tions, the USIs contracted with U.S. citizens and did not engage regional faculty who could lend practical experience and knowledge from a more locally relevant context to the Program (see Chapter 4 for a more detailed discussion). Data-Informed Design It was clear from interview respondents, including those who were engaged in the design of the HRH Program, that no specific baseline as- sessment was done prior to launching the Program. Six Government of Rwanda current program administrators referenced supporting documents, such as reports or an assessment, that informed the Program; three of these respondents specifically cited the Third Health Sector Strategic Plan and the Human Resources for Health Strategic Plan 2011â2016: [F]rom the HRH Strategic Plan, thatâs why they had all the infor- mation: How many specialists do we have? What are the problems? What are the number of physicians per population? And then from the strategy that was developed in 2010, if my memory is serving well, thatâs what informed that we need the program. We need trainees to provide care to the population. (45, Government of Rwanda HRH Program Administrator) The Third Health Sector Strategic Plan makes reference to an extensive situation analysis and comprehensive midterm review conducted in 2011, which informed the planâs priority to âimprove quantity and quality of hu- man resources for health (planning, quantity, quality, management)â (MOH, 2012d). Other HRH Strategic Plan objectives include increasing the number of trained and equitably distributed staff and improving health worker pro- ductivity and performance in part by improving the quality of pre-service training (MOH, 2011a). Similarly, PEPFARâs Rwanda Country Operational Plans reference Tulane Universityâs collaboration with other donors and the âNational Technical Working Group for Human Resources in an in-depth needs assessment â¦ designed to determine the health workforce needs in Rwanda, taking into account the disease burden, existing cadres and
HRH PROGRAM CONTEXT, VISION, AND DESIGN 87 ongoing trainings (pre- and in-service)â (PEPFAR Rwanda, 2010, 2011). However, respondents from the Clinton Health Access Initiative (CHAI), who collaborated with the MOH in the design of the HRH Program, felt the MOH was not in a position to examine or plan for their HRH needs: I think at the time, the reason the Ministry asked for our help was that they didnât have, you know, a very strong or well-established vision for where they should go in HRHâ¦. So, even though there were some documents put together, people didnât really take it seri- ously. I donât think there was any kind of plan for longer term.... I havenât seen needs assessments that were done in an objective way [or] any kind of transparent process. (22, Non-Government of Rwanda HRH Program Administrator) Collaboration During Design The HRH Program design process appeared to have involved members of the MOH and CHAI, without involving other key partners, such as the Ministry of Education (MOE), the Ministry of Finance and Economic Planning (MINECOFIN), or implementing partners working in HIV and HRH. CHAIâs role, with financial support from the ELMA Foundation, was one facilitator. At the request of then-Minister of Health Richard Sezi- bera, CHAI convened an HRH working group to prepare a road map and strategy. These documents formed the basis of the HRH Program proposal, which CHAI âunilaterally wrote â¦ and submitted to the U.S. Embassyâ (20, Government of Rwanda HRH Program Administrator). As part of this process, CHAI worked with the MOH to determine programs within the PEPFAR portfolio whose funding could be reduced or terminated to cover the costs of the Program: [CHAI] helped to figure out where the money was going to come from â¦ basically, itâs a budget and expenditure reporting from partners. We analyzed that, compared that to the Country Operat- ing Plan of the U.S. government. And then we sat down with the Ministry of Health, discussed the programs they wanted to keep, what they wanted to cut. We went through the list, suggested some cuts to find them funding. And then the Ministry went back to the U.S. government and said, âcut this, cut thatâ and thatâs how they found the funding. (20, Government of Rwanda HRH Program Administrator) This was corroborated by PEPFAR during one of the committeeâs open sessions. Once the Program was funded, CHAI was asked to stay on and
88 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA managed the HRH Program through the first year of implementation while the MOH set up internal management structures and onboarded staff. Although the MOHâMOE relationship faced challenges at the begin- ning of the HRH Program, it strengthened over time. Not involving the MOE during Program design and early implementation was perceived as a mistake: We actually started the HRH Program without the Ministry of Education involvement. Only after we launched it and it was a reality, I think, thatâs when we realized that we had made a mis- take by not involving education. To be fair, I think things were also in flux; even in the government, it was not totally clear where postgraduate training was supposed to be, whether in the School of Medicine, it was under Ministry of Health, or Education and where the respective roles started and stopped. But at some point, they [MOH officials] had to go to see the Minister of Education, Dr. Vincent Biruta, and basically apologize and explain what hap- pened â¦ from then on, things started to work well with the Minis- try of Education. (22, Non-Government of Rwanda HRH Program Administrator) One MOH respondent involved in the early stages of the Program de- scribed the MOE as âa contractorâ (18, Former Government of Rwanda HRH Program Administrator). A Non-Government of Rwanda HRH Pro- gram Administration respondent expressed his view that there was no re- lationship between the MOH and the MOE before the Program, and that a relationship had been built through MOHâMOE steering committees formed to set standards for health care professional education: We used to sit togetherâMinister of Health, Minister of Educa- tion, University [of Rwanda]âto try to understand the challenges and provide solutions together. I think from the HRH Program we strengthened this collaboration between the Ministry of Health and Education. (48, Government of Rwanda HRH Program Administrator) FINANCIAL MANAGEMENT The 2011 HRH Program proposal indicated that a Single Project Imple- mentation Unit would be established âto centralize programmatic, finan- cial, and administrative management functionsâ (MOH, 2011b). PEPFAR investments in the HRH Program flowed directly to the MOH, although according to one respondent, a âparastatalâ had been set up to manage
HRH PROGRAM CONTEXT, VISION, AND DESIGN 89 the money, where funds were disbursed by MINECOFIN. One respon- dent involved in the early phases of the Program noted that this design was intended âto try and eliminate the high cost of an international NGO go-between â¦ and having a government that had demonstrated their abil- ity to manage and oversee, monitor, and evaluate programsâ (25, Non- Government of Rwanda HRH Program Administrator). The Permanent Secretary of the MOH had oversight over the HRH Program budget. The Government of Rwanda employed a Public Financial Management system, in which funds were held in an account separate from the National Treasury. The system facilitated budget oversight, because the University of Rwanda used the same system. According to MOH data, the bulk of the expenditures from PEPFAR investments in the HRH Program went to USIs, followed by equipment pro- cured for health professional education and clinical training (see Table 3-7). Table 3-7 includes only funds provided by CDC and does not reflect other investments from sources such as the Global Fund and the MOH. According to the MOH, the $441,661 difference between the budgeted amount and the disbursed amount was not funded by CDC. The MOH articulated the importance of unmeasurable inputs that were critical for the running of the HRH Program, including overhead costs ex- pended by the University of Rwanda for managing its facilities, unallowable expenses incurred by USIs in recruiting and managing USI faculty, and other TABLE 3-7 HRH Program Expenditures of PEPFAR Investments (U.S. Dollars) Recipient âBudgeted Amount âDisbursed Amount USIs 31,180,833 29,804,744Â University of Rwanda 5,830,730 5,830,730 Equipment Procurement 16,083,568 17,901,279 Travel 429,600 429,600 Supplies 3,092,778 3,092,778 Program Management 2,529,771 1,646,487 Total HRH Program Expenditures 59,147,280 58,705,618 NOTES: HRH = human resources for health; USI = U.S. institution. The fluctuating exchange rate was set by the Rwandan National Bank. The original Program proposal had a specific line item for equipment maintenance, amounting to $1.5 million. A similar line item is not reflected in these expenditure data. SOURCE: Financial data provided by the MOH.
90 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA essential financial inputs by the MOH, such as annual travel allowances for all residents to move across sites for their rotations and visit their families when they were rotating at a site away from their primary residence. Financial audits of the Global Fundâs investments during periods concurrent with the HRH Program (2014, 2016â2018), and therefore presumably inclusive of funds applied toward the HRH Program, yielded no major concerns. Systems of internal control were typically found to be âgenerallyâ or âpartiallyâ effective. The 2014 audit identified three areas for improvement: data quality assurance, supporting documenta- tion, and low absorption of funds (Global Fund, 2014). The 2016â2018 Global Fund audit found, however, that grant oversight and assurance mechanisms needed significant improvement (Global Fund, 2019). Audits of other investments in the MOH had similar findings. An audit of the MOHâs management of Gavi funds rated the MOH as partially satisfac- tory (Gavi, 2018), and an assessment of a Swedish International Develop- ment Cooperation Agencyâsupported program that centered on building research and higher education capacity identified three key issues in the MOHâs financial management: lack of consolidated financial reporting; delays in disbursements to Swedish partner universities; and slow budget execution that affected implementation (Tvedten et al., 2018). Addition- ally, a thorough search for a CDC audit of the HRH Program was con- ducted, and none was located. Beyond the financial management processes, program administrators expressed some frustration about PEPFARâs processes and restrictions on how funding could be used, especially in light of the HRH Programâs ho- listic and integrated design: The second [challenge] was how surprisingly hard this was us- ing PEPFAR money â¦ it was a pain in the ass, Iâm sorry for my language.... For anything, any comment, any point, you need to ask for authorizationâ¦. People from Atlanta will have to sign something and Rwandans as well and â¦ you need to write 200 pages of document. The equipment that was supposed to come in year 1 came in year 3 or 4. It was really not easy to use the funds. (45, Government of Rwanda HRH Program Administrator) There are too many specialties that these fund were not really able to support and for us, I can see how [HIV] is related to that. A radiologist doesnât just treat a non-HIV patient, it treats both and the most complicated cases come from the people who have HIV. Same with dentistry. (03, Government of Rwanda HRH Program Administrator)
HRH PROGRAM CONTEXT, VISION, AND DESIGN 91 Institutional Financial Arrangements The 2011 HRH Program proposal outlined several key principles guid- ing the relationship between the MOH and USIs. The MOH would receive funds directly from the U.S. government and issue contracts to USIs. MOUs were established between USIs, the MOH, and the MINECOFIN. Each USI had separate coordinators for the academic aspects of the MOU and for finances. The mismatch of FYs between USIs, the Government of Rwanda, and CDC, compounded by yet another difference in the University of Rwandaâs academic year, presented an additional contractual challenge that affected payment scheduling: CDC came with a specific CDC fiscal year for some reason which was March-April [and] that was even more difficult because it pushes the quarter to another one, so from October you will see that it comes to the other year and all this payment to universities was done quarterly. (20, Government of Rwanda HRH Program Administrator) The 2016 midterm review also noted this challenge. Upfront stipula- tions around financial management included that USIsâ overhead to the programmatic and administrative functions could not exceed 7 percent of total direct costs for recruiting faculty, predicated on the assumption that the Government of Rwanda would be responsible for some of these ad- ministrative activities (MOH, 2011b,c). The HRH Program proposal also stipulated that USI faculty were expected to commit to residing in Rwanda for at least 11 months and would receive: Salary and benefits â¦ lower than they could obtain at their univer- sities in the U.S. This reflects the fact that the HRH Program is a development project meant to serve poor people in a resource-poor country and also reflecting the fact that this is an educational op- portunity for the U.S. faculty. (MOH, 2011b) USI faculty were mostly contract hires, not existing faculty receiving time to dedicate to the HRH Program. PROGRAMMATIC MANAGEMENT APPROACHES AND CHALLENGES The HRH Program experienced challenges in its design, launch, and execution that were both internal to Rwanda (between the MOH and the
92 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA MOE and stakeholders outside the Government of Rwanda) and external to Rwanda (between the MOH and USIs). Interinstitutional Relations MOHâMOE Relations As indicated above, the design phase did not actively engage the MOE, a gap that had implications in the early implementation period. Some fac- ulty at the University of Rwanda were aware of the HRH Program before it was launched, and reported being excited that former U.S. President Bill Clinton and Rwandan President Paul Kagame had participated (37, Univer- sity of Rwanda Administrator), although respondents knew of other faculty members who were not aware of the Program: I was a little bit ahead of my colleagues who had those challenges of not being informed ahead, but for me, I knew that the program was there. (80, USI Faculty in Nursing) This absence of communication about the Program included incoming students who were anticipating joining training programs abroad, but were told they would be enrolled in programs in Rwanda: Because we were not informed of the Program â¦ yeah it was a surprise! We did exams, competing for scholarships [in other East African countries]. We were responsible for getting a mission let- ter from in-country ourselvesâ¦. When we presented themâ¦. They said, âNo, no, we are not goingâ and they didnât inform [us] that we have a program locally. We kept asking, âWhat happened?â¦ What is next?â Then it was 6 months later they say, âAll those who have scholarship are going to have your program locally.â (47, University of Rwanda Faculty and Former Student in Nursing) The MOH received and managed external HRH Program funding and was responsible for signing MOUs with USIs. The MOE had little input into the allocation of funds. The roles and the responsibilities of the MOH and the MOE, which required a collaborative relationship between the two ministries, was articulated by a respondent representing a professional association: The Ministry of Health had nothing to do with the assessment of the students or their teaching, not very much, except that the Min- istry of Health was the one that was creating the environment in
HRH PROGRAM CONTEXT, VISION, AND DESIGN 93 which people were trained because, in the Ministry of Health they were responsible for the good running of the hospital, and with- out a hospital that is capable of providing good care you cannot talk about good teaching of postgraduates.â¦ The academic head of medicine [at the University of Rwanda] was the one who was looking at the planning of the rotations, he would be the one who decides who comes to King Faisal, who goes to CHUK [Centre Hospitalier Universitaire de Kigali/University Teaching Hospital, Kigali], who goes to Kanombe â¦ how long that person will stay and will also come for a visit to see if people are there to get reports of what is happening. (35, University of Rwanda Non- Twinned Faculty in Internal Medicine and Professional Association Representative) During implementation, University of Rwanda administrators reported that the communication between the MOE and the MOH was clear and managed through a steering committee chaired by the principal of CMHS. They reported that he had oversight over all of the operational and pro- grammatic issues with HRH trainees, while the MOH dealt with contrac- tual issues and accreditations: We have a steering committee â¦ chaired by the principal of the [College of Medicine and Health Sciences] and is composed [of] all deans of the college where we have the HRH Program, â¦ and the team of HRH from the Ministry of Health. We sometimes invite also HoDs [heads of department] and if itâs necessary, we invite the HRH faculty. (37, University of Rwanda Administrator) MOH respondents also mentioned the value of the steering committee as the entity âin charge of giving the guidance and making the policiesâ (20, Government of Rwanda HRH Program Administrator), which strengthened the ministriesâ relationship. MOH respondents also articulated the roles and responsibilities, while reinforcing the idea that âit is one governmentâ and neither ministry could unilaterally determine the priorities of both sec- tors (03, Government of Rwanda HRH Program Administrator). The steer- ing committee was also responsible for ensuring no duplication of efforts between the HRH Program and other investments in strengthening health workers, such as the Capacity Development Pooled Fund. USI faculty described the relationship between the MOH and the MOE as âpainfulâ (23, USI Twinned Faculty in Nursing), affecting their ability to support the review and approval of new or updated curricula and ac- creditation of new programs:
94 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA I would just think that the Ministry of Health would want the best teachers but thatâs not what they have been doing. (16, USI Faculty in Obstetrics and Gynecology) MOHâInternational NGO Relations As with the design, respondents from international NGOs felt the MOH was not inclusive of other organizations in the programâs manage- ment and implementation: My view is that at least the Ministry of Health doesnât try to involve other partnersâ¦. My feeling is that the Government of Rwanda, the Ministry of Health, has taken an attitude of saying: âYou know, this is our business and we are notâ¦.â (05, Interna- tional NGO Representative) Coordination [between the HRH Program and other programs] wasnât great â¦ activities tended to occur in silos. (26, International NGO Representative) MOHâUSI Relations The 2016 midterm review highlighted challenges in contracting pro- cesses: lack of clarity around leave and other policies, time line and pro- cesses for obtaining work permits and licenses for USI faculty, and funding and reporting requirements that resulted in delays in drafting MOUs; delays in annual revising and reissuing MOUs; late award notification, which delayed program funding and USI reimbursements; and a need to accom- modate USI and HRH Program processes delaying submission of invoices and payment (MOH, 2016b). Interview respondents for this evaluation echoed many of these chal- lenges. From the USI perspective, challenges with contracting included contracts being limited to 1 year, requiring annual renewal, differences in malpractice standards, and the exclusion of overhead from contracts (de- scribed in the previous section), which was viewed as an âextra horrorâ that was ânot realistic for a long-term project with a U.S. institutionâ (15, USI Faculty in Hospital and Health Administration). MOH delays in issu- ing contracts, raised in the midterm review and in interviews, had notable implications on the presence of USI faculty at the University of Rwanda and their personal and professional futures: We did not start immediately in year 1â¦. We had no contract. There were people from here who were told that there would be contracts in July. We got our contract in Novemberâ¦. They donât
HRH PROGRAM CONTEXT, VISION, AND DESIGN 95 get it. People stay here without contracts for periods of timeâ¦. [The MOH coordinator] was like, âOh, weâll get the money,â but when? My faculty is the sole provider of her family. She canât work for free. (17, USI Faculty in Obstetrics and Gynecology) We would stay on a few months to finish up or to make a transi- tion. But itâs very unsettling for the people [Rwandan faculty and residents] that you are working with when you say, âI hope Iâll be back.â (16, USI Faculty in Obstetrics and Gynecology) Program administrators who worked outside of the Government of Rwanda shared that some USI faculty âpulled outâ of the HRH Program because the financial gap (due to these contract conditions) âwas putting enormous pressure â¦ so it was too much riskâ (22, Non-Government of Rwanda HRH Program Administrator). There was also fluctuation in the number of USIs participating in the Program. In 2011, 18 programs had submitted letters of intent to join the Academic Consortium (11 in medi- cine, 6 in nursing and midwifery, and 1 in health management and global health delivery). In 2012, 1 of those institutions withdrew prior to program launch, though an additional 10 joined the program (9 in medicine, 1 in nursing and midwifery). USIs began exiting the Academic Consortium in 2015 with three exiting in 2015 (two in medicine, one in nursing and mid- wifery); three in 2016 (two in medicine, one in nursing and midwifery); one in 2017 (from nursing and midwifery); five in 2018 (four in medicine and one in health management and global health delivery); and five in 2019 (two in medicine and three in nursing and midwifery). Two USIs did join during that period: one in 2015 (in health management and global health delivery) and one in 2018 (in medicine). That USIs were expected to spend no more than 7 percent on admin- istrative activities related to recruitment reinforced a feeling that these in- stitutions were nothing but a ârecruitment firm, rather than true partnersâ (06, USI Twinned Faculty in Pediatrics) in building the institutional capac- ity to provide high-quality health professional education. This feeling was exacerbated in situations when the MOH told USIs its needs, sometimes with little warning, and with USIs conducting the screening and proposing individuals and the MOH making final decisions. Additionally, differing benefits between the two countries created obstacles, especially around maternity leave, which was part of the contracts USIs issued but was not a Government of Rwanda benefit. A similar obstacle was perceived with sick leave: In Rwanda, we know that sick leave is something that you take when you are sick. In the U.S., sick leave are accrued and you have day per month as a right and when you do not take them you will
96 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA add those accrued sick leave into your annual leave to sometimes we could have faculty doing 60 days or 40 days of leave and you know for an academic year where you have to work 1 year, it is not easy â¦ it had implications on the budget, meaning you are pay- ing someone who did not work for 60 days, did not deliver. (20, Government of Rwanda HRH Program Administrator) The 2011 proposal indicated that âbasic benefits [would] be providedâ to USI faculty, naming only health insurance, emergency evacuation insur- ance, adequate housing, and roundtrip airfare (MOH, 2011b). Government of Rwanda staff who managed the HRH Program also sometimes felt disappointed by USIs, which was seen as affecting program- matic success: There are also some institutions that disappointed us in the middle just by withdrawing from the program or not sending the required qualified staff. All those already are jeopardizing the good imple- mentation of the program. (03, Government of Rwanda HRH Program Administrator) Communication from USIs to the MOH about the physician specialists, nurses, and midwives selected to travel to Rwanda, specifically their expe- rience and seniority, also affected the relationship, although this was seen as having improved âover time as we went on mentioning this challenge the profiles changed and I think they would send even better peopleâ (48, Government of Rwanda HRH Program Administrator). MONITORING AND EVALUATION Reporting requirements for the HRH Program evolved over the years, in part because of the changing funding mechanisms as well as the broader shift in standards for M&E. In April 2012, before the Program was inte- grated into the MOHâs periodic program reports for the Clinical Services Cooperative Agreement, the MOH submitted to CDC a detailed work plan and budget for the Program. The work plan was similar to a logical frame- work and included goals, objectives, activities, a quarterly time line, and program indicators that were consistent, if not more detailed, than CDC reporting requirements at the time (CDC, 2009a; MOH, 2010, 2012a). Program indicators for the HRH Program and other components of the Clinical Services Cooperative Agreement, however, remained largely output measures (CDC, 2009a; MOH, 2010, 2012e). In 2013, the work plan for the Clinical Services Cooperative Agreement Continuing Application called for the development of an âM&E frame-
HRH PROGRAM CONTEXT, VISION, AND DESIGN 97 work and indicatorsâ as one of the HRH Programâs âmajor programmatic/ technical activitiesâ (MOH, 2013a). An open invitation for Program par- ticipants to volunteer to join one of two stakeholder groupsâan in-country M&E Working Group (comprising HRH Program members and Rwandan faculty) and a USI M&E advisory groupâwas also sent in 2013 to support the M&E plan development. As a result, the MOH submitted to CDC the Programâs âHuman Resources for Health Monitoring & Evaluation Planâ in March 2014 (MOH, 2014b). In addition to the three main strategic outcomes described previously, the M&E plan provided outputs, detailed process and outcome measures, data sources, annual performance targets, reporting frequency, and base- line and midterm evaluation components; the rigor of this plan reflected PEPFARâs increasing emphasis on M&E (PEPFAR, 2011, 2012) and use of more sophisticated metrics, which had begun in 2009 with the development of PEPFARâs Next Generation Indicators. In contrast to PEPFARâs earlier metrics, which had been largely output measures, the Next Generation Indi- cators sought to measure program coverage and quality using both process and outcome indicators. In October 2014, when CDC issued a noncompetitive funding opportu- nity announcement to continue the Programâs funding at a level of $14 mil- lion per year for 5 years under its own funding mechanism, PEPFARâs M&E requirements for the Program were both comprehensive and rigorous: CDC will work with the awardee to implement, using PEPFAR and national indicators as a base, a robust monitoring and evaluation system designed to track the implementation of HRH efforts, pre- service training, and maintenance of critical health care equipment that have already begun and will continue after [U.S. government] support for these activities concludes. This system will provide evidence needed to drive a programmatic process for decision mak- ing. This will be built on a three-tiered approach that captures data through routine program monitoring, program evaluation, and specific surveys. (CDC, 2014) To fulfill these requirements, the MOH submitted a project narrative and evaluation and performance measurement plan in its award application (MOH, 2014d). Under the new cooperative agreement, CDC required the MOH to revise and submit a more detailed evaluation and performance measurement plan within the first 6 months of the award (CDC, 2014). In 2015, the MOH submitted a revised programmatic/technical work plan that documented âSMART (specific, measurable, achievable, relevant, and time-based) Objectivesâ activities, a time line, responsible parties, and âprocess measuresâ (MOH, 2015b).
98 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA According to the 2014 M&E plan, a point person, trained in the use of data collection tools for routine reporting, would be available for every teaching hospital and health facility (MOH, 2014b). Three main evalua- tions at three time points would also be planned to measure the programâs outcomes and impact, recognizing that a baseline might only be available for some areas because the Program had already started (MOH, 2014b). Most indicator data for program monitoring would be collected semian- nually from the following sources: the Health Management Information System, Human Resources Information System, health education institution records, and program records. A Health Education Institution Survey and Hospital Survey were also planned, although no evidence was found to show that either survey occurred (MOH, 2014b). The HRH Program midterm review reported that quarterly surveys were administered to individuals involved in the Program, although re- sponse rates were low, especially among Rwandan faculty (MOH, 2016b). An organizational case study of the Program concluded that its prioritiza- tion of implementation costs and underestimate of administrative, moni- toring, and evaluation costsâcoupled with the substantial size and scope of the Programâcreated a âmismatch between needs and availability of resources and expertiseâ and a lack of resources for M&E (Cancedda et al., 2018). From the perspective of interview respondents, little investment was made in M&E processes, which had also been noted in the Programâs midterm review. A USI respondent commented that having a single M&E officer for the Program was insufficient: One M&E officer, like 100 U.S. faculty on the ground from 21 different institutions across 5 or 6 physical locations in Rwanda and in 4 major disciplines, and an officer barely had the capacity to figure out what kind of questionnaire to write, much less dis- seminate to collect it, analyze it, and report on it in a way that was receivable by funders, partners, government, and university. Impos- sible. Totally impossible. Total fail. (83, USI Faculty in Surgery) Another respondent stated that there was âa high-level M&E document [referring to the 2014 M&E plan], but itâs not specific to actual activities on a yearly basisâ (22, Non-Government of Rwanda HRH Program Ad- ministrator from an International NGO). SUSTAINABILITY PLANNING Under PEPFARâs partnership frameworks, an important component of PEPFARâs 2.0 strategy (PEPFAR, 2014, 2019b), sustainability was defined
HRH PROGRAM CONTEXT, VISION, AND DESIGN 99 as âsupporting the partner government in growing its capacity to lead, manage, and ultimately finance its health system with indigenous resources (including its civil society sector), rather than external resources, to the greatest extent possibleâ (PEPFAR, 2009). PEPFARâs role was to move from supporting the expansion and provision of direct services to offering techni- cal assistance to partner countries to work toward sustainable programming (PEPFAR, 2019b). This was operationalized through a 5-year Partnership Framework, which aimed to produce results in HIV prevention, care, and treatment, and to position countries âto assume primary responsibility for the national responses to HIV/AIDS in terms of management, strategic direction, performance monitoring, decision making, coordination, and, where possible, financial support and service deliveryâ (PEPFAR, 2009). For the HRH Program, the 2011 proposal referred to sustainability in the context of the intention that, after 8 years, the Government of Rwanda would operate the Program under its own budget. The proposal included a rationale for the availability of the financial resources that would be needed to absorb the newly trained health workers and maintain equip- ment purchased under the HRH Program. It argued that the governmentâs commitment to allocating 15 percent of the total national budget to health, as agreed to in the Abuja Declaration, combined with the projected 5 percent annual increase in the total Rwandan budget due to GDP growth, would generate enough funds to cover the estimated $43 million per year for Rwandan health professionals produced by the HRH Program, $9 million per year for tuition support, $13 million (over several years) for equipment replacement, and $1.5 million per year for equipment main- tenance (MOH, 2011b). The MOHâs 2014 funding application program narrative in response to the funding opportunity announcement articulated sustainability as the programmatic aim of building âsufficient Rwandan educators, infrastructure and equipment, and domestic financing to sup- port Rwandaâs health care and health sciences education without external supportâ (MOH, 2014d). Although it appears that some planning for sustainability took place during the proposal phase, interview respondents expressed variation in how to define and measure HRH Program sustainability, which was linked to how they conceptualized the Programâs goals and objectives, as articu- lated by an administrator at the University of Rwanda: It depends on what you mean by sustainability. If you mean a program that goes on forevermore, having a hundred people com- ing every year â¦ if thatâs your version of sustainability we need to sit down and talk about it. If itâs about building on the gains, thatâs another issue and thatâs something that the university takes very seriously.â¦ In terms of sustainability, do we need the same
100 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA thing forevermore for the next 20 years? Then, no. We would have failed. Do we want input of experts for the expertise that we donât have? Absolutely, you know! Sustainability is about us, being able to do by ourselves, better than it is done anywhere in the world, and thatâs what we are striving to do at the University of Rwanda all the time. (02, University of Rwanda Administrator) Among respondents who believed the intent was to build capacity within the University of Rwanda to continually produce high-quality health workers, sustainability was seen as built into the design: [T]he goal was to mentor teachers and train faculty. So, in the design, the program is already sustainable. (43, Government of Rwanda HRH Program Administrator) One respondent explained that the design included a gradual phasing out of USI faculty and a simultaneous increase in Rwandan faculty: The program was implemented with the target that after 7 years the program will end.â¦ [T]he sustainability plan was kind of [a] mechanism of decreasing gradually the U.S. faculty, while increas- ing retained and available local faculty. That is one thing. On the other side, the [University of Rwanda] had in their planning to have to increase the college faculty so the local Rwandan faculty in the university funded for 2 years, I think that was the initial plan for the 2 last years of the program, with the program funding while negotiating to be integrated in the structure of the university. I think that is how the sustainability plan was thought. (20, Gov- ernment of Rwanda HRH Program Administrator) In contrast, for another respondent, who had expressed that the objec- tive of the HRH Program was to build capacity of Rwandan health edu- cation institutions, most directly the University of Rwanda, the Program failed to clearly understand and build on the capacity of local institutions: [L]ocal institutions were â¦ involved, but there was nothing really done to understand the local institutionsâ capacity, strength, readi- ness, levels, interest, priorities, process. And nothing was really done to understand how we could integrate successfully this faculty and effectively in the local institutions, you know, how they were going to support the leadership, how they were going to comple- ment and not replace local faculty in the way that built on synergies
HRH PROGRAM CONTEXT, VISION, AND DESIGN 101 and not mix with redundancies which happened a lot, I think. (22, Non-Government of Rwanda HRH Program Administrator) Respondents external to the government were skeptical of the sustain- ability and long-term planning that was done in the design phase: I think the attitude of a lot of Rwandan institutions, departments, was that âOk, we will take the support and enjoy it while itâs there. It is really helpful; it is filling a gap.â And then one day it ends and thereâs something else coming in. And thatâs fine, but not re- ally long-term planning. (22, Non-Government of Rwanda HRH Program Administrator) CONCLUSIONS The HRH Program represented a confluence of unusual circumstances and an opportunity in foreign assistance. Its design endorsed Rwandaâs larger vision of strengthening the countryâs workforce, including in the health sector, although there were missed opportunities to learn systemati- cally. PEPFAR 2.0 marked âa new era of collaborative planning and health systems strengthening activities with â¦ partner governmentsâ (PEPFAR, 2014). PEPFARâs investments in the HRH Program were a critical contribu- tion to launching an Academic Consortium that leveraged (1) existing and new partnerships with individual USIs, (2) the political commitment of a government with record of successfully managing health systems programs, and (3) a holistic PEPFAR strategy that emphasized HSS. PEPFARâs investments in the HRH Program in Rwanda were relatively unique, departing from U.S government and many other big donorsâ mod- els for âbusiness as usual,â but aligned with prevailing principles estab- lished by the Busan Partnership for Effective Development Co-operation, in which funding partners would support shared goals according to specific country situations (Fourth High-Level Forum on Aid Effectiveness, 2011). PEPFAR investments have continuously supported programs and activities implemented in partner countries and, consistent with PEPFARâs strategy to move toward sustainability, have directed their funding over time to lo- cal prime partners, although the proportion of funding to partner country governments has been relatively stable (IOM, 2013). From the outset, Rwanda expressed a strong commitment to national ownership, as articulated in Vision 2020, and sought to increase the direct management of externally funded health and HIV programs. To move toward this vision, the Government of Rwanda and its PEPFAR-funded partners promoted a transition plan that shifted HIV program leadership from partners to the host country, including direct PEPFAR financing to the
102 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA government (Government of Rwanda, 2012). The management transition took place by February 2012, when the MOH became a direct recipient of PEPFAR funds and gained responsibility for coordinating critical service delivery (Binagwaho et al., 2016). These investments challenged the con- ventional model, and used country ownership and shared responsibility as a new paradigm when the HRH Program commenced in 2012 (Goosby et al., 2012a,b). The investments in the HRH Program also were an opportunity to add to the evidence around twinning because the Programâs Academic Consor- tium was designed not only as a departure from short-term faculty stints in Rwanda, but also to completely phase out by the end of the Program as the âtwinsâ and new graduates gained skills to serve as local faculty. The twinning arrangements were between individual USI faculty and University of Rwanda faculty, not between the two institutions. The implications of this arrangement, discussed more fully in subsequent chapters, were felt in terms of sustained capacity at the institutional level to provide ongoing health professional education. There was concurrence among HRH Program participants on a high- level vision and intent, which aligned with broader health-sector goals. However, there was a lack of clarity around the mechanisms and pathway for achieving a world-class health care system, the consequences of which were felt during the implementation of the Program, as discussed through- out subsequent chapters. The relative importance of building health educa- tion capacity, including enhancing the skills of Rwandan faculty and the learning environment at the University of Rwanda, was inconsistent during the life of the Program. In particular, it was unclear whether activities for building institutional capacity for health professional education were pri- oritized simply as a mechanism to produce the desired number of health professionals across cadres, or viewed as complementary goals to having a sustainable, skilled, and specialized workforce, as expressed in the later strategic outcomes (MOH, 2011b, 2014b, 2015b). The downstream effects of the shifting language in MOH documents was a lack of clarity across stakeholders around the Programâs goals and objectives, which had implications for the design and for sustainability plan- ning and were further compounded by differing (and sometimes conflict- ing or restrictive) administrative practices across participating institutions (Government of Rwanda, USIs, and PEPFAR). Planning for sustainability evolved from accounting for funding of the established training activities under the HRH Program after 2019 to more holistically ensuring the qual- ity of health professional education and service delivery. Although the Government of Rwanda recognized the unprecedented scale and nature of the HRH Programâs strategies, there was insufficient planning to exploit the opportunity to learn systematically from this en-
HRH PROGRAM CONTEXT, VISION, AND DESIGN 103 deavor by establishing rigorous M&E processes and supportive mecha- nisms at the outset (MOH, 2011b, 2014b, 2016b). The original program objectives were incorporated into various indicators for the M&E plan and midterm review; however, the development of an M&E plan after imple- mentation had begun resulted in a lack of baseline data for some cadres and program areas, as well as unexplained differences in baseline and target values (MOH, 2011b, 2014b, 2016b). Even as the post hoc M&E plan articulated an intention to âprovide a reliable data-driven approach â¦ to ensure accountability, program improvement, impact, and learning,â there was lack of follow-through by the Government of Rwanda in providing dedicated resources to establish an M&E platform (e.g., M&E working groups were composed of volunteer HRH Program participants, as opposed to designated roles), to carry out planned assessments, and to build overall capacity for monitoring, evaluation, and learning (MOH, 2014b). Overall management of the HRH Program was challenged by the lack of time for operational management, both at the outset of implementation and continuously, as unexpected circumstances arose. More time between the design, launch, and execution phases would have supported stakehold- ersâ ability to better anticipate and develop contingent strategies for issues such as PEPFAR funding processes and restrictions. REFERENCES Binagwaho, A., P. Kyamanywa, P. E. Farmer, T. Nuthulaganti, B. Umubyeyi, J. P. Nyemazi, S. D. Mugeni, A. Asiimwe, U. Ndagijimana, H. Lamphere McPherson, D. Ngirabega Jde, A. Sliney, A. Uwayezu, V. Rusanganwa, C. M. Wagner, C. T. Nutt, M. Eldon- Edington, C. Cancedda, I. C. Magaziner, and E. Goosby. 2013. The Human Resources for Health Program in Rwandaânew partnership. New England Journal of Medicine 369(21):2054â2059. Binagwaho, A., P. E. Farmer, S. Nsanzimana, C. Karema, M. Gasana, J. De Dieu Ngira- bega, F. Ngabo, C. M. Wagner, C. T. Nutt, T. Nyatanyi, M. Gatera, Y. Kayiteshonga, C. Mugeni, P. Mugwaneza, J. Shema, P. Uwaliraye, E. Gaju, M. A. Muhimpundu, T. Dushime, F. Senyana, J. B. Mazarati, C. M. Gaju, L. Tuyisenge, V. Mutabazi, P. Kyamanywa, V. Rusanganwa, J. P. Nyemazi, A. Umutoni, I. Kankindi, C. Ntizimira, H. Ruton, N. Mugume, D. Nkunda, E. Ndenga, J. M. Mubiligi, J. B. Kakoma, E. Karita, C. Sekabaraga, E. Rusingiza, M. L. Rich, J. S. Mukherjee, J. Rhatigan, C. Cancedda, D. Bertrand-Farmer, G. Bukhman, S. N. Stulac, N. M. Tapela, C. Van Der Hoof Holstein, L. N. Shulman, A. Habinshuti, M. H. Bonds, M. S. Wilkes, C. Lu, M. C. Smith-Fawzi, J. D. Swain, M. P. Murphy, A. Ricks, V. B. Kerry, B. P. Bush, R. W. Siegler, C. S. Stern, A. Sliney, T. Nuthulaganti, I. Karangwa, E. Pegurri, O. Dahl, and P. C. Drobac. 2014. Rwanda 20 years on: Investing in life. Lancet 384(9940):371â375. Binagwaho, A., I. Kankindi, E. Kayirangwa, J. P. Nyemazi, S. Nsanzimana, F. Morales, R. Kadende-Kaiser, K. W. Scott, V. Mugisha, R. Sahabo, C. Baribwira, L. Isanhart, A. Asiimwe, W. M. El-Sadr, and P. L. Raghunathan. 2016. Transitioning to country ownership of HIV programs in Rwanda. PLoS Medicine 13(8):e1002075.
104 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA Cancedda, C., P. Cotton, J. Shema, S. Rulisa, R. Riviello, L. V. Adams, P. E. Farmer, J. N. Kagwiza, P. Kyamanywa, D. Mukamana, C. Mumena, D. K. Tumusiime, L. Mukashyaka, E. Ndenga, T. Twagirumugabe, K. B. Mukara, V. Dusabejambo, T. D. Walker, E. Nkusi, L. Bazzett-Matabele, A. Butera, B. Rugwizangoga, J. C. Kabayiza, S. Kanyandekwe, L. Kalisa, F. Ntirenganya, J. Dixson, T. Rogo, N. McCall, M. Corden, R. Wong, M. Mukeshimana, A. Gatarayiha, E. K. Ntagungira, A. Yaman, J. Musabey- ezu, A. Sliney, T. Nuthulaganti, M. Kernan, P. Okwi, J. Rhatigan, J. Barrow, K. Wilson, A. C. Levine, R. Reece, M. Koster, R. T. Moresky, J. E. OâFlaherty, P. E. Palumbo, R. Ginwalla, C. A. Binanay, N. Thielman, M. Relf, R. Wright, M. Hill, D. Chyun, R. T. Klar, L. L. McCreary, T. L. Hughes, M. Moen, V. Meeks, B. Barrows, M. E. Durieux, C. D. McClain, A. Bunts, F. J. Calland, B. Hedt-Gauthier, D. Milner, G. Raviola, S. E. Smith, M. Tuteja, U. Magriples, A. Rastegar, L. Arnold, I. Maga- ziner, and A. Binagwaho. 2018. Health professional training and capacity strengthening through international academic partnerships: The first five years of the Human Resources for Health Program in Rwanda. International Journal of Health Policy and Management 7(11):1024â1039. CDC (U.S. Centers for Disease Control and Prevention). 2009a. Funding opportunity an- nouncement: Clinical services 5-year proposed work plan. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/ pa/managerequest.aspx?key=HMD-BGH-17-08.) CDC. 2009b. Funding opportunity announcement: Strengthening the Ministry of Healthâs capacity to respond to the HIV/AIDS epidemic in the Republic of Rwanda under PEP- FAR. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https:// www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) CDC. 2009c. U.S. Department of Health and Human Services Centers for Disease Control and Prevention strengthening the Ministry of Healthâs capacity to respond to the HIV/ AIDS epidemic in the Republic of Rwanda under the Presidentâs Emergency Plan for AIDS Relief (PEPFAR) funding opportunity. Atlanta, GA: U.S. Centers for Disease Control and Prevention. CDC. 2012. Request approval to award a multi-year expansion supplement exceeding 25% for the cooperative agreement PS09-9118, 3U2GPS2091 âStrengthening the Ministry of Healthâs Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the Presidentâs Emergency Plan for AIDS Relief (PEPFAR)â and approval to waive the requirement of an objective review for the single awardee to this cooperative agreement. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) CDC. 2014. Funding opportunity announcement: Strengthening human resources for health capacity in the Republic of Rwanda under the Presidentâs Emergency Plan for AIDS Relief (PEPFAR) CDC-RFA-GH15-1568. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https://www8.nationalacademies.org/pa/ managerequest.aspx?key=HMD-BGH-17-08.) FlinkenflÃ¶gel, M., P. Kyamanywa, B. Asiimwe-Kateera, S. Musafiri, P. Kayumba, M. Irakoze, P. Cotton, and A. Hibble. 2015. Umuganda for improved health professions education in Rwanda: Past, present and future in the training of health professionals at the University of Rwanda. Rwanda Journal 2:96.
HRH PROGRAM CONTEXT, VISION, AND DESIGN 105 Fourth High-Level Forum on Aid Effectiveness. 2011. Busan partnership for effective devel- opment co-operation: Fourth High-Level Forum on Aid Effectiveness, Busan, Republic of Korea, 29 Novemberâ1 December 2011. Paris, France: Organisation for Economic Co-operation and Development. Gavi (Gavi, the Vaccine Alliance). 2018. Republic of Rwanda: Programme audit of Gavi sup- port to the Ministry of Health. Geneva, Switzerland: Gavi, the Vaccine Alliance. Global Fund (Global Fund to Fight AIDS, Tuberculosis and Malaria). 2014. Audit report: Audit of global fund grants to Rwanda. Geneva, Switzerland: Global Fund. Global Fund. 2019. Audit follow up report: Global fund grants to Rwanda. Geneva, Swit- zerland: Global Fund. Goosby, E., M. Dybul, A. S. Fauci, J. Fu, T. Walsh, R. Needle, and P. Bouey. 2012a. The United States Presidentâs Emergency Plan for AIDS Relief: A story of partnerships and smart investments to turn the tide of the global AIDS pandemic. Journal of Acquired Immune Deficiency Syndrome 60(Suppl 3):S51âS56. Goosby, E., D. Von Zinkernagel, C. Holmes, D. Haroz, and T. Walsh. 2012b. Raising the bar: PEPFAR and new paradigms for global health. Journal of Acquired Immune Deficiency Syndrome 60(Suppl 3):S158âS162. Government of Rwanda. 2012. Vision 2020, revised 2012. Kigali, Rwanda: Government of Rwanda. IOM (Institute of Medicine). 2013. Evaluation of PEPFAR. Washington, DC: The National Academies Press. Kayirangwa, E., J. Hanson, L. Munyakazi, and A. Kabeja. 2006. Current trends in Rwandaâs HIV/AIDS epidemic. Sexually Transmitted Infection 82(Suppl 1):i27âi31. MOH (Ministry of Health). 2005a. Health sector policy, February 2005. Kigali, Rwanda: Ministry of Health. MOH. 2005b. Health Sector Strategic Plan, 2005â2009. Kigali, Rwanda: Ministry of Health. MOH. 2009a. Health Sector Strategic Plan, July 2009âJune 2012. Kigali, Rwanda: Ministry of Health. MOH. 2009b. National Strategic Plan on HIV and AIDS 2009â2012. Kigali, Rwanda: Min- istry of Health. MOH. 2010. Clinical services work plan. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2011a. Human Resources for Health Strategic Plan, 2011â2016. Kigali, Rwanda: Ministry of Health. MOH. 2011b. Rwanda Human Resources for Health Program, 2011â2019: Funding pro- posal. Kigali, Rwanda: Ministry of Health. MOH. 2011c. U.S. academic institutionsâselection rationale. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https://www8.nationalacademies.org/pa/managerequest. aspx?key=HMD-BGH-17-08.) MOH. 2012a. HRH application brief cabinet paper, May 2012â30 Sept 2013. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/ managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2012b. HRH U.S. Academic Consortium budget years 1 & 2 Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https://www8.nationalacademies.org/pa/managerequest. aspx?key=HMD-BGH-17-08.)
106 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA MOH. 2012c. Rwanda health resource tracker output report for FY 2009â10 exp. & FY 2010â11 budget & FY 2010â11 exp. & FY 2011â12 budget. Kigali, Rwanda: Ministry of Health. MOH. 2012d. Third Health Sector Strategic Plan, July 2012âJune 2018. Kigali, Rwanda: Ministry of Health. MOH. 2012e. Work plan. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8. nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2013a. CDC-RFA-PS09-9118-05CONT13: Strengthening the Capacity of the Minis- try of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the Presidentâs Emergency Plan for AIDS Relief (PEPFAR). Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https://www8.nationalacademies.org/pa/managerequest. aspx?key=HMD-BGH-17-08.) MOH. 2013b. Rwanda health resource tracker output report December 2013: Expenditures FY 2011/12 and budget FY 2012/13. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2014a. Annual report on the implementation of CDC-funded projects, period: October 2012-September 2013. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https:// www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2014b. Human Resources for Health monitoring & evaluation plan, March 2014. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/ pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2014c. National Human Resources for Health Policy, October 2014. Kigali, Rwanda: Ministry of Health. MOH. 2014d. Project narrative. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https:// www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2015a. Health sector policy, January 2015. Kigali, Rwanda: Ministry of Health. MOH. 2015b. Programmatic/technical work plan. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Of- fice [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/managerequest. aspx?key=HMD-BGH-17-08.) MOH. 2016a. Rwanda health resource tracking output report: Expenditure FY 2013/14 and budget FY 2014/15. Kigali, Rwanda: Ministry of Health. MOH. 2016b. Rwanda Human Resources for Health Program midterm review report (2012â2016). Kigali, Rwanda: Ministry of Health. (Available by request from the Na- tional Academies Public Access Records Office [email@example.com] or via https://www8. nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.) MOH. 2018a. Fourth Health Sector Strategic Plan, July 2018âJune 2024. Kigali, Rwanda: Ministry of Health. MOH. 2018b. Rwanda health resource tracking output report: Expenditure FY 2014/15 and budget FY 2015/16. Kigali, Rwanda: Ministry of Health. MOH. 2018c. Rwanda HIV and AIDS National Strategic Plan 2013â2018: Extension: 2018â2020. Kigali, Rwanda: Ministry of Health. Mtiro, E. 2018. Termination of Rwanda cooperative agreement. Kigali, Rwanda. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)
HRH PROGRAM CONTEXT, VISION, AND DESIGN 107 Nsanzimana, S., K. Prabhu, H. McDermott, E. Karita, J. I. Forrest, P. Drobac, P. Farmer, E. J. Mills, and A. Binagwaho. 2015. Improving health outcomes through concurrent HIV program scale-up and health system development in Rwanda: 20Â years of experi- ence. BMC Medicine 13(1):216. PEPFAR (Presidentâs Emergency Plan for AIDS Relief). 2009. Guidance for PEPFAR part- nership frameworks and partnership framework implementation plans. Version 2.0, September 14, 2009. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEFPAR. 2010. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2011. FY 2012 country operational plan (COP) guidance. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2012. FY 2013 country operational plan (COP) guidance. Version 2. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2014. PEPFAR 3.0: Controlling the epidemic; delivering on the promise of an AIDS-free generation. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2019a. Executive summary of PEPFARâs strategy. https://www.pepfar.gov/about/ strategy/document/133244.htm (accessed February 24, 2019). PEPFAR. 2019b. PEPFARâs five-year strategy. https://www.pepfar.gov/about/strategy/ document/133251.htm (accessed February 21, 2019). PEPFAR. 2019c. PEPFAR panorama spotlight. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR. 2019d. PEPFAR Panorama spotlight glossary. https://data.pepfar.gov/glossary (ac- cessed October 11, 2019). PEPFAR Rwanda. 2008. Populated printable COP without TBD partners 2008 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2009. Populated printable COP 2009 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2010. Rwanda operational plan report FY 2010. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2011. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2012. Rwanda country operational plan FY 2012. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2013. Rwanda operational plan report FY 2013. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2014. Rwanda operational plan report FY 2014. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. PEPFAR Rwanda. 2015. Rwanda country operational plan (COP) 2015 strategic direction summary. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy. Seymour, B., I. Muhumuza, C. Mumena, M. Isyagi, J. Barrow, and V. Meeks. 2013. Includ- ing oral health training in a health system strengthening program in Rwanda. Global Health Action 6:1â6. TAGGS (Tracking Accountability in Government Grants System). 2019. Ministry of Health Rwanda. https://taggs.hhs.gov (accessed July 9, 2019). Tvedten, I., A. Byabagambi, J. Lindstrom, and M. Tedre. 2018. Evaluation of the SIDA- supported research capacity and higher education development program in Rwanda, 2013â2017. Stockholm, Sweden: Swedish International Development Cooperation Agency.
108 EVALUATION OF PEPFARâS CONTRIBUTION TO RWANDA USAID (United States Agency for International Development). 1996. Rebuilding postwar Rwanda: The role of the international community. Washington, DC: United States Agency for International Development. Uwizeye, G., D. Mukamana, M. Relf, W. Rosa, M. J. Kim, P. Uwimana, H. Ewing, P. Munyiginya, R. Pyburn, N. Lubimbi, A. Collins, I. Soule, K. Burke, J. Niyokindi, and P. Moreland. 2018. Building nursing and midwifery capacity through Rwandaâs Human Resources for Health Program. Journal of Transcultural Nursing 29(2):192â201. WHO (World Health Organization). 2016. Health workforce requirements for universal health coverage and the Sustainable Development Goals. Geneva, Switzerland: World Health Organization. WHO. 2019. Global health expenditure database. Geneva, Switzerland: World Health Organization. Willis, B. M., and B. S. Levy. 2000. Recognizing the public health impact of genocide. JAMA 284(5):612â614. World Bank. 2019a. Current health expenditure per capita, PPP (current international $)â Rwanda. https://data.worldbank.org/indicator/SH.XPD.CHEX.PP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019b. Domestic general government health expenditure per capita, PPP (cur- rent international $)âRwanda. https://data.worldbank.org/indicator/SH.XPD.GHED. PP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019c. GDP (current US$)âRwanda. https://data.worldbank.org/indicator/ NY.GDP.MKTP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019d. GDP per capita, PPP (current international $)âRwanda. https://data. worldbank.org/indicator/NY.GDP.PCAP.PP.CD?locations=RW (accessed November 4, 2019). World Bank. 2019e. General government final consumption expenditure (current US$). https:// data.worldbank.org/indicator/NE.CON.GOVT.CD (accessed November 4, 2019). World Bank. 2019f. Inflation, consumer prices (annual %)âRwanda. https://data.world- bank.org/indicator/FP.CPI.TOTL.ZG?end=2018&locations=RW&start=2000 (accessed November 4, 2019). World Bank. 2019g. Official exchange rate (LCU per US$, period average). https://data. worldbank.org/indicator/PA.NUS.FCRF?locations=RW (accessed November 4, 2019). World Bank. 2019h. PPP conversion factor, GDP (LCU per international $)âRwanda. https:// data.worldbank.org/indicator/PA.NUS.PPP?locations=RW (accessed November 4, 2019).