3

Human Resources for Health Program Context, Vision, and Design

CONTEXT IN RWANDA LEADING UP TO THE PROGRAM

The Costs of Conflict

Rwanda’s recent history of conflict is essential context for the Human Resources for Health (HRH) Program’s origin and implementation progress. The 1994 genocide against the Tutsi in Rwanda resulted in an estimated 1 million deaths and 2 million displaced people. All social services

were devastated, bringing to a halt all foreign assistance programs related to health, education, agriculture, and other economic development efforts. The toll on the health system included physical destruction of hospitals, laboratories, and equipment, not to mention the death or displacement of more than 80 percent of the country’s health professionals (USAID, 1996). By 1995, fewer than 10 pediatricians were reported to have been practicing, and there were no trained medical personnel, such as psychiatrists or trauma surgeons, to address issues emerging from the conflict (Binagwaho et al., 2014; Nsanzimana et al., 2015). The time and other investments required to counter the long-term public health effects of damaged social institutions were evident in the fact that it took 6 years for the first class of medical students to graduate (Willis and Levy, 2000).

During and immediately following the genocide against the Tutsi, international organizations prioritized restoring primary health care services and water and sanitation systems (USAID, 1996). The International Committee of the Red Cross, United Nations Children’s Fund (UNICEF), and nongovernmental organizations (NGOs) such as Médecins Sans Frontières focused on repairing and operationalizing clinics with emergency health kits, medicines, supplies, and staff to provide on-the-job training to auxiliary health workers (USAID, 1996). Immunization service delivery was disrupted, but the Ministry of Health (MOH) received international support to restore vaccine stocks and function to its immunization supply chain and logistics system (USAID, 1996). However, other capacities to detect and control infectious diseases, particularly the spread of HIV and cholera from migration and refugee camps at Rwanda’s border, would remain a challenge for years (Binagwaho et al., 2014; Nsanzimana et al., 2015).

Although donors and NGOs were essential for providing necessary emergency relief, these agencies did not fully coordinate with the Rwandan government in the transition from emergency operations to rehabilitation of the health system. Consequently, health care services were fragmented, and the MOH was slower to build institutional capacity to manage the country’s health priorities and sustain a health care delivery system (USAID, 1996).

Major Developments in Rwanda’s Health-Sector Planning

During the recovery period following the genocide against the Tutsi, the Government of Rwanda has consistently planned for how the health sector could better the health status of the population and support national poverty reduction goals. The release of the Health Sector Policy in 2005 outlined an overhaul of the sector, in light of decentralization efforts under way and in pursuit of more significant health gains (MOH, 2005a). Seven intervention priorities were established in seven areas:

1. Availability of human resources;
2. Availability of high-quality drugs, vaccines, and consumables;
3. Geographical access to health services;
4. Financial access to health services;
5. Quality of and demand for services in disease control;
6. Strengthening national referral hospitals and treatment centers; and
7. Strengthening the health sector’s institutional capacity.

In particular, the Government of Rwanda would support financial access to health services by increasing public funding of health services (MOH, 2005a).

The government strategy around this policy has evolved over time. The first Health Sector Strategic Plan (2005–2009) laid out plans for achieving the health-related Millennium Development Goals (MDGs), with attention to getting maternal and child mortality on track (MOH, 2005b). The following iteration of the Health Sector Strategic Plan (2009–2012) highlighted health systems strengthening in each of the seven intervention areas and continued to emphasize increased financial access to health services, noting the scale-up of community-based health insurance, as well as increased quality of care promoted by performance-based financing (MOH, 2009a). The World Health Organization (WHO) framework of health systems building blocks served as a foundation for the third Health Sector Strategic Plan (2012–2018), in which the MOH started to orient toward objectives in the post-MDG era and increasingly considered health resource management and governance mechanisms (MOH, 2012d). The resulting Health Sector Policy of 2015 touted key health achievements, including improved maternal and child health, increased community health worker coverage, and antiretroviral therapy (ART) and malaria program successes since 2004 (MOH, 2015a). However, it also outlined policies to address challenges in support of vulnerable and marginalized populations, sustainable health system financing, and noncommunicable disease prevention and control, taking into account Rwanda’s epidemiologic transition and socioeconomic progress (MOH, 2015a). The current Health Sector Strategic Plan (2018–2024) centers on the Sustainable Development Goals and fully acknowledges these challenges and the need to reorganize the health system and involve other sectors of development to ensure universal health coverage for all needed services at all stages of life (MOH, 2018a).

HIV in Rwanda

The first case of HIV in Rwanda was documented in 1983; the Government of Rwanda initiated its response in 1985 with a blood donor screening program and has since sustained efforts to address the HIV epidemic

(Kayirangwa et al., 2006). In 2009, Rwanda published its first National Strategic Plan on HIV/AIDS (MOH, 2009b). The development of national strategic plans on HIV/AIDS, the decentralization of the Rwandan health system, and the movement toward community-based health insurance and performance-based financing facilitated its key achievements and remarkable progress toward achieving HIV epidemic control (MOH, 2009a,b, 2018c). Rwanda has also made steady improvements in increasing access to and coverage of ART over the past decade. The government’s commitment to confronting its HIV epidemic has accelerated progress toward the Joint United Nations Programme on HIV/AIDS’ (UNAIDS’) 90-90-90 targets prior to the HRH Program, as discussed in more detail in Chapter 7.

HRH in Rwanda

Despite improvements since the genocide against the Tutsi nearly destroyed the health infrastructure and resulted in acute health workforce shortages, which hindered health service delivery and served as a major barrier to HIV care and treatment, Rwanda continues to fall far below WHO’s recommended critical minimum threshold of 4.45 doctors, nurses, and midwives per 1,000 people (WHO, 2016). This shortfall comes from an insufficient number of trained health professionals relative to the need. Addressing HRH capacity had been featured in prior health-sector planning, but specific direction to develop a “competent, dedicated, productive, and accessible workforce” in support of the MOH’s mission of “providing quality preventive, curative, rehabilitative, and promotional services” was not articulated until the National Human Resources for Health Policy in 2014 (MOH, 2014c).

These guidelines were developed for the planning, management, use, and monitoring of health-sector resources to operationalize the National HRH Strategic Plan (2011–2016) (MOH, 2011a), building on Rwanda’s Health Systems Strengthening Framework and Consolidated Plan 2009–2012 (MOH, 2009a). Notably, the National HRH Strategic Plan called for the development of a clear health service delivery plan, delineating specific competencies for each cadre providing services at each level of care, more emphasis on the quality of trained professionals and their distribution, and demand for reliable data to inform health resource management and evaluation of health system effectiveness (MOH, 2011a).

Economic and Financial Context for Health in Rwanda

As Rwanda recovered and rebuilt from the genocide against the Tutsi, its economy eventually also started to experience considerable growth. Figure 3-1 shows the trajectory of growth from the early 2000s, as illustrated

by gross domestic product (GDP), the total national expenditure, and the portion of that national expenditure that is government spending. The HRH Program was situated in this period of economic growth.

Concurrently, there was growth in spending on health in Rwanda. Figure 3-2 shows the trajectories of growth in both per capita total health expenditures and per capita government spending on health, alongside the growth in per capita GDP.

In this same time period, as Figure 3-3 shows, close to half of the health expenditure in Rwanda has consistently come from external aid, ranging from 41 percent to 54 percent between 2000 and 2016. Spending specifically for Rwanda’s response to HIV increased drastically during this period, and as of 2014 it has been consistently estimated at about $200 million annually. Of this, 80 percent comes from external funders, a larger proportion than for health overall (Nsanzimana et al., 2015).

PEPFAR Funding Context

Before the HRH Program was launched in 2012, the President’s Emergency Plan for AIDS Relief (PEPFAR) had an established funding history in Rwanda going back to 2003 (Binagwaho et al., 2016). Figure 3-4 shows PEPFAR total annual planned funding for programs in Rwanda from fiscal year (FY) 2009 to FY 2019, including the years surrounding the HRH Program.¹ Amounts shown in the bar graph were PEPFAR allocations to treat-

___________________

¹ This graph was generated from publicly available PEPFAR planned funding data, downloaded June 16, 2019, from PEPFAR’s Panorama Spotlight website. Planned funding information was used because publicly available expenditure information was less granular and was not available from the start of the HRH Program.

ment, testing, prevention, health systems strengthening (HSS), government and systems, care, and applied pipeline. Percentages indicate proportion of annual funding allocated for HSS over time (PEPFAR, 2019c).

PEPFAR’s total planned funding for its Rwanda program portfolio declined overall from 2009 to 2019. The proportion of funding allocated to HSS increased during the period between FY 2013 and FY 2016 relative to surrounding years, with a peak in FY 2013. This period is concurrent with the timing of financial support for the HRH Program. Before FY 2013, PEPFAR planned funding in Rwanda for HSS had not exceeded $9 million and had remained under 10 percent of total planned funding. Between FY 2013 and FY 2016, PEPFAR funding for HSS activities ranged from $24.7 million in FY 2013 (27 percent of its total portfolio) to $11.8 million (17 percent of its portfolio) in FY 2016. Since FY 2016, funding for HSS activities in the portfolio has not exceeded $1.9 million, or 2 percent of total planned funding. In the same period between FY 2013 and FY 2016, there was a decrease in the proportion of funding allocated to other program areas. In contrast, the proportion of PEPFAR planned funding devoted to HSS activities between FY 2009 and FY 2019 in other East African Community countries has been less variable and has not exceeded 13 percent of the total portfolio.²

PEPFAR Investments in the HRH Program

At the inception and during the first 3 years of its funding, PEPFAR considered the HRH Program to be a strategic or key priority within its overall portfolio (PEPFAR Rwanda, 2012, 2013, 2014). Initially, PEPFAR shifted resources from other activities to “transfer significant support to the area of HRH” (PEPFAR Rwanda, 2012) and thus fund the Program (PEPFAR Rwanda, 2012, 2013). Subsequently, the Program’s placement relative to other PEPFAR programmatic priorities decreased over time (PEPFAR Rwanda, 2013, 2014, 2015). The HRH Program was being funded during the window of peak PEPFAR funding for HSS in Rwanda, shown in Figure 3-4.

Between FY 2011 and FY 2013, the Program was funded as a component of an existing U.S. Centers for Disease Control and Prevention (CDC) cooperative agreement with the MOH called “Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under PEPFAR” (also known as the Clinical Services

___________________

² The other East African Community countries include Kenya, Tanzania, and Uganda. In Kenya, PEPFAR-planned funding for HSS activities has ranged from 1 percent to 3 percent of total planned funding between FY 2009 and FY 2019. In Uganda, funding for these activities has constituted between 2 percent and 5 percent of the PEPFAR portfolio, and in Tanzania, between 3 percent and 13 percent of the PEPFAR portfolio during this time. South Sudan and Burundi have smaller total PEPFAR portfolios, averaging less than $16 million per year.

Cooperative Agreement) (Mtiro, 2018). Preceding the HRH Program, CDC had already been providing direct support to the MOH for its HIV/AIDS-related programs (CDC, 2009c, 2012):

• The integration of HIV services and programs into the health system at all levels;
• Capacity building for infection control;
• Injection safety;
• Medical waste management;
• Epidemiological investigation, lab management, pharmaceutical management, health communication, electronic medical record use, and data collection and analysis;
• Use of terminology standards and registries; and
• Training for physicians, community health workers, and service providers in clinical handling and management of sexual and gender-based violence.

This funding also provided salary support and benefits for physicians, technicians, and data managers, as well as supporting quality improvement initiatives, performance-based financing evaluations, technical meetings with stakeholders, and management and audits of MOH/PEPFAR-supported facilities and sites (MOH, 2014a).

The United States Agency for International Development (USAID) also provided some funds for the Program, although the amounts could not be confirmed (CDC, 2012; PEPFAR Rwanda, 2013). In addition to PEPFAR funding, the Program had a diversified funding base, evidenced by the engagement of the Global Fund to Fight AIDS, Tuberculosis and Malaria.³

Starting in FY 2014, CDC used a separate cooperative agreement with the MOH, “Strengthening Human Resources for Health Capacity in the Republic of Rwanda under PEPFAR,” to fund the HRH Program (Mtiro, 2018). In 2015, PEPFAR’s priorities shifted, and the HRH Program was classified as a “noncore” investment as part of an analysis in alignment with PEPFAR’s new 3.0 strategy. The decision was made to end funding for the Program on March 30, 2017, instead of continuing PEPFAR support through June 30, 2019, the Program’s official closing date (Mtiro, 2018; PEPFAR Rwanda, 2015).

Public reporting systems provided limited information about the actual amount invested by PEPFAR in the HRH Program. Table 3-1 shows

___________________

³ The Global Fund specifically sponsored the launch of Rwanda’s first dental school and development of dental curricula and partnerships (Seymour et al., 2013), as PEPFAR did not include dentistry in the list of specialties it would support. Examination of the dentistry school and program was not included in this evaluation.

TABLE 3-1 PEPFAR Funding Sources That Contributed to the HRH Program

NOTES: N/A = not applicable (funding mechanism not used to contribute to the HRH Program that year). All amounts are in U.S. dollars.

* Annual total disbursed funding by award/mechanism as reported through the U.S. HHS Tracking Accountability in Government Grants System (TAGGS) (accessed July 7, 2019). TAGGS reports the total amount disbursed through such agreements, but does not provide any further breakdown by program or activity. Therefore, the proportion of the amount that was specific to the HRH Program is not known in FY 2012 to FY 2014, when the HRH Program was funded as a component of the cooperative agreement, “Strengthening Capacity to Respond to the HIV/AIDS Epidemic,” that started in FY 2009. Starting in FY 2015, the HRH Program was funded through its own cooperative agreement, “Strengthening Human Resources for Health Capacity in the Republic of Rwanda Under PEPFAR,” and the amounts reported in TAGGS are reflective of the amounts disbursed for the HRH Program: $11 million in FY 2015 and $10.5 million in FY 2016. Any amounts disbursed through CDC after FY 2016 are also not known.

** Information is not publicly available on whether amounts were disbursed through this or any other CDC award or mechanism after FY 2016.

*** Although there is documentation that some PEPFAR investments in the HRH Program came from other implementing agencies, those amounts and sources are not reported publicly. Approval of USAID maternal and child health funds to be used for implementing the HRH Program is referenced in a 2012 CDC memo requesting a multiyear expansion supplement for the “Strengthening the Ministry of Health’s Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda Under PEPFAR” cooperative agreement. However, amounts were redacted, and data on the portion of the award/mechanism allocated to the HRH Program were not publicly available (correspondence provided by CDC Rwanda in August 2018). In addition, Rwanda’s Country Operational Plan FY 2013 referenced a planned shift of $2 million from USAID’s Family Health Project to the HRH Program, but the actual amount disbursed and allocated for the HRH Program is not publicly available.

SOURCES: CDC, 2012; PEPFAR Rwanda, 2014; TAGGS, 2019.

available information regarding the mechanisms PEPFAR used to provide funding for the Program.

HRH Program Budget and Ministry of Health Expenditures on Health

The HRH Program budget proposed in 2011 was $151.8 million over 8 years (MOH, 2011b). The Program received external funding from the U.S. government, through PEPFAR, and from the Global Fund. The total funding amount from these external sources, as provided by the MOH, was just under $100 million, with approximately 60 percent coming from the U.S. government (see Table 3-2). This comprised 12 percent to 16 percent of the total annual PEPFAR investment in Rwanda. The amounts disbursed from CDC to the MOH, however, were slightly lower than the budgeted amounts (see Table 3-3). Where there was a balance, CDC approved carrying it over to the next year, with the exception of 2015 to 2016, after PEPFAR had deemed the Program noncore.

The total amount budgeted for the HRH Program comprised about 4 percent of the total annual health budget for the MOH as reported in the Ministry’s Health Resource Output Tracking Report for available concurrent years (see Table 3-4). Given that government expenditure comprises less than half of the total expenditure on health in Rwanda (as described in Figure 3-3), the HRH Program likely represented less than 3 percent of total health spending in Rwanda at the time it was implemented.

TABLE 3-2 HRH Program Budget by Year (U.S. Dollars)

NOTES: Amounts and totals are reported as provided by the MOH in current U.S. dollars. On average, inflation was about 5.2 percent in the same period, and the average growth of the exchange rate was about 5.8 percent between 2012 and 2018 (World Bank, 2019f,g). U.S. government and Rwandan FYs are not the same: U.S. FY runs October 1 through September 30; Rwandan FY runs July 1 through June 30.

SOURCE: Financial data provided by the MOH.

Figure 3-5 provides the context of the MOH’s broader concurrent workforce-related expenditures before and during the HRH Program. From FY 2010/2011 to FY 2014/2015, as the amount of MOH expenditure going toward health workforce increased there was a small decline in workforce expenditure as a proportion of the total MOH expenditure, from 42 percent to 39 percent. Concurrently, there was an increase in the proportion of investments for drugs and other consumables and for materials and equipment. However, trends in these data are difficult to interpret because of changes from year to year in how expenditures were reported by the MOH. It is also not possible to clearly interpret how the overall expenditures of the MOH intersected with the HRH Program expenditures because these reports do not indicate whether these ex-

TABLE 3-4 Ministry of Health Budget and Expenditures Reported Preceding and During the HRH Program (U.S. Dollars, Rounded to Millions)

NOTES: Amounts shown are the current amounts reported at the time each source report was published. Amounts were provided in Rwandan francs (RWF) after FY 2011/2012 and therefore converted to U.S. dollars (USD) using the following World Bank historical exchange rates, rounded to the nearest dollar: in 2013, 1 USD = 647 RWF; in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g).

* Expenditure data for FY 2012/13 and budget data for FY 2013/14 were not reported. SOURCES: ^a MOH, 2012c; ^b MOH, 2012c, 2013b; ^c MOH, 2013b; ^d MOH, 2018b.

TABLE 3-5 MOH Health Workforce Expenditures with Category Breakdowns by Year (U.S. Dollars)

NOTES: Amounts shown are the current amounts reported at the time each source report was published. In FY 2013/2014 and FY 2014/2015 expenditure categories were reported as percentages of the total expenditure. Amounts reported in Rwandan francs (RWF) were converted to U.S. dollars (USD) using the World Bank’s historical exchange rates, rounded to the nearest dollar: in 2014, 1 USD = 682 RWF; in 2015, 1 USD = 721 RWF (World Bank, 2019g).

SOURCES: MOH, 2012c, 2013b, 2016a, 2018b.

penditures include or exclude the funds expended as part of the HRH Program. In addition, there may be other sources of expenditures on HRH in Rwanda.

The categories of expenditures aggregated into the broad category of “workforce,” and how they were reported, varied from year to year (see Table 3-5).

A consistent pattern was that salaries and labor costs represented the largest proportion of expenditures. When reported, categories such as capacity building, pre-service training, and workshops represent a very small amount of the total investment.

HUMAN RESOURCES FOR HEALTH PROGRAM VISION AND DESIGN

Political, Economic, and Social Context During the Design of the HRH Program

By 2012, when the HRH Program was launched, Rwanda was viewed as an ideal environment, not least because it was perceived as being a “peaceful and stable country,” which was seen as an important factor for successful program implementation. Health-sector achievements in the intervening years (see Chapter 7) made the landscape especially attractive for international donors:

These achievements built on years of previous engagements by and with the United States and other partners in Rwanda, which created energy and interest in continuing to develop the health sector:

The perceived political will, on the part of the United States at the time and the Government of Rwanda, also created enabling conditions to develop and fund a holistic HSS program such as the HRH Program:

Across all respondent groups, the perception was that the Rwandan government’s leadership in the context of a strong health system was key to successful programming:

The facilitating environment in Rwanda, from the perspective of U.S. institution (USI) faculty participating in the HRH Program, had three parts. First, Rwanda was viewed as having a “flexible” health system that could change with an evolving health workforce. Second, the government

was confident in its successes in fighting HIV. Finally, existing infrastructure, such as roads and Internet access, was seen as facilitating successful implementation.

HRH Program Vision

At the time of the HRH Program’s design and funding, Rwanda was implementing the Health Sector Strategic Plan 2009–2012 and the HIV/AIDS National Strategic Plan 2009–2012, both of which had goals around the availability, quality, and rational use of HRH in service of improved health outcomes, in alignment with Vision 2020 goals of health equity, universal health coverage, and increased access to and delivery of quality health care (CDC, 2014; MOH, 2011b; Uwizeye et al., 2018). However, there were several critical obstacles within the Rwandan health care system, including the shortage of skilled health workers, poor quality of health worker education, and inadequate infrastructure, equipment, and management in health facilities (MOH, 2011b, 2014b, 2016b).

According to program documents, the HRH Program was designed to address those challenges as a large, systems-based, health-sector-wide initiative to scale up institutional and training capacity and create a high-quality, sustainable health system in Rwanda capable of providing “world-class care” (CDC, 2012; MOH, 2011b, 2012b, 2014b, 2016b). Upgrading the health professional workforce to be of “sufficient quantity and quality to meet the national need” would “contribute to the distal goal of improved availability and quality of care in Rwanda” (MOH, 2014b). Strategies would focus on increasing skill levels and specialization of health professionals and educators, establishing high-quality training sites, accelerating recruitment and retention of students, creating a culture and career ladder for health professional teaching, and expanding research partnerships and academic exchange (MOH, 2011b).

According to the Government of Rwanda, sustainable impact of the HRH Program would be the country’s ability to produce a supply of new skilled, specialized health workers and Rwandan health educators and to ensure adequate infrastructure, equipment, and supplies without external financial support (MOH, 2014b). This country vision aligned with the PEPFAR 2.0 (2008–2014) strategic vision and focus on developing a sustainable HIV response that supported HSS activities with partner governments (PEPFAR, 2019a,b).

With the inception of PEPFAR 3.0 in late 2014, PEPFAR’s strategic vision pivoted away from this horizontal approach and toward targeted epidemic control (PEPFAR, 2014). This shift in priorities did not fully align with Rwanda’s strategic goals, as the Government of Rwanda has consistently planned for developing the capacity of its health system. Iterations of

Rwanda’s Health Sector Strategic Plans for 2009 to 2024 have emphasized HSS in key intervention areas, community-based health insurance, and quality of care through performance-based financing, while outlining policies to address challenges for vulnerable and marginalized populations, sustainable health system financing, and prevention and control of noncommunicable diseases (MOH, 2009a, 2012d, 2018a). Against this backdrop, the HRH Program was conceived with the goal of ultimately increasing the quality of health care delivery and the overall health care system in Rwanda, contributing to the government’s “mandate in terms of developing capacity of the health staff” (45, Government of Rwanda HRH Program Administrator).

Interview respondents for this evaluation shared the same understanding of the HRH Program’s rationale, which they saw as being borne out of a need to rebuild the health system following the genocide against the Tutsi and in the face of an HIV epidemic and an aging population of people living with HIV (PLHIV):

Thus, the HRH Program “was fully integrated into the health system” with potential impacts beyond “just one disease” such as HIV (87, Government of Rwanda Program Administrator). This is in keeping with the evolution of Rwanda’s health sector strategic planning at the time, which focused on health-related MDGs, and specifically maternal and child mortality in the first phase (2005–2009) and transitioning to a focus on HSS and financial access to health services in the second phase (2009–2012). The third phase (2012–2018) emphasized health resource management and governance mechanisms (MOH, 2012d).

HIV Achievements in Rwanda

By 2011, when the HRH Program was being designed, Rwanda had made notable achievements in addressing the HIV epidemic relative to other countries in the region, as discussed in Chapter 1. Concurrent to the evolution in the health-sector strategic planning was a process of decentralization that facilitated increased access to HIV services. The first phase (2000–2005) facilitated the expansion of HIV services to lower-level facili-

ties (Binagwaho et al., 2016). This effort helped to distribute and expand HIV care by integrating community health workers who were providing services at the village level with the more advanced care offered at health centers, and district and referral hospitals (Binagwaho et al., 2016).

In 2008, during the second phase of decentralization (2006–2010), the Rwandan health system moved authority to the district level to launch new HIV treatment sites. However, coordination of services between HIV and other care remained poor. While districts received funding for health, they received funding for essential HIV/AIDS services much more slowly as PEPFAR and the Global Fund followed different paths to decentralization (Nsanzimana et al., 2015).

The second Health Sector Strategic Plan in 2009 called for the need to better integrate HIV/AIDS care into routine health services. To support this effort, the Government of Rwanda entered into a cooperative agreement with CDC to increase staffing (CDC, 2009b). It also started transferring management of HIV patients from international partners to government-run programs (PEPFAR, 2010; PEPFAR Rwanda, 2011). This transition resulted in a sustainable and successful HIV program that now has the internal expertise to manage HIV care at all levels (Binagwaho et al., 2016).

During the third phase of decentralization, starting in 2011, the Rwanda Biomedical Center restructured HIV coordinating mechanisms to facilitate better integration with other disease-specific programs (Nsanzimana et al., 2015). The previous HIV programs were dissolved, including the National AIDS Control Commission, so other disease-focused programs could be leveraged to create operational efficiencies, especially given dwindling resources. The third Health Sector Strategic Plan (2012–2018) called for the integration of HIV services at a decentralized level, the need to improve quality, and the need to maintain trained and adequate numbers of staff at all facilities (MOH, 2012d). The result of this decentralization has been a rapid increase in the number of facilities offering ART services, from 4 in 2002 to 552 in 2016, as reported in the Rwanda Integrated Health Management Information System.

Respondents related the success of Rwanda’s health sector to the window of opportunity provided to the HRH Program. Respondents perceived Rwanda as having “achieved all the expressed outcomes that we entered with the intent of achieving,” providing an opportunity to explore what could be done

Rwanda was seen as an interesting test case for a different model of development that leveraged the gains achieved through a vertical program to strengthen an entire health system. To the extent that HIV outcomes were considered in the design of the HRH Program, the MOH viewed building a specialized health workforce as a priority in addressing long-term care for PLHIV and infectious disease control, which was considered a moral, epidemiologic, and economic necessity—a “triple imperative” (Binagwaho et al., 2013).

Respondents working with NGOs in the HRH and HIV space in Rwanda felt that an HRH Program with the goals of strengthening the health system could have an impact on HIV outcomes:

HRH Program Goals, Objectives, and Strategy

While there was general agreement on the overarching vision of the HRH Program, there was less congruence between program documentation and interview respondents on the intended mechanisms for achieving this vision. Program documents illustrated that developing the Rwandan health education system remained a principal component of the HRH Program. Throughout the Program, the MOH noted that focusing on health education was a mechanism for filling the gap between the supply and demand of highly qualified health professionals in Rwanda (MOH, 2011b, 2015b, 2016b). By the midterm review, however, key HRH Program activities such as twinning through participating USIs that formed the Academic Consortium were deemed necessary to “gap-fill” and “simultaneously build the long-term capacity of their counterparts and host institutions” (MOH, 2016b).

This evolved from the initial premise that the Program was designed to simply fund the mechanism for dramatically increasing production of health workers in a short time frame (MOH, 2011b). As highlighted in the 2011 HRH Program proposal, the original objectives were to increase the number of physicians and physician specialists, continue advancing the skill levels of nurses and midwives, introduce the role of health manager into the Rwandan health system, launch the Rwanda School of Dentistry (the first class began in fall 2013 at the University of Rwanda), and build the institutional capacity of Rwanda’s health professional schools and clinical teaching hospitals to sustain high-quality health education (MOH, 2011b).

Most of these objectives continued to be referenced as core program expectations informing the development of core indicators in the Program’s

eventual monitoring and evaluation (M&E) plan and as objectives in a results framework for the midterm review (see Table 3-6). Many outlined areas of work, such as expanding research partnerships and academic exchange, were also generally consistent through the years though with varying specificity. For example, the 2015 programmatic/technical work plan emphasized “internationally benchmarked curricula” in one of the areas

of work, whereas other program documents stated that the HRH Program would “implement quality, competency-based instruction” (MOH, 2011b, 2015b, 2016b).

Notably, it was not until the MOH’s performance and measurement plan development process, which began after program implementation, that three overarching strategic outcomes emerged. Two of these outcomes highlighted the role of the University of Rwanda’s College of Medicine and Health Sciences (CMHS) in providing high-quality health professional education programs and accompanying training environments, alongside the third outcome of establishing a sustainable, skilled, and specialized health professional workforce (MOH, 2014b, 2015b). In addition, M&E had not been designated as a specific area of work until this point; the 2014 M&E plan acknowledged the challenge of establishing a baseline for all program areas, because M&E efforts were being articulated after implementation (MOH, 2014b). This was reflected in inconsistencies in baseline and target goals for the number of health professionals in several cadres, particularly in the number of nurses and midwives to be produced by the HRH Program once it began to deemphasize the upgrading of A2 to A1 nurses (MOH, 2011b, 2014b).⁴

Qualitative data reveal a lack of congruence in the pathways to reach the Program vision; respondents reported a range of strategies including improving quality of care, producing high-quality health workers, building primary care, building specialty care, strengthening the medical education system (including faculty), and improving the availability of equipment and infrastructure. Among Government of Rwanda respondents, the Program’s main objective was unequivocally to build a larger cadre of health care workers across specialties:

To do this, as one HRH Program trainee articulated, it was necessary to build postgraduate training programs (32, University of Rwanda Non-Twinned Faculty and Former University of Rwanda Student in Obstetrics and Gynecology).

In contrast, the perceived objectives of the HRH Program among most University of Rwanda and USI respondents were to upgrade the number

___________________

⁴ A2 nurses have completed secondary school education; A1 nurses receive a diploma after 3 years of training at a higher education institute; A0 nurses are graduates of a 4-year bachelor’s program and may go on to enroll in a master’s program (Uwizeye et al., 2018).

and skills of Rwandan health educators. When probed, one Government of Rwanda respondent who had been involved in the Program’s design vehemently disagreed with the assertion that building capacity within the University of Rwanda was a programmatic objective:

Some USI faculty reported that the objectives and design of specific specialties changed significantly over the course of the Program. For example, the initial objective of the nursing and midwifery activity was to contribute to the skills upgrading of A2 nurses, but this shifted to focusing on building an A0 nursing cadre through the development of the Master of Science in Nursing (MSN) program:

Similarly, the objectives around the Master of Hospital and Healthcare Administration program evolved as its developers gained an understanding of the context and needs in hospital administration:

The emphasis on specialized care over primary care evolved throughout the Program and was not clearly understood among different stakeholder groups. The Program comprised both pre- and in-service training activities, but the former were prioritized in accordance with national HRH policy because pre-service education “is less costly and gives immediate hand[s]

on skills to the health professional” (MOH, 2014c). The strategy to focus on immediate hands-on skills was a central tenet of train-the-trainer models and the twinning program, which planned for Rwandan faculty and new graduates to phase out Academic Consortium faculty or to train colleagues in district hospitals, building capacity for local ownership and sustainability.

Efforts to strengthen the medical and nursing programs and the medical education system were under way before the HRH Program. In FY 2008 and FY 2009, USAID funded the University of Colorado to second “a family medicine faculty member … to provide extensive practical teaching, postgraduate supervision, and assistance with [the] development of the Family Medicine program” and to assist in integrating HIV/AIDS into postgraduate medical program curricula (PEPFAR Rwanda, 2008, 2009). By FY 2010, the University of Colorado’s efforts had been subsumed under a large CDC-funded capacity-building program with Tulane University and had expanded to include support for a 4-year postgraduate medical program aimed at preparing “physicians to function with a broad clinical scope … to better address the burden of disease existing in Rwanda’s rural communities.” Twenty-three physicians, “including seven in Family and Community Medicine,” were enrolled (PEPFAR Rwanda, 2010).

The in-country postgraduate medical program was initiated under the former Minister of Health, Jean Damascene Ntawukuriryayo, and continued by his successor, Richard Sezibera (Flinkenflögel et al., 2015). Although some of the work performed under Tulane’s cooperative agreement with CDC was transitioned to the University of Rwanda, the agreement was not renewed at the close of its 5-year term under Agnes Binagwaho’s tenure as Minister of Health (PEPFAR Rwanda, 2012).

One respondent who was involved in the design and early implementation phases noted that primary care was central to the HRH Program at the start, but was excluded from the list of clinical areas during implementation:

According to one respondent representing an international NGO, the reasoning behind supporting specialized care over primary care was not well understood, because it

In contrast, a senior MOH official commented that there was no need to build primary care in Rwanda when the HRH Program started, because there were—and continue to be—other investments and efforts to strengthen primary care, but no efforts focused on specialized care.

Notably, few respondents spoke about the objectives of the HRH Program in relation to HIV. Two University of Rwanda administrators reported that the Program’s goal should have been positioned in the context of “transition from acute burdens of HIV as a signature illness to much more chronic disease management” (02, University of Rwanda Administrator). A former HRH Program administrator expanded on this idea:

The literature echoes the HRH Program–HIV connection, in which it was imperative for the MOH to build a specialized health workforce to address long-term care for PLHIV and infectious disease control (Binagwaho et al., 2013).

DESIGN PROCESS

Figure 3-6 depicts the overall time line of the HRH Program. In 2011, the Government of Rwanda submitted an unsolicited proposal, which was funded in 2012 under the expanded Clinical Services Cooperative Agreement (CDC, 2012). Prior to funding, 18 USIs submitted letters of intent to join the Academic Consortium, although one withdrew before the Program was launched (MOH, 2011c). USI participation in the Program increased until 2015, when institutions began withdrawing. An MOU was established between the MOH and a new USI in 2018, after PEPFAR’s

investment had ended. Membership in the Academic Consortium provided USIs with a mechanism through which to establish annual MOUs with the MOH. The Consortium was also a mechanism for determining the clinical purview of each USI. For example, Yale University was an obstetrics

and gynecology partner, and New York University and Emory University provided support to the nursing programs, both upgrading nurses and the establishment of the MSN program.

USIs under the Academic Consortium contracted with individuals to work at the University of Rwanda to “twin” with and mentor existing Rwandan faculty, aid in strengthening existing residency programs and establish new programs, provide direct teaching services to Rwandan students, and, in some cases, provide direct care to patients. With few exceptions, the USIs contracted with U.S. citizens and did not engage regional faculty who could lend practical experience and knowledge from a more locally relevant context to the Program (see Chapter 4 for a more detailed discussion).

Data-Informed Design

It was clear from interview respondents, including those who were engaged in the design of the HRH Program, that no specific baseline assessment was done prior to launching the Program. Six Government of Rwanda current program administrators referenced supporting documents, such as reports or an assessment, that informed the Program; three of these respondents specifically cited the Third Health Sector Strategic Plan and the Human Resources for Health Strategic Plan 2011–2016:

The Third Health Sector Strategic Plan makes reference to an extensive situation analysis and comprehensive midterm review conducted in 2011, which informed the plan’s priority to “improve quantity and quality of human resources for health (planning, quantity, quality, management)” (MOH, 2012d). Other HRH Strategic Plan objectives include increasing the number of trained and equitably distributed staff and improving health worker productivity and performance in part by improving the quality of pre-service training (MOH, 2011a). Similarly, PEPFAR’s Rwanda Country Operational Plans reference Tulane University’s collaboration with other donors and the “National Technical Working Group for Human Resources in an in-depth needs assessment … designed to determine the health workforce needs in Rwanda, taking into account the disease burden, existing cadres and

ongoing trainings (pre- and in-service)” (PEPFAR Rwanda, 2010, 2011). However, respondents from the Clinton Health Access Initiative (CHAI), who collaborated with the MOH in the design of the HRH Program, felt the MOH was not in a position to examine or plan for their HRH needs:

Collaboration During Design

The HRH Program design process appeared to have involved members of the MOH and CHAI, without involving other key partners, such as the Ministry of Education (MOE), the Ministry of Finance and Economic Planning (MINECOFIN), or implementing partners working in HIV and HRH. CHAI’s role, with financial support from the ELMA Foundation, was one facilitator. At the request of then-Minister of Health Richard Sezibera, CHAI convened an HRH working group to prepare a road map and strategy. These documents formed the basis of the HRH Program proposal, which CHAI “unilaterally wrote … and submitted to the U.S. Embassy” (20, Government of Rwanda HRH Program Administrator). As part of this process, CHAI worked with the MOH to determine programs within the PEPFAR portfolio whose funding could be reduced or terminated to cover the costs of the Program:

This was corroborated by PEPFAR during one of the committee’s open sessions. Once the Program was funded, CHAI was asked to stay on and

managed the HRH Program through the first year of implementation while the MOH set up internal management structures and onboarded staff.

Although the MOH–MOE relationship faced challenges at the beginning of the HRH Program, it strengthened over time. Not involving the MOE during Program design and early implementation was perceived as a mistake:

One MOH respondent involved in the early stages of the Program described the MOE as “a contractor” (18, Former Government of Rwanda HRH Program Administrator). A Non-Government of Rwanda HRH Program Administration respondent expressed his view that there was no relationship between the MOH and the MOE before the Program, and that a relationship had been built through MOH–MOE steering committees formed to set standards for health care professional education:

FINANCIAL MANAGEMENT

The 2011 HRH Program proposal indicated that a Single Project Implementation Unit would be established “to centralize programmatic, financial, and administrative management functions” (MOH, 2011b). PEPFAR investments in the HRH Program flowed directly to the MOH, although according to one respondent, a “parastatal” had been set up to manage

the money, where funds were disbursed by MINECOFIN. One respondent involved in the early phases of the Program noted that this design was intended “to try and eliminate the high cost of an international NGO go-between … and having a government that had demonstrated their ability to manage and oversee, monitor, and evaluate programs” (25, Non-Government of Rwanda HRH Program Administrator).

The Permanent Secretary of the MOH had oversight over the HRH Program budget. The Government of Rwanda employed a Public Financial Management system, in which funds were held in an account separate from the National Treasury. The system facilitated budget oversight, because the University of Rwanda used the same system.

According to MOH data, the bulk of the expenditures from PEPFAR investments in the HRH Program went to USIs, followed by equipment procured for health professional education and clinical training (see Table 3-7).

Table 3-7 includes only funds provided by CDC and does not reflect other investments from sources such as the Global Fund and the MOH. According to the MOH, the $441,661 difference between the budgeted amount and the disbursed amount was not funded by CDC.

The MOH articulated the importance of unmeasurable inputs that were critical for the running of the HRH Program, including overhead costs expended by the University of Rwanda for managing its facilities, unallowable expenses incurred by USIs in recruiting and managing USI faculty, and other

TABLE 3-7 HRH Program Expenditures of PEPFAR Investments (U.S. Dollars)

NOTES: HRH = human resources for health; USI = U.S. institution. The fluctuating exchange rate was set by the Rwandan National Bank. The original Program proposal had a specific line item for equipment maintenance, amounting to $1.5 million. A similar line item is not reflected in these expenditure data.

SOURCE: Financial data provided by the MOH.

essential financial inputs by the MOH, such as annual travel allowances for all residents to move across sites for their rotations and visit their families when they were rotating at a site away from their primary residence.

Financial audits of the Global Fund’s investments during periods concurrent with the HRH Program (2014, 2016–2018), and therefore presumably inclusive of funds applied toward the HRH Program, yielded no major concerns. Systems of internal control were typically found to be “generally” or “partially” effective. The 2014 audit identified three areas for improvement: data quality assurance, supporting documentation, and low absorption of funds (Global Fund, 2014). The 2016–2018 Global Fund audit found, however, that grant oversight and assurance mechanisms needed significant improvement (Global Fund, 2019). Audits of other investments in the MOH had similar findings. An audit of the MOH’s management of Gavi funds rated the MOH as partially satisfactory (Gavi, 2018), and an assessment of a Swedish International Development Cooperation Agency–supported program that centered on building research and higher education capacity identified three key issues in the MOH’s financial management: lack of consolidated financial reporting; delays in disbursements to Swedish partner universities; and slow budget execution that affected implementation (Tvedten et al., 2018). Additionally, a thorough search for a CDC audit of the HRH Program was conducted, and none was located.

Beyond the financial management processes, program administrators expressed some frustration about PEPFAR’s processes and restrictions on how funding could be used, especially in light of the HRH Program’s holistic and integrated design:

Institutional Financial Arrangements

The 2011 HRH Program proposal outlined several key principles guiding the relationship between the MOH and USIs. The MOH would receive funds directly from the U.S. government and issue contracts to USIs. MOUs were established between USIs, the MOH, and the MINECOFIN. Each USI had separate coordinators for the academic aspects of the MOU and for finances. The mismatch of FYs between USIs, the Government of Rwanda, and CDC, compounded by yet another difference in the University of Rwanda’s academic year, presented an additional contractual challenge that affected payment scheduling:

The 2016 midterm review also noted this challenge. Upfront stipulations around financial management included that USIs’ overhead to the programmatic and administrative functions could not exceed 7 percent of total direct costs for recruiting faculty, predicated on the assumption that the Government of Rwanda would be responsible for some of these administrative activities (MOH, 2011b,c). The HRH Program proposal also stipulated that USI faculty were expected to commit to residing in Rwanda for at least 11 months and would receive:

USI faculty were mostly contract hires, not existing faculty receiving time to dedicate to the HRH Program.

PROGRAMMATIC MANAGEMENT APPROACHES AND CHALLENGES

The HRH Program experienced challenges in its design, launch, and execution that were both internal to Rwanda (between the MOH and the

MOE and stakeholders outside the Government of Rwanda) and external to Rwanda (between the MOH and USIs).

Interinstitutional Relations

MOH–MOE Relations

As indicated above, the design phase did not actively engage the MOE, a gap that had implications in the early implementation period. Some faculty at the University of Rwanda were aware of the HRH Program before it was launched, and reported being excited that former U.S. President Bill Clinton and Rwandan President Paul Kagame had participated (37, University of Rwanda Administrator), although respondents knew of other faculty members who were not aware of the Program:

This absence of communication about the Program included incoming students who were anticipating joining training programs abroad, but were told they would be enrolled in programs in Rwanda:

The MOH received and managed external HRH Program funding and was responsible for signing MOUs with USIs. The MOE had little input into the allocation of funds. The roles and the responsibilities of the MOH and the MOE, which required a collaborative relationship between the two ministries, was articulated by a respondent representing a professional association:

During implementation, University of Rwanda administrators reported that the communication between the MOE and the MOH was clear and managed through a steering committee chaired by the principal of CMHS. They reported that he had oversight over all of the operational and programmatic issues with HRH trainees, while the MOH dealt with contractual issues and accreditations:

MOH respondents also mentioned the value of the steering committee as the entity “in charge of giving the guidance and making the policies” (20, Government of Rwanda HRH Program Administrator), which strengthened the ministries’ relationship. MOH respondents also articulated the roles and responsibilities, while reinforcing the idea that “it is one government” and neither ministry could unilaterally determine the priorities of both sectors (03, Government of Rwanda HRH Program Administrator). The steering committee was also responsible for ensuring no duplication of efforts between the HRH Program and other investments in strengthening health workers, such as the Capacity Development Pooled Fund.

USI faculty described the relationship between the MOH and the MOE as “painful” (23, USI Twinned Faculty in Nursing), affecting their ability to support the review and approval of new or updated curricula and accreditation of new programs:

MOH–International NGO Relations

As with the design, respondents from international NGOs felt the MOH was not inclusive of other organizations in the program’s management and implementation:

MOH–USI Relations

The 2016 midterm review highlighted challenges in contracting processes: lack of clarity around leave and other policies, time line and processes for obtaining work permits and licenses for USI faculty, and funding and reporting requirements that resulted in delays in drafting MOUs; delays in annual revising and reissuing MOUs; late award notification, which delayed program funding and USI reimbursements; and a need to accommodate USI and HRH Program processes delaying submission of invoices and payment (MOH, 2016b).

Interview respondents for this evaluation echoed many of these challenges. From the USI perspective, challenges with contracting included contracts being limited to 1 year, requiring annual renewal, differences in malpractice standards, and the exclusion of overhead from contracts (described in the previous section), which was viewed as an “extra horror” that was “not realistic for a long-term project with a U.S. institution” (15, USI Faculty in Hospital and Health Administration). MOH delays in issuing contracts, raised in the midterm review and in interviews, had notable implications on the presence of USI faculty at the University of Rwanda and their personal and professional futures:

Program administrators who worked outside of the Government of Rwanda shared that some USI faculty “pulled out” of the HRH Program because the financial gap (due to these contract conditions) “was putting enormous pressure … so it was too much risk” (22, Non-Government of Rwanda HRH Program Administrator). There was also fluctuation in the number of USIs participating in the Program. In 2011, 18 programs had submitted letters of intent to join the Academic Consortium (11 in medicine, 6 in nursing and midwifery, and 1 in health management and global health delivery). In 2012, 1 of those institutions withdrew prior to program launch, though an additional 10 joined the program (9 in medicine, 1 in nursing and midwifery). USIs began exiting the Academic Consortium in 2015 with three exiting in 2015 (two in medicine, one in nursing and midwifery); three in 2016 (two in medicine, one in nursing and midwifery); one in 2017 (from nursing and midwifery); five in 2018 (four in medicine and one in health management and global health delivery); and five in 2019 (two in medicine and three in nursing and midwifery). Two USIs did join during that period: one in 2015 (in health management and global health delivery) and one in 2018 (in medicine).

That USIs were expected to spend no more than 7 percent on administrative activities related to recruitment reinforced a feeling that these institutions were nothing but a “recruitment firm, rather than true partners” (06, USI Twinned Faculty in Pediatrics) in building the institutional capacity to provide high-quality health professional education. This feeling was exacerbated in situations when the MOH told USIs its needs, sometimes with little warning, and with USIs conducting the screening and proposing individuals and the MOH making final decisions. Additionally, differing benefits between the two countries created obstacles, especially around maternity leave, which was part of the contracts USIs issued but was not a Government of Rwanda benefit. A similar obstacle was perceived with sick leave:

The 2011 proposal indicated that “basic benefits [would] be provided” to USI faculty, naming only health insurance, emergency evacuation insurance, adequate housing, and roundtrip airfare (MOH, 2011b).

Government of Rwanda staff who managed the HRH Program also sometimes felt disappointed by USIs, which was seen as affecting programmatic success:

Communication from USIs to the MOH about the physician specialists, nurses, and midwives selected to travel to Rwanda, specifically their experience and seniority, also affected the relationship, although this was seen as having improved “over time as we went on mentioning this challenge the profiles changed and I think they would send even better people” (48, Government of Rwanda HRH Program Administrator).

MONITORING AND EVALUATION

Reporting requirements for the HRH Program evolved over the years, in part because of the changing funding mechanisms as well as the broader shift in standards for M&E. In April 2012, before the Program was integrated into the MOH’s periodic program reports for the Clinical Services Cooperative Agreement, the MOH submitted to CDC a detailed work plan and budget for the Program. The work plan was similar to a logical framework and included goals, objectives, activities, a quarterly time line, and program indicators that were consistent, if not more detailed, than CDC reporting requirements at the time (CDC, 2009a; MOH, 2010, 2012a). Program indicators for the HRH Program and other components of the Clinical Services Cooperative Agreement, however, remained largely output measures (CDC, 2009a; MOH, 2010, 2012e).

In 2013, the work plan for the Clinical Services Cooperative Agreement Continuing Application called for the development of an “M&E frame-

work and indicators” as one of the HRH Program’s “major programmatic/technical activities” (MOH, 2013a). An open invitation for Program participants to volunteer to join one of two stakeholder groups—an in-country M&E Working Group (comprising HRH Program members and Rwandan faculty) and a USI M&E advisory group—was also sent in 2013 to support the M&E plan development. As a result, the MOH submitted to CDC the Program’s “Human Resources for Health Monitoring & Evaluation Plan” in March 2014 (MOH, 2014b).

In addition to the three main strategic outcomes described previously, the M&E plan provided outputs, detailed process and outcome measures, data sources, annual performance targets, reporting frequency, and baseline and midterm evaluation components; the rigor of this plan reflected PEPFAR’s increasing emphasis on M&E (PEPFAR, 2011, 2012) and use of more sophisticated metrics, which had begun in 2009 with the development of PEPFAR’s Next Generation Indicators. In contrast to PEPFAR’s earlier metrics, which had been largely output measures, the Next Generation Indicators sought to measure program coverage and quality using both process and outcome indicators.

In October 2014, when CDC issued a noncompetitive funding opportunity announcement to continue the Program’s funding at a level of $14 million per year for 5 years under its own funding mechanism, PEPFAR’s M&E requirements for the Program were both comprehensive and rigorous:

To fulfill these requirements, the MOH submitted a project narrative and evaluation and performance measurement plan in its award application (MOH, 2014d). Under the new cooperative agreement, CDC required the MOH to revise and submit a more detailed evaluation and performance measurement plan within the first 6 months of the award (CDC, 2014). In 2015, the MOH submitted a revised programmatic/technical work plan that documented “SMART (specific, measurable, achievable, relevant, and time-based) Objectives” activities, a time line, responsible parties, and “process measures” (MOH, 2015b).

According to the 2014 M&E plan, a point person, trained in the use of data collection tools for routine reporting, would be available for every teaching hospital and health facility (MOH, 2014b). Three main evaluations at three time points would also be planned to measure the program’s outcomes and impact, recognizing that a baseline might only be available for some areas because the Program had already started (MOH, 2014b). Most indicator data for program monitoring would be collected semiannually from the following sources: the Health Management Information System, Human Resources Information System, health education institution records, and program records. A Health Education Institution Survey and Hospital Survey were also planned, although no evidence was found to show that either survey occurred (MOH, 2014b).

The HRH Program midterm review reported that quarterly surveys were administered to individuals involved in the Program, although response rates were low, especially among Rwandan faculty (MOH, 2016b). An organizational case study of the Program concluded that its prioritization of implementation costs and underestimate of administrative, monitoring, and evaluation costs—coupled with the substantial size and scope of the Program—created a “mismatch between needs and availability of resources and expertise” and a lack of resources for M&E (Cancedda et al., 2018).

From the perspective of interview respondents, little investment was made in M&E processes, which had also been noted in the Program’s midterm review. A USI respondent commented that having a single M&E officer for the Program was insufficient:

Another respondent stated that there was “a high-level M&E document [referring to the 2014 M&E plan], but it’s not specific to actual activities on a yearly basis” (22, Non-Government of Rwanda HRH Program Administrator from an International NGO).

SUSTAINABILITY PLANNING

Under PEPFAR’s partnership frameworks, an important component of PEPFAR’s 2.0 strategy (PEPFAR, 2014, 2019b), sustainability was defined

as “supporting the partner government in growing its capacity to lead, manage, and ultimately finance its health system with indigenous resources (including its civil society sector), rather than external resources, to the greatest extent possible” (PEPFAR, 2009). PEPFAR’s role was to move from supporting the expansion and provision of direct services to offering technical assistance to partner countries to work toward sustainable programming (PEPFAR, 2019b). This was operationalized through a 5-year Partnership Framework, which aimed to produce results in HIV prevention, care, and treatment, and to position countries “to assume primary responsibility for the national responses to HIV/AIDS in terms of management, strategic direction, performance monitoring, decision making, coordination, and, where possible, financial support and service delivery” (PEPFAR, 2009).

For the HRH Program, the 2011 proposal referred to sustainability in the context of the intention that, after 8 years, the Government of Rwanda would operate the Program under its own budget. The proposal included a rationale for the availability of the financial resources that would be needed to absorb the newly trained health workers and maintain equipment purchased under the HRH Program. It argued that the government’s commitment to allocating 15 percent of the total national budget to health, as agreed to in the Abuja Declaration, combined with the projected 5 percent annual increase in the total Rwandan budget due to GDP growth, would generate enough funds to cover the estimated $43 million per year for Rwandan health professionals produced by the HRH Program, $9 million per year for tuition support, $13 million (over several years) for equipment replacement, and $1.5 million per year for equipment maintenance (MOH, 2011b). The MOH’s 2014 funding application program narrative in response to the funding opportunity announcement articulated sustainability as the programmatic aim of building “sufficient Rwandan educators, infrastructure and equipment, and domestic financing to support Rwanda’s health care and health sciences education without external support” (MOH, 2014d).

Although it appears that some planning for sustainability took place during the proposal phase, interview respondents expressed variation in how to define and measure HRH Program sustainability, which was linked to how they conceptualized the Program’s goals and objectives, as articulated by an administrator at the University of Rwanda:

Among respondents who believed the intent was to build capacity within the University of Rwanda to continually produce high-quality health workers, sustainability was seen as built into the design:

One respondent explained that the design included a gradual phasing out of USI faculty and a simultaneous increase in Rwandan faculty:

In contrast, for another respondent, who had expressed that the objective of the HRH Program was to build capacity of Rwandan health education institutions, most directly the University of Rwanda, the Program failed to clearly understand and build on the capacity of local institutions:

Respondents external to the government were skeptical of the sustainability and long-term planning that was done in the design phase:

CONCLUSIONS

The HRH Program represented a confluence of unusual circumstances and an opportunity in foreign assistance. Its design endorsed Rwanda’s larger vision of strengthening the country’s workforce, including in the health sector, although there were missed opportunities to learn systematically. PEPFAR 2.0 marked “a new era of collaborative planning and health systems strengthening activities with … partner governments” (PEPFAR, 2014). PEPFAR’s investments in the HRH Program were a critical contribution to launching an Academic Consortium that leveraged (1) existing and new partnerships with individual USIs, (2) the political commitment of a government with record of successfully managing health systems programs, and (3) a holistic PEPFAR strategy that emphasized HSS.

PEPFAR’s investments in the HRH Program in Rwanda were relatively unique, departing from U.S government and many other big donors’ models for “business as usual,” but aligned with prevailing principles established by the Busan Partnership for Effective Development Co-operation, in which funding partners would support shared goals according to specific country situations (Fourth High-Level Forum on Aid Effectiveness, 2011). PEPFAR investments have continuously supported programs and activities implemented in partner countries and, consistent with PEPFAR’s strategy to move toward sustainability, have directed their funding over time to local prime partners, although the proportion of funding to partner country governments has been relatively stable (IOM, 2013).

From the outset, Rwanda expressed a strong commitment to national ownership, as articulated in Vision 2020, and sought to increase the direct management of externally funded health and HIV programs. To move toward this vision, the Government of Rwanda and its PEPFAR-funded partners promoted a transition plan that shifted HIV program leadership from partners to the host country, including direct PEPFAR financing to the

government (Government of Rwanda, 2012). The management transition took place by February 2012, when the MOH became a direct recipient of PEPFAR funds and gained responsibility for coordinating critical service delivery (Binagwaho et al., 2016). These investments challenged the conventional model, and used country ownership and shared responsibility as a new paradigm when the HRH Program commenced in 2012 (Goosby et al., 2012a,b).

The investments in the HRH Program also were an opportunity to add to the evidence around twinning because the Program’s Academic Consortium was designed not only as a departure from short-term faculty stints in Rwanda, but also to completely phase out by the end of the Program as the “twins” and new graduates gained skills to serve as local faculty. The twinning arrangements were between individual USI faculty and University of Rwanda faculty, not between the two institutions. The implications of this arrangement, discussed more fully in subsequent chapters, were felt in terms of sustained capacity at the institutional level to provide ongoing health professional education.

There was concurrence among HRH Program participants on a high-level vision and intent, which aligned with broader health-sector goals. However, there was a lack of clarity around the mechanisms and pathway for achieving a world-class health care system, the consequences of which were felt during the implementation of the Program, as discussed throughout subsequent chapters. The relative importance of building health education capacity, including enhancing the skills of Rwandan faculty and the learning environment at the University of Rwanda, was inconsistent during the life of the Program. In particular, it was unclear whether activities for building institutional capacity for health professional education were prioritized simply as a mechanism to produce the desired number of health professionals across cadres, or viewed as complementary goals to having a sustainable, skilled, and specialized workforce, as expressed in the later strategic outcomes (MOH, 2011b, 2014b, 2015b).

The downstream effects of the shifting language in MOH documents was a lack of clarity across stakeholders around the Program’s goals and objectives, which had implications for the design and for sustainability planning and were further compounded by differing (and sometimes conflicting or restrictive) administrative practices across participating institutions (Government of Rwanda, USIs, and PEPFAR). Planning for sustainability evolved from accounting for funding of the established training activities under the HRH Program after 2019 to more holistically ensuring the quality of health professional education and service delivery.

Although the Government of Rwanda recognized the unprecedented scale and nature of the HRH Program’s strategies, there was insufficient planning to exploit the opportunity to learn systematically from this en-

deavor by establishing rigorous M&E processes and supportive mechanisms at the outset (MOH, 2011b, 2014b, 2016b). The original program objectives were incorporated into various indicators for the M&E plan and midterm review; however, the development of an M&E plan after implementation had begun resulted in a lack of baseline data for some cadres and program areas, as well as unexplained differences in baseline and target values (MOH, 2011b, 2014b, 2016b). Even as the post hoc M&E plan articulated an intention to “provide a reliable data-driven approach … to ensure accountability, program improvement, impact, and learning,” there was lack of follow-through by the Government of Rwanda in providing dedicated resources to establish an M&E platform (e.g., M&E working groups were composed of volunteer HRH Program participants, as opposed to designated roles), to carry out planned assessments, and to build overall capacity for monitoring, evaluation, and learning (MOH, 2014b).

Overall management of the HRH Program was challenged by the lack of time for operational management, both at the outset of implementation and continuously, as unexpected circumstances arose. More time between the design, launch, and execution phases would have supported stakeholders’ ability to better anticipate and develop contingent strategies for issues such as PEPFAR funding processes and restrictions.

REFERENCES

Cancedda, C., P. Cotton, J. Shema, S. Rulisa, R. Riviello, L. V. Adams, P. E. Farmer, J. N. Kagwiza, P. Kyamanywa, D. Mukamana, C. Mumena, D. K. Tumusiime, L. Mukashyaka, E. Ndenga, T. Twagirumugabe, K. B. Mukara, V. Dusabejambo, T. D. Walker, E. Nkusi, L. Bazzett-Matabele, A. Butera, B. Rugwizangoga, J. C. Kabayiza, S. Kanyandekwe, L. Kalisa, F. Ntirenganya, J. Dixson, T. Rogo, N. McCall, M. Corden, R. Wong, M. Mukeshimana, A. Gatarayiha, E. K. Ntagungira, A. Yaman, J. Musabeyezu, A. Sliney, T. Nuthulaganti, M. Kernan, P. Okwi, J. Rhatigan, J. Barrow, K. Wilson, A. C. Levine, R. Reece, M. Koster, R. T. Moresky, J. E. O’Flaherty, P. E. Palumbo, R. Ginwalla, C. A. Binanay, N. Thielman, M. Relf, R. Wright, M. Hill, D. Chyun, R. T. Klar, L. L. McCreary, T. L. Hughes, M. Moen, V. Meeks, B. Barrows, M. E. Durieux, C. D. McClain, A. Bunts, F. J. Calland, B. Hedt-Gauthier, D. Milner, G. Raviola, S. E. Smith, M. Tuteja, U. Magriples, A. Rastegar, L. Arnold, I. Magaziner, and A. Binagwaho. 2018. Health professional training and capacity strengthening through international academic partnerships: The first five years of the Human Resources for Health Program in Rwanda. International Journal of Health Policy and Management 7(11):1024–1039.

CDC (U.S. Centers for Disease Control and Prevention). 2009a. Funding opportunity announcement: Clinical services 5-year proposed work plan. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

CDC. 2009b. Funding opportunity announcement: Strengthening the Ministry of Health’s capacity to respond to the HIV/AIDS epidemic in the Republic of Rwanda under PEPFAR. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

CDC. 2009c. U.S. Department of Health and Human Services Centers for Disease Control and Prevention strengthening the Ministry of Health’s capacity to respond to the HIV/AIDS epidemic in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR) funding opportunity. Atlanta, GA: U.S. Centers for Disease Control and Prevention.

CDC. 2012. Request approval to award a multi-year expansion supplement exceeding 25% for the cooperative agreement PS09-9118, 3U2GPS2091 “Strengthening the Ministry of Health’s Capacity to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR)” and approval to waive the requirement of an objective review for the single awardee to this cooperative agreement. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

CDC. 2014. Funding opportunity announcement: Strengthening human resources for health capacity in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR) CDC-RFA-GH15-1568. Atlanta, GA: U.S. Centers for Disease Control and Prevention. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

Flinkenflögel, M., P. Kyamanywa, B. Asiimwe-Kateera, S. Musafiri, P. Kayumba, M. Irakoze, P. Cotton, and A. Hibble. 2015. Umuganda for improved health professions education in Rwanda: Past, present and future in the training of health professionals at the University of Rwanda. Rwanda Journal 2:96.

Fourth High-Level Forum on Aid Effectiveness. 2011. Busan partnership for effective development co-operation: Fourth High-Level Forum on Aid Effectiveness, Busan, Republic of Korea, 29 November–1 December 2011. Paris, France: Organisation for Economic Co-operation and Development.

Gavi (Gavi, the Vaccine Alliance). 2018. Republic of Rwanda: Programme audit of Gavi support to the Ministry of Health. Geneva, Switzerland: Gavi, the Vaccine Alliance.

Global Fund (Global Fund to Fight AIDS, Tuberculosis and Malaria). 2014. Audit report: Audit of global fund grants to Rwanda. Geneva, Switzerland: Global Fund.

Global Fund. 2019. Audit follow up report: Global fund grants to Rwanda. Geneva, Switzerland: Global Fund.

Goosby, E., M. Dybul, A. S. Fauci, J. Fu, T. Walsh, R. Needle, and P. Bouey. 2012a. The United States President’s Emergency Plan for AIDS Relief: A story of partnerships and smart investments to turn the tide of the global AIDS pandemic. Journal of Acquired Immune Deficiency Syndrome 60(Suppl 3):S51–S56.

Goosby, E., D. Von Zinkernagel, C. Holmes, D. Haroz, and T. Walsh. 2012b. Raising the bar: PEPFAR and new paradigms for global health. Journal of Acquired Immune Deficiency Syndrome 60(Suppl 3):S158–S162.

Government of Rwanda. 2012. Vision 2020, revised 2012. Kigali, Rwanda: Government of Rwanda.

IOM (Institute of Medicine). 2013. Evaluation of PEPFAR. Washington, DC: The National Academies Press.

Kayirangwa, E., J. Hanson, L. Munyakazi, and A. Kabeja. 2006. Current trends in Rwanda’s HIV/AIDS epidemic. Sexually Transmitted Infection 82(Suppl 1):i27–i31.

MOH (Ministry of Health). 2005a. Health sector policy, February 2005. Kigali, Rwanda: Ministry of Health.

MOH. 2005b. Health Sector Strategic Plan, 2005–2009. Kigali, Rwanda: Ministry of Health.

MOH. 2009a. Health Sector Strategic Plan, July 2009–June 2012. Kigali, Rwanda: Ministry of Health.

MOH. 2009b. National Strategic Plan on HIV and AIDS 2009–2012. Kigali, Rwanda: Ministry of Health.

MOH. 2010. Clinical services work plan. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2011a. Human Resources for Health Strategic Plan, 2011–2016. Kigali, Rwanda: Ministry of Health.

MOH. 2011b. Rwanda Human Resources for Health Program, 2011–2019: Funding proposal. Kigali, Rwanda: Ministry of Health.

MOH. 2011c. U.S. academic institutions—selection rationale. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2012a. HRH application brief cabinet paper, May 2012–30 Sept 2013. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2012b. HRH U.S. Academic Consortium budget years 1 & 2 Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2012c. Rwanda health resource tracker output report for FY 2009–10 exp. & FY 2010–11 budget & FY 2010–11 exp. & FY 2011–12 budget. Kigali, Rwanda: Ministry of Health.

MOH. 2012d. Third Health Sector Strategic Plan, July 2012–June 2018. Kigali, Rwanda: Ministry of Health.

MOH. 2012e. Work plan. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2013a. CDC-RFA-PS09-9118-05CONT13: Strengthening the Capacity of the Ministry of Health to Respond to the HIV/AIDS Epidemic in the Republic of Rwanda under the President’s Emergency Plan for AIDS Relief (PEPFAR). Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2013b. Rwanda health resource tracker output report December 2013: Expenditures FY 2011/12 and budget FY 2012/13. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2014a. Annual report on the implementation of CDC-funded projects, period: October 2012-September 2013. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2014b. Human Resources for Health monitoring & evaluation plan, March 2014. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2014c. National Human Resources for Health Policy, October 2014. Kigali, Rwanda: Ministry of Health.

MOH. 2014d. Project narrative. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2015a. Health sector policy, January 2015. Kigali, Rwanda: Ministry of Health.

MOH. 2015b. Programmatic/technical work plan. Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2016a. Rwanda health resource tracking output report: Expenditure FY 2013/14 and budget FY 2014/15. Kigali, Rwanda: Ministry of Health.

MOH. 2016b. Rwanda Human Resources for Health Program midterm review report (2012–2016). Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

MOH. 2018a. Fourth Health Sector Strategic Plan, July 2018–June 2024. Kigali, Rwanda: Ministry of Health.

MOH. 2018b. Rwanda health resource tracking output report: Expenditure FY 2014/15 and budget FY 2015/16. Kigali, Rwanda: Ministry of Health.

MOH. 2018c. Rwanda HIV and AIDS National Strategic Plan 2013–2018: Extension: 2018–2020. Kigali, Rwanda: Ministry of Health.

Mtiro, E. 2018. Termination of Rwanda cooperative agreement. Kigali, Rwanda. (Available by request from the National Academies Public Access Records Office [paro@nas.edu] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)

Nsanzimana, S., K. Prabhu, H. McDermott, E. Karita, J. I. Forrest, P. Drobac, P. Farmer, E. J. Mills, and A. Binagwaho. 2015. Improving health outcomes through concurrent HIV program scale-up and health system development in Rwanda: 20 years of experience. BMC Medicine 13(1):216.

PEPFAR (President’s Emergency Plan for AIDS Relief). 2009. Guidance for PEPFAR partnership frameworks and partnership framework implementation plans. Version 2.0, September 14, 2009. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEFPAR. 2010. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR. 2011. FY 2012 country operational plan (COP) guidance. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR. 2012. FY 2013 country operational plan (COP) guidance. Version 2. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR. 2014. PEPFAR 3.0: Controlling the epidemic; delivering on the promise of an AIDS-free generation. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR. 2019a. Executive summary of PEPFAR’s strategy. https://www.pepfar.gov/about/strategy/document/133244.htm (accessed February 24, 2019).

PEPFAR. 2019b. PEPFAR’s five-year strategy. https://www.pepfar.gov/about/strategy/document/133251.htm (accessed February 21, 2019).

PEPFAR. 2019c. PEPFAR panorama spotlight. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR. 2019d. PEPFAR Panorama spotlight glossary. https://data.pepfar.gov/glossary (accessed October 11, 2019).

PEPFAR Rwanda. 2008. Populated printable COP without TBD partners 2008 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2009. Populated printable COP 2009 Rwanda. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2010. Rwanda operational plan report FY 2010. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2011. Rwanda operational plan report FY 2011. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2012. Rwanda country operational plan FY 2012. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2013. Rwanda operational plan report FY 2013. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2014. Rwanda operational plan report FY 2014. Kigali, Rwanda: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

PEPFAR Rwanda. 2015. Rwanda country operational plan (COP) 2015 strategic direction summary. Washington, DC: Office of the U.S. Global AIDS Coordinator and Health Diplomacy.

Seymour, B., I. Muhumuza, C. Mumena, M. Isyagi, J. Barrow, and V. Meeks. 2013. Including oral health training in a health system strengthening program in Rwanda. Global Health Action 6:1–6.

TAGGS (Tracking Accountability in Government Grants System). 2019. Ministry of Health Rwanda. https://taggs.hhs.gov (accessed July 9, 2019).

Tvedten, I., A. Byabagambi, J. Lindstrom, and M. Tedre. 2018. Evaluation of the SIDAsupported research capacity and higher education development program in Rwanda, 2013–2017. Stockholm, Sweden: Swedish International Development Cooperation Agency.

USAID (United States Agency for International Development). 1996. Rebuilding postwar Rwanda: The role of the international community. Washington, DC: United States Agency for International Development.

Uwizeye, G., D. Mukamana, M. Relf, W. Rosa, M. J. Kim, P. Uwimana, H. Ewing, P. Munyiginya, R. Pyburn, N. Lubimbi, A. Collins, I. Soule, K. Burke, J. Niyokindi, and P. Moreland. 2018. Building nursing and midwifery capacity through Rwanda’s Human Resources for Health Program. Journal of Transcultural Nursing 29(2):192–201.

WHO (World Health Organization). 2016. Health workforce requirements for universal health coverage and the Sustainable Development Goals. Geneva, Switzerland: World Health Organization.

WHO. 2019. Global health expenditure database. Geneva, Switzerland: World Health Organization.

Willis, B. M., and B. S. Levy. 2000. Recognizing the public health impact of genocide. JAMA 284(5):612–614.

World Bank. 2019a. Current health expenditure per capita, PPP (current international $)—Rwanda. https://data.worldbank.org/indicator/SH.XPD.CHEX.PP.CD?locations=RW (accessed November 4, 2019).

World Bank. 2019b. Domestic general government health expenditure per capita, PPP (current international $)—Rwanda. https://data.worldbank.org/indicator/SH.XPD.GHED.PP.CD?locations=RW (accessed November 4, 2019).

World Bank. 2019c. GDP (current US$)—Rwanda. https://data.worldbank.org/indicator/NY.GDP.MKTP.CD?locations=RW (accessed November 4, 2019).

World Bank. 2019d. GDP per capita, PPP (current international $)—Rwanda. https://data.worldbank.org/indicator/NY.GDP.PCAP.PP.CD?locations=RW (accessed November 4, 2019).

World Bank. 2019e. General government final consumption expenditure (current US$). https://data.worldbank.org/indicator/NE.CON.GOVT.CD (accessed November 4, 2019).

World Bank. 2019f. Inflation, consumer prices (annual %)—Rwanda. https://data.worldbank.org/indicator/FP.CPI.TOTL.ZG?end=2018&locations=RW&start=2000 (accessed November 4, 2019).

World Bank. 2019g. Official exchange rate (LCU per US$, period average). https://data.worldbank.org/indicator/PA.NUS.FCRF?locations=RW (accessed November 4, 2019).

World Bank. 2019h. PPP conversion factor, GDP (LCU per international $)—Rwanda. https://data.worldbank.org/indicator/PA.NUS.PPP?locations=RW (accessed November 4, 2019).