An educational institution’s capacity to provide ongoing, high-quality health professional education requires adequate administrative infrastructure and institutional support to recruit and sustain dedicated faculty, enable them to improve their teaching skills and update curricula as the evidence base changes, and provide them with the necessary teaching equipment and materials. A 2008 report by the World Health Organization (WHO) and the Global Health Workforce Alliance (GWHA) calls for a rapid and significant scale-up of education and training of health workers and highlights
several innovative educational activities shown to be an efficient and effective means for expanding teachers’ capacity, improving the quality of care provided by trainees, increasing the number of available instructors and the stock of future teachers, promoting the quality of instruction, and meeting the needs of the health system (WHO and GHWA, 2008). These activities include curriculum innovations for pre- and in-service training, academic partnerships, the use of nontraditional teachers and teaching programs, and incentives for health workers. Nontraditional teaching programs have encompassed twinning programs, continuing education, programs that reduce overall training time, train-the-trainer models, and “chain” education (WHO and GHWA, 2008). In Rwanda, several training programs and other efforts along these lines have been expanded or introduced to address the critical shortage of qualified health professionals.
In 2013, concurrent with the initiation of the Human Resources for Health (HRH) Program, the Rwandan government restructured its overall public higher education system, merging seven institutions into the newly established University of Rwanda to improve coordination,1 collaboration, and coherence (Cubaka et al., 2015; Flinkenflögel et al., 2015a). The former National University of Rwanda Faculty of Medicine merged with the Kigali Health Institute and three former schools of nursing and midwifery to form the College of Medicine and Health Sciences (CMHS) in the University of Rwanda (Flinkenflögel et al., 2015b). CMHS also joined the newly launched East Africa Health Professions Educators Association, a regional network of universities and health professional educators (Flinkenflögel et al., 2015b). Five schools are housed under CMHS: Medicine and Pharmacy, Nursing and Midwifery, Dentistry, Public Health, and Health Sciences.
This chapter describes HRH Program outcomes across the landscape of health professional education in the University of Rwanda, as well as in clinical training sites. Literature on the HRH Program has documented processes and outputs primarily within the following domains: trainees’ perceptions of HRH Program curricula or activities; measurements of knowledge acquisition or transfer of teaching or clinical skills; factors affecting recruitment of trainees and faculty; and general gaps and improvements to the HRH Program curricula. Increased institutional capacity for health professional education during the HRH Program included training a new cadre of health professionals and educators, expanding the skills of existing faculty, and building capacity within the University of Rwanda to continue to produce these professionals in the future.
Although the 2013 integration brought together students in the health professions, U.S. institution (USI) faculty reported that the university’s in-
1 In addition to the public University of Rwanda, three private institutions in Rwanda provide medical training.
frastructure was not initially set up to accommodate such large cohorts and that classrooms were unavailable for some classes. Although there were improvements in institutional capacity through the HRH Program, many of those challenges remain. Trainees and University of Rwanda faculty reported two key challenges—large student cohort sizes and insufficient equipment and infrastructure to practice on at the university and in teaching hospitals. The latter challenge is addressed in Chapter 6.
HRH trainees, University of Rwanda faculty, and USI faculty respondents in this evaluation all concurred that the quality of teaching improved dramatically during the HRH Program, in terms of the teaching methodologies and the approaches. However, these improvements were reported solely in reference to USI faculty members.2 HRH trainees did not report observed improvements in Rwandan faculty members’ teaching abilities or competencies, partly due to challenges in recruiting HRH faculty to participate in the twinning initiative, and partly owing to the lack of availability among existing faculty members at the University of Rwanda to teach HRH trainees. In the obstetrics and gynecology program, for example, there were reportedly only two Rwandan faculty members in the entire country when the HRH Program started, so by default, many of the students were trained only by USI faculty members.
USI Faculty Teaching Approaches and Methodologies
HRH trainees highlighted the improved teaching methodologies USI faculty brought to their studies, especially the importance of introducing the concepts of evidence-based medicine and an increased focus on research and publication. USI and University of Rwanda faculty reported didactic teaching as an important output of their HRH experiences:
With the HRH Program, I really, really appreciated the quality [of] education I received, because [they] really taught us to follow evidence-based treatment or any procedure, to try to know why—is this really the advanced method to do anything that we are doing? They really showed us … to do evidence-based medicine…. [I]t was
a very good approach. (85, University of Rwanda, Former Student in Pediatrics)
HRH trainees also reported that the quality of teaching was linked to the instructors’ focus on building their capacity to be effective health educators—teaching skills on presentation, teaching methodologies, and management:
Our presentation skills have improved and teaching skills have improved. I think some of my … management skills have improved, because I am much more mature, can take decisions, I can initiate priorities and engage people to do it, and I think that’s through my education. (49, University of Rwanda Faculty and Current Student in Nursing)
USI faculty, University of Rwanda faculty, and HRH trainees highlighted an important change in pedagogical approaches to teaching, based on questioning and iterative dialogue, with simulations and other methods, rather than the previous, more lecture-based instruction:
[T]here was different educational styles that people talked about forever, francophonic style, top-down, the presenter knows everything and the students just absorb, versus what we like to think of [as] in America, and not sure that this is always the case but a flatter organizational structure, more interchange and more inductive…. There’s certainly things like dedicated time for teaching and learning and question[ing]. Certainly, [this approach] became integrated early on and has remained as such for learning. (84, University of Rwanda, Former Student in Pediatrics)
Some University of Rwanda faculty also reported a clear difference in the quality of teaching after the HRH Program. USI faculty and HRH trainees attributed this increase to USI faculty members’ holding HRH trainees to a “higher standard” in terms of what was expected in class and during rotations:
There’s a difference, especially if you see how the practical teaching and learning was done before and how it is done and was done during this period where HRH Program started, and even today. There is a satisfaction. I mean, I can say it from some of the students, but also mainly from the faculty. (80, University of Rwanda Faculty in Nursing)
As further reported in Chapter 6, respondents from the Government of Rwanda made particular note of increased professionalism.
Hands-on training was another improvement all stakeholders mentioned as being a feature of the HRH Program. Government of Rwanda HRH Program administrators, students, and staff highlighted more practical, hands-on experience with patients because of the exposure to the high standards for quality of education USI faculty members implemented at the clinical teaching sites. HRH trainees and USI faculty also identified the value and importance of the more informal and interactive teaching approaches USI faculty employed, which treated HRH trainees as colleagues, rather than students. HRH trainees reported that USI faculty were more approachable than their previous Rwandan instructors, and appreciated having the time to sit with instructors, ask questions, and debrief after treating patients:
I always say that HRH is the best thing that probably that has ever happened to the Rwandan medical system…. I think the standards have changed, even when you look at now really that they are willing to kind of be part of a global kind of practice community and live up to the global standards…. So, I think there was that mindset that came up, and … the way we interacted also was changed. (29, University of Rwanda, Former Student in Internal Medicine)
All respondents for this evaluation reported that training under the HRH Program increased motivation, confidence, and professionalism, which better prepared HRH trainees for their careers. Government of Rwanda respondents highlighted professionalism in particular as critical for enabling HRH trainees to succeed not only in Rwanda, but at a regional and global level:
[W]ith the HRH [Program], there were a lot of changes, especially in professionalism. I can say that maybe you could not really have all specialties you have, like surgical everything, but at least you could see people who guide you to become a good professional. I was lucky to do one rotation at one of the universities in the region for a year during my third year and I could find really a difference. So, I can say that when I completed I was really competent, not only for the country, but even outside. (28, University of Rwanda Administrator and Former Student in Obstetrics and Gynecology)
HRH trainees highlighted that it was not only their new specializations that made them confident, but also the ability to think analytically:
I got used to the HRH personnel because when they gave us a standard … it helped a lot in the professional life that we are now in, because we also learned something important, it is to allow when you don’t know something, it might be somewhere else and enough to know where to find that thing. And it helped me a lot. (85, University of Rwanda Former Student in Pediatrics)
At the facility level, particularly in the nursing program, some HRH trainees reported being better teachers and mentors to their colleagues. For example, in-service midwifery mentoring was introduced at Muhima Hospital in Kigali (Rwanda’s largest maternity specialty hospital) in August 2012, before expanding to Centre Hospitalier Universitaire de Kigali/University Teaching Hospital of Kigali (CHUK), Byumba, Kabgayi, Masaka, Ramagana, Muhima, and some health centers (Ndayambaje et al., 2017). Mentors were experienced midwives with at least master’s degrees and were employed by USIs through the HRH Program. Their role was to support midwives through their Bachelor of Science in Nursing upgrade at the University of Rwanda, promote collaborative in-service training and individual bedside training activities, and increase exposure to theories and best practices:
[A]t CHUB [Centre Hospitalier Universitaire de Butare/University Teaching Hospital, Butare] I remember that I never learnt how to teach others, those twins helped me to be confident and stand in front of people teaching them about different topics. (60, A0 Nurse Trained in the HRH Program)
Alongside the overwhelming praise for USI teaching approaches, there was a recognition that USI faculty members’ effectiveness had been, at first, hindered by cultural and linguistic challenges. Before the HRH Program, Rwandan students had been taught in French, but French was not a requirement for USI faculty, and all HRH-supported programs were taught in English. HRH trainees reported that cultural differences exacerbated this issue in identifying when they were struggling. For example, trainees reported that most Rwandans who did not understand English would not admit that to the instructor during class, instead relying on their peers to translate outside of class. Not surprisingly, those who spoke English were reported to be some of the highest-performing trainees, and self-reported getting more out of their HRH training than classmates with weaker English skills. As this disparity became more apparent, the HRH Program adapted its requirements. HRH trainees who came out of the facility mi-
crosystem sample reported that English fluency became a requirement to participate in the HRH Program during its later years.3
In addition, a few USI faculty and HRH trainees mentioned USI faculty who had not been qualified or prepared to teach. For example, although the HRH Program had an important focus on research, not all USI faculty were qualified to support and teach research skills:
One of my biggest challenges actually within HRH was so many of the other [USI] faculty had research as a goal and had [been] developing long-term research collaboration as a goal, but they just didn’t have the skills. So, the number of requests … to help faculty think through research projects they could do with their twins—I think that could have been positions funded or even … nested training programs within HRH that could join the [USI] and Rwandan faculties’ research projects together. (64, USI Faculty in Public Health)
From the facility microsystem perspective, the quality of teaching varied by specialty. In surgery, one HRH trainee respondent did not feel the USI faculty had contributed to his education, although when speaking of teachers more broadly, he referred to them as “family” (57, University of Rwanda, Former Student in Surgery). In midwifery and nursing, the quality of classroom teaching was seen as very high, but there was a deficiency in the quality of supervision during clinical internships (58, University of Rwanda, Former Student in Nursing). Moreover, students who were also working full time did not gain access to as much practice time and felt “overloaded” (71, HRH Program–Trained A1 Nurse).
Quality of teaching also depended on the specialty. For example, the anesthesiology residency faced consistently low enrollment for its 20 residency spots (5 per year); no new residents entered in the 2013 to 2014 period and only two enrolled in 2014 to 2015 (Chan et al., 2016). One factor affecting recruitment to this specialty was lack of meaningful exposure to anesthesia training for medical students before graduation. Medical students were often trained in the same operating rooms as anesthesia technicians, but given the reality that most patients were seen by technicians, the focus was not dedicated to residents, who sought more specialized training and mentorship (Chan et al., 2016). Similar to surgery, medical students perceived anesthesiology to be more stressful than most specialties because of the long hours without the ability to supplement their base pay in the
3 The facility microsystem in-depth examination looked at a teaching hospital and a lower-level hospital that referred to the teaching hospital. Both were located outside of Kigali.
private sector, as well as the lack of proper equipment, medicines, and supplies in their work environments (Chan et al., 2016).
Finally, HRH trainees reported that the quality of teaching declined when funding from the President’s Emergency Plan for AIDS Relief (PEPFAR) was cut. Trainees and faculty members attributed this decline to the fact there were fewer HRH faculty available to teach and not enough Rwandan faculty with the skills or time to take over. Although the HRH Program may have facilitated improved teaching skills while it was fully funded and running, the decline in teaching quality following a drop in investment indicates that the effect of the Program may have been a short-term injection of faculty.
Recruiting HRH Trainees to University of Rwanda Faculty
The University of Rwanda had mixed results in retention of Rwandan faculty, but showed early progress in recruiting those trained under the HRH Program. In 2019, the university revised its policies and procedures for hiring academic staff to lessen the requirements for promotion, emphasizing the importance of research, and quantifying student mentorship responsibilities (University of Rwanda, 2019). Published data report that between 2012 and 2016, 24 new Rwandan faculty members and 31 tutorial assistants were recruited by CMHS, and an additional 22 faculty members were recruited by the School of Nursing and Midwifery (Cancedda et al., 2018). Sixty graduates from the HRH Program who were employed by teaching hospitals in Kigali and Butare were issued contracts that devoted 20 percent of their time to mandatory teaching responsibilities (Cancedda et al., 2018). Primary data received from the University of Rwanda for this evaluation indicated that 54 faculty were generated in the Master of Hospital and Healthcare Administration (MHA) program between 2015 and 2018. The HRH Program also generated 10 School of Nursing and Midwifery faculty for the postgraduate program but none for the undergraduate program. USI and University of Rwanda faculty, program administrators, and students also reported in interviews that HRH trainees were starting to filter back into the university, in both faculty and administrative roles. This is also evident in this evaluation’s data collection; of the 22 HRH trainees interviewed, 8 are now University of Rwanda staff.
When interviewed, HRH trainees reported varied reasons for their interest in continuing with academia at University of Rwanda, rather than in clinics, as part of their Ministry of Health (MOH) bonding period. HRH trainees most frequently mentioned an interest in conducting additional
research and aspirations to continue knowledge exchange with their HRH professors. A few also mentioned that they were interested in becoming faculty members to supplement their low salaries as health care professionals in clinics and hospitals.
Current and former HRH trainees also reported significant challenges to continuing their careers in the health profession (in education and in clinical work). Reasons for not continuing their education included a lack of funding support from the MOH; low salary after graduation was also a deterrent. According to the University of Rwanda, in 2011–2012 the MOH provided tuition support for 35 percent of the total cost of either 4 years of medical school or 2 years of postgraduate training. This increased to 48 percent starting in 2013, because the total cost of tuition declined from 8,400,000 Rwandan francs to 6,400,000 Rwandan francs. For MHA students, the MOH offered 100 percent tuition support from 2013–2017 and no tuition support starting in the 2018 academic year. The MOH supported the entire first cohort (2015–2017) of Master of Science in Nursing (MSN) students (tuition cost 2,400,000 Rwandan francs). Subsequent cohorts were self-financed (an estimated 70 percent) or supported by individual hospitals, or were employees of the University of Rwanda.
Although few activities focused on recruiting and retaining students, nursing student dropout rates were low (1.43 percent in the undergraduate program; 3.53 percent in the postgraduate program). Tuition and travel costs to Kigali continued to be an obstacle to further education for many respondents working at CHUB and Bushenge hospitals (facility microsystem), including for respondents who had received training under the HRH Program:
I would like to continue studying, but if I get an opportunity to study for a Master’s, even if I can afford the tuition, studying from here cannot work for me. I don’t have accessibility because [the program] is in Kigali, and I can’t study in Kigali while residing here. There may be opportunities to study abroad, but when you work in this [remote] environment, you barely get to hear that information. (68, Nurse Who Had Upgraded from A2 to A1 under the HRH Program)
Reportedly, the Minister of Health had recently announced that clinical staff could not work while pursuing further education, compounding the obstacles to gain more skills and knowledge.
Recruiting and Retaining Existing University of Rwanda Faculty
As reported in the HRH Program midterm review and confirmed in data collection for this evaluation, there was a recognition that not enough
attention had been paid to building current faculty to train future generations of health workers. The midterm review noted there were no formal planning exercises to facilitate the phasing out of USI faculty and ensure a permanent faculty pipeline for a sustainable health system as the program time lines evolved (MOH, 2016). With significant increases in overall enrollment across training programs, the production of health workers was not matched by a Rwandan faculty recruitment plan to sustain production (some programs like obstetrics and gynecology were at risk of falling below baseline enrollment with the departure of USI faculty).
One of the identified challenges was developing a career ladder with incentivizing pay structures for clinicians who train at teaching hospitals without faculty appointments. Health cadre subgroups were intended to review faculty terms of reference, and a key recommendation was for the University of Rwanda to start developing an attractive educator career ladder for all cadres (MOH, 2016). One Government of Rwanda HRH Program administrator (03) cited that career progression and pay structures were developed in 2013 to motivate and incentivize health professionals to be retained. The midterm review recommended that university heads of departments intentionally deploy graduates toward clinical and faculty positions, and identify the number of Rwandan faculty required to provide quality education to the growing student cohorts and graduates to be recruited as faculty. A possible scheme could give honorary or adjunct faculty appointments to clinicians during their 5-year public service contracts before promoting to full-time faculty roles (MOH, 2011b, 2016).
Faculty recruitment was a major issue; for example, the obstetrics and gynecology program had lost and had not recruited any additional Rwandan faculty since 2013, and the capacity to enroll first-year students was at risk of dipping below baseline to only three or four students per cohort without Academic Consortium faculty starting in 2017 (MOH, 2016).
Consortium faculty contributed to updating obstetrics and gynecology guidelines, hosting weekly interdisciplinary morbidity and mortality conferences, and introducing quality measures in 12 district and 4 teaching hospitals to standardize care and prevent maternal deaths (Hill et al., 2015). Long hours, lack of exposure to surgical faculty during medical school training, and heavier workload than nonsurgical physicians without higher pay also have been perceived as barriers to postgraduate recruitment. Furthermore, surgical jobs in the private sector are scarce, unlike for other specialties such as obstetrics and gynecology, where physician specialists can supplement their income by working after hours in private clinics in addition to their public hospital positions (Quinn et al., 2015). Despite political will apparent in the HRH Program investments to develop new surgical programs in neurosurgery, orthopedics, and urology, recruitment
of surgical postgraduates in year 2 of the HRH Program actually decreased from 19 trainees in the first year to 9 trainees (Quinn et al., 2015)
Since the midterm review, University of Rwanda and USI faculty reported the HRH Program had not been successful in increasing motivation and retention of existing University of Rwanda faculty. In fact, although all faculty acknowledged an increase in the number of doctors and nurses created through the Program, they did not see this translating into an increase in existing faculty. One HRH trainee suggested that faculty motivation was reduced owing to the frustrations with this program, to the point where he saw some going into private clinics or leaving the university system altogether:
I think all of the faculty are still there, but they just end [up] filling the roles that they are not prepared for. I think some of the brightest stars from that group go straight from “I just got my Ph.D.” to [pause] deans or whatever; it’s just like again, should be a much longer arc. And then another swath of faculty who are very talented but maybe not tapped for those high-level positions, just kind of idle and doing fine for what the task is at hand, but not really growing in ways that would be helpful for them or for the university…. I don’t see them leaving … but I don’t see them necessarily flourishing in their careers in public health. (64, USI Faculty in Public Health)
The University of Rwanda has tried to retain faculty by offering accredited Ph.D. programs (see “Accreditation and Specialty Programs” in Chapter 5), which have shown early success and interest:
I am doing a Ph.D. and I am not alone. We [have] nursing faculty who are doing their Ph.D.s … this is what the school has been working on. Maybe not through HRH, but through other partnerships and through the university to have people who can sustain what has started so that we don’t go back to where we came from … we know that even a master’s holder who were trained, were skilled can also help in sustaining what is there. So I found that we are having that force, that willingness, good communication, and good leadership that has been developed and implemented through the HRH Program. (80, University of Rwanda Faculty in Nursing)
University of Rwanda faculty and many of its staff attributed some of these challenges to a lack of building institutional capacity into the HRH Program’s original design. Because the Program was designed for only 7
years, it did not allow enough time to train sufficient master’s students to become Ph.D. holders, who would be able to train, mentor, and support future generations of HRH trainees. Consequently, it had implications for recruitment and retention, as well as the long-term sustainability of benefits. In emergency medicine, for example, the first nine participants in the postgraduate diploma course in emergency and critical care medicine enrolled in 2013 (Mbanjumucyo et al., 2015). By 2017, the program was expected to graduate the first 5 emergency physicians, with more than 40 general practitioners completing the postgraduate course (Kabeza et al., 2013); 19 residents were still in the first 3 years of the 4-year program (Yi et al., 2017). The intention was to have enough Rwandan emergency physicians to begin phasing out Consortium emergency medicine faculty to take over training of both tiers by 2019. However, the time required to build a cadre of faculty was not aligned with the pace at which HRH trainees were graduating.
The challenge that still remains is to have Ph.D. holders to run the master’s program. We have just a number of Ph.D. holders, but it is still low and if the program was at least extended for a given period to make sure that the programs are self-sustained, it may be much better. (67, University of Rwanda Faculty and Former Student in Nursing and Midwifery)
Overall Curriculum Developments
Curricula for nursing and midwifery and medical training had developed over the years, with a shift to competency-based curricula and a mix of didactic teaching and more interactive activities becoming evident since the nursing and midwifery education reforms implemented in 2007 (Flinkenflögel et al., 2015c; Uwizeye et al., 2018). For example, flipped classroom models and team-based learning became frequent approaches (Uwizeye et al., 2018).4 Competency-based curricula were seen as tools to move away from a theoretical, hospital-based education to a more practical, patient-centered, and community-oriented education (Muraraneza and Mtshali, 2017). As part of the 2007–2008 curriculum review for undergraduate medical training, modular teaching was introduced in first-
4 In a flipped classroom, students are first exposed to material outside of the classroom, through readings and assignments, and then process the material through discussion in the classroom, while building problem-solving skills (Berrett, 2012).
semester basic courses, such as math and study skills, as well as physiology, which was taught in parallel with anatomy, histology, embryology, and cell/molecular biology and biochemistry. The first semester of the second year included modules on microbiology, pathology, and organ systems (Gahutu, 2010).
In addition to increasing engagement with students, curriculum innovations aimed to increase flexibility and access to education for existing health workers. In particular, e-learning programs for nurses and midwives were designed to rapidly increase the number of qualified health workers without a significant increase in faculty (Harerimana et al., 2016). With sufficient technological infrastructure and institutional support, e-learning can help provide access to education in resource-constrained environments. In Rwanda, it has helped the existing nursing workforce upgrade from A2 to A1,5 increasing the number of qualified nurses in the workforce overall, although effective e-learning would require continuous capacity building and planning (Harerimana and Mtshali, 2018).
Successes in Curriculum Development
Development of curricula or strengthening of existing curricula was reported across all stakeholder groups, as well as specialties interviewed from USIs (pediatrics, obstetrics and gynecology, nursing and midwifery, internal medicine, the MHA, and surgery), who perceived this as a key success of their work:
[W]e definitely created a curriculum for them. There were a lot of new [curricula] that did not exist and a lot of new material did not exist before. That is actually, I think, the legacy of this program, because now they have something they can actually use. What they need to do, they probably need to revisit the curriculum or do a curriculum review on a regular basis, maybe every 3 years or 2 years, so they can see whether the curriculum is up to date or needs modification. (15, USI Faculty in the MHA Program)
Nursing and Midwifery Training
In 2004, the MOH deemed the quality of services delivered by A2 nurses insufficient and initiated plans to cease the A2 training level by 2007 (Mukamana et al., 2015). A0 level nursing, offered only at Kigali Health Institute, was introduced in 2006, with two tracks: a direct entry, 4-year
Bachelor of Science in Nursing degree, and a 2-year Bachelor of Nursing Education degree for nurses upgrading their A1 to A0 (MOH, 2011b; Mukamana et al., 2015). In 2007, the number of A1 nursing programs was reduced to five schools (Byumba, Kabgayi, Kibungo, Nyagatare, and Rwamagana) to focus on the quality of education, but this limited the capacity to admit approximately 100 A2 nurses per year (MOH, 2011b; Munyemana, 2012).
As part of these reforms, the A1 curriculum was revamped to be competency-based and mix didactic learning in the first year with clinical mentorship by year 3 (MOH, 2011a), placing a value on community-oriented training over hospital-based training (Muraraneza and Mtshali, 2017). The IntraHealth Capacity Project’s Learning for Performance: A Guide and Toolkit for Health Worker Education and Training Programs was used to strengthen the HIV/AIDS prevention and treatment, family planning, and gender components of the nursing and midwifery curricula, as part of a 5-year (2004–2009) cooperative agreement (Capacity Project, 2009). In January 2007, the Capacity Project, Belgian Technical Corporation, APEFE,6 and Columbia University provided funding and operational support to the five nursing schools, including for classroom renovations, dormitories, office equipment, computers, Internet, training equipment, technical reference materials, library management training, and updated faculty and clinical preceptors (Capacity Project, 2007, 2009). Despite these efforts, a shortage of A1 nurses persisted and more than 90 percent of Rwandan nurses were still at the A2 level in 2012 (Harerimana et al., 2016).
The HRH Program supported the upgrading of nurses and midwives from A2 to A1 and the development of postgraduate master’s programs for eight nurse specialty tracks (alongside a review of the undergraduate physician training curricula and the launch of six new postgraduate residency programs, discussed later in this section). Development or improvement of curricula were reported across specialties and stakeholders as a main success and contributor of the HRH Program, but one that was dependent on the specialty and department within the University of Rwanda:
I know there were new programs that were created. The curricula were put in place which didn’t exist before in nursing, medicine, and postgraduate programs. (01, Government of Rwanda, HRH Program Administrator)
The HRH Program built on previous nursing training reforms to dramatically increase the nursing and midwifery workforce through the E-Learning Diploma Nursing program, established in 2012, to upgrade A2
6 Association pour la Promotion de l’Éducation et de la Formation à l’Etranger.
nurses to A1 levels, targeted to upgrade 1,500 nurses and midwives by 2020 (Uwizeye et al., 2018). Administered through the University of Rwanda’s Center for Instructional Technology, this competency-based curriculum alternated between face-to-face sessions and self-directed learning. Uwizeye and colleagues (2018) reported that the Government of Rwanda viewed this curriculum as a prime example of adapting to the learning needs of existing health workers who needed to advance their training without leaving their jobs and disrupting the functioning of health centers and hospitals:
Because it was an e-learning program, we did a face-to-face part which didn’t go further than 2 weeks. We returned at work but after work hours, we would work on our online courses. We used a platform called Moodle…. Teachers would upload notes, assignment on there, then we would take exams. We also had a chat room where you could chat with your teacher. We would all meet online at the same hour and whoever had a question could ask the teacher. (68, A0 Nurse Trained under the HRH Program)
In a study assessing the satisfaction levels of full-time undergraduate nursing students in years 2–4 at the University of Rwanda during the 2016–2017 academic year, a majority of students reported high satisfaction with the clinical learning environment (58 percent), the ward atmosphere (54 percent), the leadership of the ward manager (58 percent), and the supervisory relationship (62 percent). However, at every class level and clinical placement, there was need for individual student supervision and improvement to the supervisory relationship (Musabyimana et al., 2019).
No postgraduate programs in nursing and midwifery were available before the HRH Program started. USI faculty collaborated with University of Rwanda faculty to develop curricula for eight master’s-level nursing specialty tracks in pediatrics, critical care and trauma, nephrology, medical–surgical, neonatal, oncology, perioperative, and education leadership and management. Development of these specialty nursing curricula was cited as a significant contribution of the HRH Program:
So, not only they came for the curriculum and improve the existing one, but during that time, especially in the nursing department, they created … a relative[ly] big number of new programs…. So, that was amazing. Not only bachelor’s level nursing program, but also master’s-level nursing program as well. (45, Government of Rwanda, HRH Program Administrator)
The MOH reported that the support of USI faculty also helped develop nursing clinical guidelines for prenatal, newborn, and postpartum history
and physicals, as well as a pre-op checklist, which were implemented in 2013 (MOH, 2016). This is consistent with reports from HRH trainees and University of Rwanda faculty, who noted that updates to curricula and national protocols were conducted with the support of USI faculty. Furthermore, evidence-based practice guidelines were integrated into the didactic teaching curriculum, which emphasized the role of the nurse in implementation of best practices (Relf et al., 2015).
As more nurses were upgraded from A2 to A1, the HRH Program shifted priorities to building a cohort of master’s-level trained nurses, contributing to the skills and quality of care provided by nurses. The 2-year MSN program was launched in 2015 with a first cohort of 121 students, and more than 100 graduated in 2017, as the second cohort entered their second year (MOH, 2016; NYU, 2017). Despite a delay in launching the program, the MOH expanded intake to meet the HRH Program’s overall goal of enrolling 120 MSN-level nurses by 2019, reflecting a deprioritizing of activities to strengthen A0 nursing and midwifery programs in order to develop the master’s programs (MOH, 2016).
Academic Consortium faculty collaborated with Rwandan faculty to develop curricula for the eight master’s-level nursing specialty tracks described above; these areas were selected by the MOH based on morbidity and mortality rates and health needs in Rwanda (Uwizeye et al., 2018). In addition, a “mini-consortium” (the American Association of Critical Care Nurses, the American Nephrology Nurses’ Association, the Association of Perioperative Registered Nurses, the Oncology Nursing Society, and the Academy of Medical-Surgical Nurses) supported the MSN program with membership donations and educational materials, such as core curriculum textbooks for the specialty tracks (Mukamana et al., 2016a,b,c).
Graduate nursing students described learning concepts of global health, research design and biostatistics, leadership and management, nursing theory, and pathophysiology and clinical management before applying their knowledge in a two-part focused clinical, starting in the second semester (Mukamana et al., 2016a). Practicums took place at CHUK, King Faisal Hospital, and Rwanda Military Hospital where, despite reported challenges with limited resources and suboptimal equipment (see also “Equipment Procurement” in Chapter 6), trainees noted benefits from the clinical training and from the experience of mentoring other hospital nurses (Ryamukuru et al., 2018). One emphasis of the MSN program was on molding nursing graduates into “change agents” who could drive policies and develop solutions in the Rwandan health care system (Mukamana et al., 2016b).
Primary data from the University of Rwanda indicate that 82 students graduated from the MSN program. According to one respondent, because the initial focus in nursing was on upgrading A2 nurses to A1 nurses, the
MSN program started “3 years late” (46), which was a barrier to recruiting and graduating even more nurses. The University of Rwanda reported low dropout rates among students in the undergraduate nursing program (1.43 percent), with only slightly higher dropout rates in postgraduate programs (3.53 percent).
Upgrading A2 nurses
Upgrading nurses was seen as essential to building a health workforce capable of addressing the needs of the Rwandan population and viewed as the key for “transform[ing] care for the vast majority of people living with HIV” (12, International NGO Representative). The existing A2 nurses’ limited training hindered their ability to provide direct care without clear direction and supervision:
An A2-level person only does what you tell him/her to do for the patient without enough critical thinking, which means that they are working as robots. (60, A0 Nurse Trained Under the HRH Program)
Even though A1 nurses were trained to provide high-quality services, many Rwandans were seen as still preferring to receive care from doctors:
Being a doctor is the highest degree and in some of us, in our minds, the higher you go, the more knowledgeable you are … we will need time and public health education to convince the entire population that your boss can be a nurse. (10, Professional Association Representative in Obstetrics and Gynecology and Public Health)
The e-learning mechanism offered an opportunity for A2 nurses to upgrade their skills to A1 while continuing to work as A2 nurses. However, one nursing respondent expressed that the quality of students graduating from the A1 programs had declined, due to less emphasis on supervision and follow-up during internships:
They are just putting the students over there in the hospital, and yet there is nobody to follow up [with] them with that responsibility. They are saying that there is someone who is supposed to … [but] the nurse who is in the ward, they have a lot of work and … they are looking to someone who can help in the ward instead of just coming to study…. [T]hat is why internships take longer and the knowledge is lower, as students do not have any follow-up. When nurses see internees, they consider them as help workforces; they start to share their responsibilities with them instead of teaching them first. (60, A0 Nurse Trained Under the HRH Program)
Under the HRH Program, six new postgraduate residency programs in emergency medicine, neurosurgery, orthopedic surgery, pathology, psychiatry, and urology were launched in the CMHS School of Medicine and Pharmacy in 2013. The HRH Program funding proposal had planned to launch eight residencies, including family and community medicine, oncology, neurology, and radiology. However, investments to strengthen internal medicine and general surgery programs were prioritized over family and community medicine, oncology, and neurology before the start of the HRH Program, and the launch of the radiology program was delayed for 2016 (Cancedda et al., 2018; MOH, 2016).
For internal medicine, the HRH Program aimed to improve the integration of didactic modules with clinical rotations through bedside teaching that correlated with competency-based didactic content. The residency program included the establishment of a medical chief resident position to improve the educational and administrative structure (Kabakambira et al., 2017; Walker et al., 2017). Kabakambira and colleagues reported an overall improvement in the collaborative training program, where 73.7 percent of residents perceived the medical chief residents as their role model, with first-year residents (100 percent) being the most enthusiastic about this statement (Kabakambira et al., 2017).
For surgery and anesthesia, curricula aimed to embed more interactive, hands-on sessions, and dedicated consultant–student interaction in operating rooms (Chan et al., 2016). Gaps in this training curriculum for surgery were identified to inform its development by evaluating the operative activity (Rickard et al., 2016). The pediatrics specialty centered on a 2-year didactic core program that included hospital-based learning through morning reports, bedside teaching rounds, simulations and practical skills sessions, morbidity and mortality meetings, research training, and leadership development workshops (McCall et al., 2019). Representatives from professional associations cited that USI faculty were critical in developing clinical content for the curricula, as well as instilling a teaching ethic. In the “other” stakeholder group interviewed for this evaluation, although most may have not had direct involvement with the development of curricula, a select sample were aware that the HRH Program did improve or strengthen curricula.
In addition to the strong emphasis on supporting postgraduate medical education, the curricula for undergraduate medical training was reviewed and several reforms enacted during the HRH Program (Flinkenflögel et al., 2015a). The curriculum was reduced from 6 years to 5 and adapted the CanMEDS physician competency framework to describe core competencies for the “desired Rwandan medical health care provider” (Flinkenflögel et
al., 2015a). Additionally, a new integrated Social and Community Medicine curriculum under the Discipline of Primary Health Care (formerly the family and community medicine department at the College of Medicine and Pharmacy) was structured to cover five domains: public health, health systems, social medicine, communication, and professionalism in years 1–4, culminating in a senior internship in year 5 that would use the primary health care knowledge and skills trainees had gained in previous years (Flinkenflögel et al., 2015a). The curriculum incorporated team-based learning, a flipped classroom model, and online learning platforms (Flinkenflögel et al., 2015b). The design was based on the premise that continued exposure to social medicine concepts would help students gain greater appreciation in their practice, particularly in serving rural populations (Flinkenflögel et al., 2015a; Stevens et al., 2015).
Health Administration and Public Health
Under the HRH Program, the MHA was launched in 2013 in the University of Rwanda School of Public Health after 3 years of curriculum development with Yale University (MOH, 2011b, 2016).7 With the decentralization of health to the district level, health managers trained in public health or management were deemed essential to a functional health facility. The 2-year executive program targeted professionals who were already performing facility management and administration functions in the health system, and focused on increasing the number of trained hospital managers, ensuring equitable distribution of managers, and strengthening institutional capacity to sustain the MHA program locally (MOH, 2016). The curriculum included didactic blocks and hospital rotations, in which students participated in and designed hospital-based or quality improvement projects (MOH, 2016). Program administrators and faculty reported that the curriculum had been adapted from a similar MHA program in Ethiopia, also developed by Yale University (MOH, 2011a,b). USI respondents included the MHA program in their reports, across all specialties, that development or strengthening of curricula was a marked success of the HRH Program. By 2015, 52 trained managers had resulted from the first graduated cohort of the MHA program (MOH, 2016). A Senior Leadership Program also trained 30 senior hospital leaders in 2013.
The MHA program was initially housed in the School of Public Health, but it was shifted to the School of Health Sciences, formerly the Kigali Health Institute. The Vice Chancellor of the University decided to relocate the program because the School of Health Sciences “has this experience working with hospitals … the School of Public Health was not really experienced
with working with hospitals in terms of training and supervision of student trainees in the hospitals” (50, University of Rwanda Administrator in Public Health). The faculty delivering the courses did not change after the program moved, and the School of Health Sciences had no experience with administering a master’s-level program, raising questions among some University of Rwanda respondents about the effectiveness of the move. The change in Minister of Health and the resulting perceived shift in priorities were also seen as contributing to changes in the MHA program’s effectiveness:
[T]his program was created by the former Minister of Health. She sees the need of hospital management or this training. Now, when the new Minister of Health came to the picture, I’m not sure if their objective or mission aligned. (15, USI Faculty in the MHA Program)
An international nongovernmental organization (NGO) respondent felt that additional investments were needed to build a cadre of effective health managers “to manage our health systems better and to have more functional health system and more motivated workers [which] could have really had more transformation across the country” (12, International NGO Representative). However, respondents expressed that the MHA program graduates did effect change in hospital administration, financial management at the hospital level, and quality:
[A] health facility is totally different from a transport company, from a financial company, … because you must know that any delays in implementing anything can affect the health of Rwandan population. So, training them, equipping them with that knowledge of managing medical teams, improving the management finance which links to the health care system was really a big achievement. (03, Government of Rwanda, HRH Program Administrator)
Other initiatives were concurrently being implemented. For example, as of 2017, a Master in Health Supply Chain Management was offered through the East African Community Regional Centre of Excellence for Vaccines, Immunization, and Health Supply Chain Management (RCE-VIHSCM, 2019). The 2-year blended learning program employs face-to-face teaching, distance learning, and multiple field placements for its target cohort of health and supply managers across the East African Community partner countries (RCE-VIHSCM, 2018, 2019). Similarly, building on their existing partnerships, in February 2012 Partners in Health, Harvard Medical School, and Brigham & Women’s Hospital with the MOH launched the first Global Health Delivery course for health professionals in Rwanda, bringing
Harvard Medical School faculty to Rwanda (Cancedda et al., 2014). The course was delivered twice per year in the village of Rwinkwavu, and six local students were trained in the first course to become faculty members for future offerings of this field course (Novak, 2012).
In 2015, the HRH Program also launched the Master in Global Health Delivery program. This 2-year, part-time curriculum emphasized global health policy, management, public health research, health finance, and leadership, and enrolled a first cohort of 27 students from varied disciplines (such as veterinary medicine, agriculture, and NGO management) (MOH, 2016). Partners in Health subsequently established the University of Global Health Equity (UGHE) in 2015 as a private institution to deliver community-centered health care and facilitate retention of qualified health professionals oriented toward serving rural populations (Binagwaho, 2017; UGHE, 2017). The first cohort of 24 students graduated in 2017; the university expected to enroll its first cohort of medical students in its 6.5-year dual degree program in July 2019, with the Master in Global Health Delivery degree woven into its medical training (UGHE, 2017). Although this view was not widely expressed, one USI faculty respondent involved in the design and accreditation of the Master in Global Health Delivery program commented that the UGHE “stole the curriculum from the University of Rwanda and have been running it ever since. The University of Rwanda was left with nothing” (24, USI Twinned Faculty in Internal Medicine and Public Health).
The committee performed comparison analysis, drawing on Program documents, published literature on the Program, and interview data, to examine why the MSN program had achieved greater success and sustainability than the MHA program (see Table 5-1). Both programs were developed under the HRH Program, building on similar programs elsewhere. For example, the MHA program was reportedly based on a similar curriculum in Ethiopia. The quality of the USI faculty and the interest among Rwanda faculty to teach in the MSN program was greater than in the MHA program. Both programs faced challenges with sustainability, and more time is needed to evaluate how these programs will be sustained.
Challenges in Curriculum Support
Despite notable achievements with the curricula for various health professional education programs, USI faculty reported that those who were recruited as USI faculty often did not have experience in curricula development or that if curricula were developed, they were not formally approved. Meanwhile, an HRH trainee reported the need for more revisions to existing curricula.
|New program in Rwanda.|
|Blended program: Students spent 25 percent of their time in Kigali in classroom learning and 75 percent of their time in their hospital applying what they learned. In the early cohorts most students were paired with a USI faculty who would work with them at their hospital; however, this changed after a few years to a more theoretical education with less practical application. (06, USI Faculty in Pediatrics)|
|As initially planned, USI faculty would work alongside hospital administrators in the facility—more practice-based learning on “day-to-day operations”—but eventually a decision was made to develop an “ofﬁcial curriculum for hospital management” in the SPH. However, the SPH “did not know what [health administration] means” so the program was moved to the “School of Health Sciences” in 2014–2015. (15, USI Faculty in the MHA Program) Perception was of miscommunication (“people were not talking on the same level”) when the program was in the SPH, which prompted the move. Vice Chancellor made the decision to move the program from SPH to Health Sciences, but faculty delivering the lectures were still under SPH. SPH had previous experience with postgraduate training, while Health Sciences did not.|
|Adapted from MHA program in Ethiopia: Rwandans were not engaged in the process, but had “ﬁnal say” before submitting it to the Higher Education Council for approval. (15, USI Faculty in the MHA Program)|
|It took about 10 months (2012–2013) for approval from the Higher Education Council.|
|Students were not being recruited at the time of data collection. In the ﬁrst two cohorts, all current hospital administrators were required to participate as a condition of employment though they had to complete an exam and interview to “make sure they meet the academic standards” (50, University of Rwanda Administrator of Public Health and the MHA Program); the program was expanded to include Chief of Nursing—the “idea initially was to train people, three people, at the hospital so if there is any turnover, or there is any trained staff who leave the hospital, they remained with someone to continue working at the hospital.” (50, University of Rwanda Administrator of Public Health and the MHA Program)|
|New program in Rwanda.|
|Blended program: Eight specialties (critical care; education, leadership, and management; medical–surgical; neonatal; nephrology; oncology; pediatric; perioperative) were selected based on morbidity and mortality rates with a focus on upskilling A2 and A1 nurses already working in the health system, at the University of Rwanda, or in undergraduate nursing studies. One block of face-to-face learning was followed by two to three blocks of “hospital attachments,” followed by a Capstone. (79, University of Rwanda Administrator and Faculty in Nursing and Former Student in the MHA Program)|
|The ﬁrst phase of nursing under the HRH Program emphasized upgrading A2 nurses to A1. In 2014, there was a shift in focus to develop an MSN program, with the ﬁrst cohort enrolled in 2015.|
|Adapted from existing curriculum elsewhere: Rwandan and USI faculty collaboratively created curricula in eight specialties, though the extent of curriculum development in each track is unclear.|
|It is unclear whether some/all of the curriculum were approved, but USI respondents were aware of the approval process: “It’s got to go through the Minister of Education and that is going to be painful.” (23, USI Faculty in Nursing)|
|The ﬁrst cohort of students (2015–2017) was chosen from (1) existing University of Rwanda faculty; (2) clinical nurses that had been planning to go abroad for their masters; and (3) existing undergraduate students that wanted to continue their studies. The ﬁrst cohort was fully funded by the HRH Program. The second cohort (2017–2019) was not funded by the HRH Program and was funded by University of Rwanda for their own staff. Private money, other donors, and hospitals were also reported to be used to support the second and third cohorts.|
|Yes: “Usually they had a contract to make sure that the person, when you complete, you are in the hands of the Ministry of Health.” (13, Government of Rwanda, HRH Program Administrator)|
|Perceived Impact of Program on Students|
|Hospital administrators enjoyed the program—they beneﬁted from the training and “having an extra body to teach them what to do and work with them.” (15, USI Faculty in the MHA Program)|
|Perceived Impact of Program on Faculty|
|No apparent impact of the efforts to build the MHA program on MHA faculty could be discerned.|
|Perceived Impact of Program on Quality of Care|
|The MHA program was viewed as contributing to quality improvement projects at hospitals, addressing patient ﬂow, infection control, organization of medical records, and patient satisfaction (81, University of Rwanda Faculty in MHA Program). However, in the ﬁrst few years this seemed to be limited to CHUK and a few district hospitals. (06, USI Faculty in Pediatrics)|
|The MHA program was not seen as having been sustained due to: no University of Rwanda faculty taking over; change in MOH priorities; restructuring of leadership in hospitals, which meant that those who were trained were no longer in leadership positions; and not being as well supported as other areas of the HRH Program.|
|Future of the Program|
|There was hope the MHA training would be expanded to health center managers.|
NOTE: CHUK = Centre Hospitalaire de Kigali/The University Teaching Hospital of Kigali; HRH = human resources for health; MHA = Master of Hospital and Healthcare Administration; MOH = Ministry of Health; MSN = Master of Science in Nursing; SPH = School of Public Health; USI = U.S. institution.
|No bonding period by the MOH was mentioned by nursing staff, faculty, or students. However, some speciﬁc hospitals or private institutions may have individual policies. The University of Rwanda may also have a policy if the MSN was paid for by the university.|
|Overall there were clear examples of beneﬁts to MSN students, both in upgrading their skills and being on par with the rest of the region. Many reported that others from the region are now interested in the MSN program.|
|MSN students were able to take over administration of the MSN program, which may indicate sustainability.|
|The MSN program was seen as producing specialized nurses who could provide more complex care and could train others working in their department.|
|Faculty teaching in a master’s program were required to have a Ph.D., and there was concern about whether there would be enough Ph.D.-level staff to train the next cohort of students, though a new Ph.D. program in nursing reportedly began in 2017–2018 (this could not be conﬁrmed).|
|The future plans of the MSN program are unclear. However, it does seem that the twinning model switched from 2-1 U.S. to Rwandan faculty to 2/3-1 Rwandans to U.S. faculty. Additionally, there has been a reduction in the number of students accepted.|
The first year of the HRH Program we revised completely the EM-MED [emergency medicine], which is the residency program curriculum. That revision was never truly approved. We have been working the whole HRH Program through this draft. It is complicated to get it approved [at] the University of Rwanda level…. It never made it all the way up, probably because our head of departments have never pushed this as priority, because they are also overwhelmed and don’t have any admin[istrative] support…. So, the curriculum was revamped, the rotation style was revamped, the evaluation was revamped and changed. (06, USI Faculty in Pediatrics)
For the MHA program, the curriculum that was developed underwent a lengthy process for approval and was reportedly copied, as noted in the previous section.
Indeed, use of the HRH Program–supported curricula seemingly expanded beyond the University of Rwanda and the HRH Program to UGHE, other established schools of medicine or health professional education institutions, and the East African Community for accreditation. One USI faculty member reported taking the curriculum they had developed with them to a similar program in another country in the region. As described previously, this expansion had mixed reviews from evaluation respondents.
Although it took time for different institutions and the East African Community to develop, adapt, and approve curricula, evaluations of the curricula were very positive:
[The HRH Program] did a wonderful job, because curricula was not existing in country because no one was trained to be a specialist here in Rwandan. So, at least you should recognize that achievement is really linked to the vision of HRH, because curricul[a] were developed for different specialists in different domains in Rwanda…. [W]e can really prepare for candidates that will be in the training for specialization. So today you have new faculty of medicine coming in country the Global Health University, and another one with the Seventh Day Adventist Church … coming with an established school of medicine and [they] are using curricul[a] that are here in country and they were developed and refined by HRH. (65, Government of Rwanda, HRH Program Administrator)
Alongside assessment of academic curricula, accreditation processes and standards were also widely discussed in the region. Mullan and colleagues state that in a survey of 146 medical schools, there was “little use
the most common approach to accreditation of institutions and programs is the process model that includes self-evaluation based on agreed standards; a peer-review that usually includes a site visit and a report indicating the outcome of the accreditation.… However, in more than half the countries of the world, reviews of schools and programs are not done at all or not adequately.
However, it should be noted that “there is only weak evidence of a causal link between accreditation and higher quality” (WHO, 2013).
Respondents in this evaluation highlighted improved institutional capacity within the University of Rwanda due to the HRH Program with regard to accreditation at the university level and advances in each specialty program. One University of Rwanda faculty member attributed the university’s most recent accreditation by the East African Accreditation Board partly to the HRH Program and suggested that improvements in the quality of education were linked with this accreditation:
[T]he program has been upgraded…. It has been accredited. So now they are doctors, graduated, and can go work in all Eastern African countries. So many have left to go to work in East Africa. Now we have international recognition because they have a standard training, which has really benefited from the HRH support … that was also a success from the [University of Rwanda]. To be able to have a structure, organized, which can be confirmed by the accreditation Board of the East African Community. (14, University of Rwanda Administrator in Pediatrics)
However, East African Accreditation Board accreditation could not be verified. All residencies across all disciplines were university-based, and graduates received a Master in Medicine (M.Med.) from the Ministry of Education (MOE). All HRH programs were approved by the University of Rwanda Senate, Higher Education Council, under the MOE. However, different specialties also reported varying additional approvals. The nursing program was also approved by the Nursing Council for Nurses and Midwives.8 Surgeons looked to a separate, nonuniversity body, the College of Surgeons of East, Central, and Southern Africa (COSECSA), and University of Rwanda surgical graduates were encouraged to sit for the COSECSA exams to become fellows—FCS(ECSA).
8 Levels A2 and A1 were both approved by the MOE, but A2 no longer exists.
In 2014, the Rwanda Emergency Care Association was established as the first interdisciplinary organization bringing together emergency medicine physicians, residents, emergency nurses, and prehospital providers in Rwanda, as a result of introducing the postgraduate diploma in emergency medicine that evolved into the Emergency M.Med. (MOH, 2016; RECA, 2019). The launch of the Emergency M.Med. program also led to the development of the Emergency Medicine Clinical Guidelines in 2016 to standardize care at district and referral hospitals, the introduction of the first formal emergency triage system in Rwanda’s public health system, and restructuring of the CHUK Emergency Department (MOH, 2016). HRH Program-purchased equipment specific to emergency medicine, such as resuscitation equipment and portable x-ray machines at CHUK, was anticipated to facilitate a high-quality training environment for the new residents (MOH, 2011b).
Data from this evaluation show that the HRH Program also increased HRH trainees’ research outputs, professional development opportunities, and overall preparation and motivation as they entered the health workforce.
Support for Research Outputs
The 2011 HRH Program proposal stated that “research undertaken related to the HRH Program by U.S. faculty must be co-authored with Rwandan researchers and must adhere to the conditions established by the Rwandan Authorities” (MOH, 2011b). According to an evaluation of a partnership program between the University of Rwanda and Swedish universities, which was funded by the Swedish International Development Cooperation Agency (Sida), Rwanda’s research output has grown by an average of 26 percent annually since the early 2000s, with medical research constituting almost half of the country’s more recent output (Tvedten et al., 2018). U.S. universities and medical institutions “dominate” the list of affiliated international partner institutions for research articles with Rwandan authors and co-authors published between 2013 and 2017 (Tvedten et al., 2018). Although this Sida-funded evaluation credited the HRH Program as a driving factor in these research partnerships, two of these institutions were not involved in the Program and two had multiple longstanding relationships with the MOH and other Rwandan institutions that predated the HRH Program; it is more likely that the Program was one among many factors supporting Rwanda’s increased research output during this period (Tvedten et al., 2018).
A more recent review of the literature, concurrent with the timing of the HRH Program, notes similar findings with respect to overall Rwandan re-
search output. A Scopus review of published primary literature,9 performed as part of this evaluation, found 1,626 articles in the fields of medicine and health professions, nursing, public administration, public health, social sciences, and life sciences published between 2012 and 2018 by authors affiliated with Rwandan institutions.10 Moreover, annual research output among Rwandan-affiliated authors grew at an average of 20 percent per year, nearly tripling during this period (from 117 articles in 2012 to 334 in 2018). Of the total research output for Rwandan-affiliated authors, nearly half (782) of all publications were in the fields of medicine and health professions, and 320 publications focused on HIV. Rwandan-affiliated authors produced on average 46 HIV-focused publications annually during this period.
Average annual growth in HIV-focused publications was slightly negative, however, with 2015 as the year of peak output. Further analysis indicated that for more than one-third of these HIV articles, a Rwandan-affiliated author was listed as the first author; moreover, among these first authors, fully 40 percent were serving as lead author for the first time. Across all 320 HIV-focused publications with Rwandan-affiliated authorship, 14 percent had a lead Rwandan-affiliated author serving in this role for the first time. Although this increased research output cannot be attributed to the HRH Program, it is likely that the Program was one of multiple contributing initiatives.
Metrics such as “first author” and “first-time first author” are important to consider, because much of the research in Rwanda is both funded by foreign donors from higher-income countries and conducted in partnership with academic and other institutions from these countries. The literature indicates that such partnerships “are often imbalanced,” fostering the “careers and priorities” of Western researchers over those of their African collaborators (Boum et al., 2018). A higher proportion of Rwandan first authors and first-time first authors may indicate a more equitable partnership. This is desirable, as more equal and “truly cooperative” (Costello and Zumla, 2000) research partnerships are more likely to foster the development of academic infrastructure within the country and result in the translation of research into practice and national policy (Boum et al., 2018; Costello and Zumla, 2000).
Across all respondents for this evaluation, one of the main reported outcomes of the HRH Program was the development of HRH trainees’ skills and competencies in research, grant writing, and publication. A focus on research
9 Primary literature is defined as literature that excludes the Scopus-defined categories of “editorials,” “notes,” and “letters.”
10 Authors affiliated with Rwandan institutions, also referred to as Rwandan-affiliated authors, indicate authors who are listed in a publication as being affiliated with an institution that is based in Rwanda.
was emphasized in some curricula, such as the MHA program and pediatrics specialty, whose graduation requirements included a built-in requirement to integrate research and write a thesis. HRH trainees from the facility microsystem reported that increased research collaboration with USI faculty increased their understanding of data, statistics, and evidence-based practices:
I was trained how to do research and how to write a manuscript and how to produce an article to publish, and before I was not aware of it. Now … I can write a research project in 6 months and it is completed; before I would not even think about how do it. So, my research activities [were] improved. (49, University of Rwanda Faculty and Current Student in Nursing)
Respondents reported that research collaborations, publications and mentorship with HRH trainees continued after USI faculty left Rwanda (see Chapter 4 for details, in the context of the HRH Program’s twinning activities). Many HRH trainees are now pursuing their Ph.D.s, and have contributed their research knowledge to the HRH Program.
People continue doing research together, publishing, even if they are no longer part of the program. Do you know that in the University of Rwanda—I think that was last year it came out this information?—from 2015 to 2018, think we had 700 and something articles published in good journals. (37, University of Rwanda Administrator)
USI faculty continued collaborating with students they had advised after their departure from Rwanda, predominantly in supporting students to complete their theses and prepare and submit publications.
Other support for research outputs included a 6-month scholarly writing workshop, designed and implemented by the HRH Program from January to July 2015 with funds from the Wellcome Trust, to help University of Rwanda faculty learn the process and strategies for preparing manuscripts for publication and other research activities (Ewing et al., 2017). Participants of the writing workshop met every 2 weeks and were offered mentorship and supportive feedback, with the goal of producing a research-related output by the end of the workshop. Although low post-test responses and a high workshop attrition rate led to no significant conclusions, Ewing and colleagues report a high level of overall satisfaction with the workshop; 77 percent also self-reported they had developed an abstract or draft manuscript during the workshop, 69 percent planned to publish in the next year, and 85 percent noted they planned to deliver a presentation at a conference within the next year (Ewing et al., 2017).
Increased research also led to increased opportunities to present research at regional and international conferences. A few USI faculty reported supporting HRH trainees and twins to present at a conference, with additional funding raised outside of the HRH Program. Across respondents, there were multiple examples of HRH trainees speaking and attending international conferences in the United States, the United Kingdom, and regionally.
A few HRH trainees reported an increase in research on HIV. One pediatric HRH trainee presented HIV research on antiretroviral therapy adherence at an international conference. However, according to the Scopus review of published primary literature, the number of HIV-related primary publications by authors affiliated with Rwandan institutions rose from 45 in 2012, peaked at 57 in 2015, then dropped each subsequent year to 40 publications in 2018.
Support for Continuing Professional Development
Professional councils have supported continuing professional development (CPD) across cadres in Rwanda for a decade. In February 2009, the Rwanda Medical and Dental Council established a CPD program. The National Council for Nurses and Midwives followed with its own CPD strategic plan in June 2013. Subsequently, four professional councils (Rwanda Medical and Dental Council, National Council for Nurses and Midwives, National Pharmacy Council, and Rwanda Allied Health Professionals Council) established a joint CPD policy, requiring health professionals to continually update their skills and knowledge as a condition of licensure (Health Professional Councils of Rwanda, 2013). International NGOs also reported working to strengthen professional councils and associations:
We’ve been working on a number of different initiatives particularly in this project. One is strengthening the professional association or professional councils in their regulatory role. You know, we’ve been supporting them for data management so they can put systems in place so they can license and register people, things like that. So mainly working with the nurses and midwives’ council, and medical council. And then there’s allied professional, is another group. (05, Other International NGO)
Each profession has a minimum number of credits required every 3 years (see Table 5-2). Being the first author on a scientific paper or article in a peer-reviewed journal is worth 20 credits (Health Professional Councils of Rwanda, 2013).
|Profession||CPD Credits Required Every 3 Years|
|Allied Health Professionals||90|
NOTE: CPD = continuing professional development.
HRH trainees from the facility microsystem and the University of Rwanda reported CPD activities hosted by the MOH, Rwanda Biomedical Center, and professional associations as critical to their development. At the facility level, HRH trainees reported that the MOH deployed 2-day to 3-day CPD training courses on topics including tuberculosis, malaria, and hepatitis. Professional associations played an important role in setting the policy on CPD as well as delivering the courses. One respondent reported a new five-module certificate course for midwives, which trainees could present to the National Council for Nurses and Midwives for renewal of their license. One respondent speculated that CPD activities motivated local health staff to further pursue training in subspecialties as they witnessed HRH staff pursuing such opportunities (28, University of Rwanda Administrator and Former Student in Obstetrics and Gynecology).
Since that time that [the] HRH [Program] started, it was a privilege, not only [for] clinics and not only [for] nursing, but even [for] medical staff. So, we lived with them in the clinics…. They were helping us in skills development, in coaching, training, CPD. They were helping us to get CPD because they prepared the lessons so that they help us to refresh our daily practice. In clinics it was about skills development, coaching, and CPD. (62, University of Rwanda Former Student in Nursing)
Aside from formal CPD activities, respondents spoke of more informal ways the HRH Program advanced their careers. For example, one respondent who had formerly worked with the MOH as part of the HRH Program attributed her exposure to health care improvement through the Program as something she carried to her community-level work with people living with HIV (PLHIV).
Many HRH trainees were fast-tracked to higher positions following their training, not only as faculty and deans of departments at the Uni-
versity of Rwanda, but also at the facility level. One respondent discussed being promoted to head of her department after HRH training, and another spoke of moving from being a nurse to head of department:
Today I am the matron of internal medicine and I have been working as such for 1 year, but from 2011 up to 2013 I was normal nurse working in surgery, from 2012 to 2013 I was the coordinator of infection control and prevention, from 2013 to May 2018 I was the matron in surgery department. (58, University of Rwanda Former Student in Nursing and A0 Nurse)
Another nurse respondent reported not feeling well prepared to provide clinical services immediately following her pre-service education, but that working alongside doctors following her placement had allowed her to learn practical applications of her knowledge. Another respondent highlighted that some HRH Program colleagues had continued to advance their education outside of the health system and outside of Rwanda:
I know most of my colleagues I went to medical school with … some of them have even moved on to become consultants and some of them have used the HRH Program to advance their skills and are now in better positions even in the U.S. and elsewhere. All of this is thanks to the HRH Program. (48, Government of Rwanda HRH Program Administrator)
As many graduates under the Program were still bound by the bonding period,11 and other cohorts were still in their studies at the time of this evaluation, the movement of HRH Program trainees could not be assessed. It will be important to monitor the flow of HRH Program trainees over time to determine any resulting brain drain. Chapter 6 expands on student and faculty advancements.
In reexaming health professional education, a 2010 Lancet commission identified a series of instructional and institutional reforms for postsecondary education in medicine, nursing, and public health to be not only responsive to local needs, but also connected globally and extended beyond the silos of individual health professions. These reforms provided a set of
11 Students who received tuition support from the MOH were bonded, or required to serve for a certain number of years depending on cadre and specialty, in the public health system before being able to pursue employment opportunities outside of the public sector.
components to transform education and strengthen country health systems, most of which the HRH Program addressed in some capacity. Examples in instructional reforms include
- adapting to local contexts “while using global knowledge, experience, and shared resources, including faculty, curriculum, didactic materials, and students linked internationally through exchange programs”;
- strengthening educational resources such as faculty, syllabuses, didactic materials, and infrastructure;
- promoting “—interprofessional and transprofessional education that breaks down professional silos while enhancing collaborative and non-hierarchical relationships in effective teams”; and
- using “the power of information technology for learning” (Frenk et al., 2010).
Examples in institutional reforms include
- establishing joint planning mechanisms “—to engage key stakeholders, especially ministries of education and health, professional associations, and the academic community, to overcome fragmentation by assessment of national conditions, setting priorities, shaping policies, tracking change, and harmonizing the supply of and demand for health professionals to meet the health needs of the population;” and
- “[l]inking together through networks, alliances, and consortia between educational institutions worldwide and across to allied actors, such as governments, civil society organisations, business, and media” (Frenk et al., 2010).
At the outset of the HRH Program, there was an underestimation of the degree of structural change within the University of Rwanda, and across sectors, that was needed for institutionalized capacity for health professional education and that would occur as a consequence of the HRH Program. The perception of its success was first in training health workers, second in augmenting service delivery, and third in building the University of Rwanda faculty capacity in teaching. The institutional reform of greater joint planning among ministries was an outcome that emerged over time. Beyond the relationships with USIs, expanding linkages with regional and global networks was not an emphasis of the Program. Literature on the HRH Program since 2015 has documented processes and outputs primarily in the following domains: trainees’ perceptions of HRH Program curricula or activities; measurements of knowl-
edge acquisition or teaching and clinical skills transfer among HRH trainees; factors affecting recruitment of trainees and faculty; and general gaps and improvements to the HRH Program curricula.
Development of University of Rwanda faculty capacity occurred through a number of activities, including mentoring and twinning, as discussed in Chapter 4. In a faculty development program for medical education in the East African region, implementation of varied activities (including faculty development workshops, visiting professor programs, mentorship exchange visits, advanced leadership training, and curriculum development) provided positive developments in strengthening health professional education at the college as faculty enhanced their expertise and skills (Matsika et al., 2018). Bringing in external faculty and other experts via memoranda of understanding with USIs added value to the quality of health professional education and training, as determined by the development of new programs and curricula or the updating of existing curricula, increased research output, and the provision of high-quality teaching by USI faculty. As illustrated by some of the instructional reforms above (Frenk et al., 2010), this was a key in strengthening and developing the capacity of health professional education. Exposure to the twinning model, USI faculty, and teaching quality also had a positive impact on Rwandan faculty and students, providing a mechanism for those trained to take on leadership roles in providing quality of care and to train the next generation of health professionals in Rwanda; this resonates with the notion that twinning is a values-based methodology (Kelly et al., 2015).
Although there were successes in producing an increased number of health workers across cadres and specialties, the story is less clear in the production and retention of University of Rwanda faculty to educate future cohorts of students. Theoretical frameworks addressing worker turnover and retention point to the idea that for faculty, structural factors within an academic institution (e.g., “collegial communication, equitable rewards, work autonomy, job security, and a role in organizational decision making”) may influence retention (Ngure and Waiganjo, 2017). Faculty members who perceive the institution as meeting their expectations in these areas may be more likely to “report higher levels of job satisfaction and stronger commitment to the employing organization,” thereby “strengthening the[ir] intent to stay” in their position (Ngure and Waiganjo, 2017). In contrast, for faculty with unmet expectations in these areas, “levels of satisfaction and commitment decline and intent to leave increases” (Ngure and Waiganjo, 2017). Thus, faculty perceptions surrounding these structural factors may affect “psychological dispositions toward staying or leaving the institution” (Ngure and Waiganjo, 2017). This evaluation’s proximity to the end of PEPFAR’s investments make it difficult to say conclusively that additional faculty have been produced or
retained; however, the qualitative data do point to organizational limitations at the University of Rwanda level that could hinder faculty retention, including career progressions and compensation.
In addition to the value added to the quality of health professional education via new and upgraded education programs and curricula, research output continued to increase. Research capacity was highlighted as a key priority in the original proposal, although there was no clear funding mechanism or pathway. Given the number of strong institutional research collaborations that predated the HRH Program (such as the University of Rwanda–Sweden program, which provided research grants and funded doctoral students, and Harvard University and its affiliate institutions, which had longstanding partnerships with Partners In Health and the MOH), the HRH Program was likely one of many efforts contributing to a steadily growing body of research output during this period. For HRH trainees, the Program specifically increased their research outputs, professional development opportunities, and overall preparation and motivation as they entered the health workforce, although a cross-sectional descriptive study of interns and pediatric residents at the University of Rwanda identified “barriers to research such as faculty lacking time to mentor, lack of funding, lack of statistical support, and lack of faculty experienced in conducting research” (Habineza et al., 2019). As a successful example, the institutional and long-term partnership between Makerere University in Uganda and Karolinska Institute in Sweden, which started in 2000, successfully bolstered research capacity and collaboration. While that program’s focus was on a joint Ph.D. program, students spent a majority of their time in Uganda to ensure research remained focused on local issues, with the remainder spent in Sweden, where they enrolled in specialty courses. The productive nature of this collaboration and focus had positive implications in terms of publications, increasing research output with a majority of Ugandan first or last authors (Sewankambo et al., 2015).
Improvements in the accreditation of programs developed or improved under the HRH Program were uneven (Rogo, 2019). Accreditation standards differed by specialty, with varying levels of subsequent approval, such as those for the nursing program or the surgery specialty, for which additional approvals were required by a Council or separate, nonuniversity bodies. Although the University of Rwanda’s medical, dentistry, and pharmacy programs were noted as participating in the regional East African Community accreditation process, their regional accreditation status could not be independently verified (Yumbya, 2019). In the East African Community, under a common supranational inspectorate framework, programs currently undergo two layers of accreditation, first
at the national level and then at the regional level. At this regional level, there is a harmonization of curricula and a formal joint inspection by regulators representing the six countries in the East African Community (Rogo, 2019).
Although experiences with twinning were positive, as described in Chapter 4, the HRH Program’s individual twinning approach, as opposed to institutional twinning, resulted in limited improvements in institutional capacity for health professional education. During the period of PEPFAR’s investment, the HRH Program did not place adequate emphasis on building and retaining faculty, and insufficient time was dedicated to building a cohort with terminal degrees to fill gaps in faculty following the reduction in numbers of USI faculty. The result was new programs and curricula, but without sufficient University of Rwanda faculty to deliver the courses. There was an initial pipeline of trainees feeding back into the education and training systems as faculty and administrative leaders, but retention strategies for current and new faculty were an underdeveloped aspect of the HRH Program’s portfolio of activities. Following the end of PEPFAR investments, the University of Rwanda did not have sufficient resources to provide the same volume of high-quality postgraduate teaching; this manifested in large class sizes, insufficient infrastructure, and low salaries that challenged retention (see Chapter 6 for more information on how salaries affect retention). However, the improved relationship and communication between the MOH and the MOE, as well as with professional associations and professional councils, as a result of the HRH Program could provide momentum to continue building institutional capacity.
Berrett, D. 2012. How “flipping” the classroom can improve the traditional lecture. https://www.chronicle.com/article/How-Flipping-the-Classroom/130857 (accessed January 31, 2020).
Binagwaho, A. 2017. University of Global Health Equity’s contribution to the reduction of education and health services rationing. International Journal of Health Policy and Management 6(8):427–429.
Boum II, Y., B. F. Burns, M. Siedner, Y. Mburu, E. Bukusi, and J. E. Haberer. 2018. Advancing equitable global health research partnerships in Africa. BMJ Global Health 3(4):e000868.
Cancedda, C., P. E. Farmer, P. Kyamanywa, R. Riviello, J. Rhatigan, C. M. Wagner, F. Ngabo, M. Anatole, P. C. Drobac, T. Mpunga, C. T. Nutt, J. B. Kakoma, J. Mukherjee, C. Cortas, J. Condo, F. Ntaganda, G. Bukhman, and A. Binagwaho. 2014. Enhancing formal educational and in-service training programs in rural Rwanda: A partnership among the public sector, a nongovernmental organization, and academia. Academic Medicine 89(8):1117–1124.
Cancedda, C., P. Cotton, J. Shema, S. Rulisa, R. Riviello, L. V. Adams, P. E. Farmer, J. N. Kagwiza, P. Kyamanywa, D. Mukamana, C. Mumena, D. K. Tumusiime, L. Mukashyaka, E. Ndenga, T. Twagirumugabe, K. B. Mukara, V. Dusabejambo, T. D. Walker, E. Nkusi, L. Bazzett-Matabele, A. Butera, B. Rugwizangoga, J. C. Kabayiza, S. Kanyandekwe, L. Kalisa, F. Ntirenganya, J. Dixson, T. Rogo, N. McCall, M. Corden, R. Wong, M. Mukeshimana, A. Gatarayiha, E. K. Ntagungira, A. Yaman, J. Musabeyezu, A. Sliney, T. Nuthulaganti, M. Kernan, P. Okwi, J. Rhatigan, J. Barrow, K. Wilson, A. C. Levine, R. Reece, M. Koster, R. T. Moresky, J. E. O’Flaherty, P. E. Palumbo, R. Ginwalla, C. A. Binanay, N. Thielman, M. Relf, R. Wright, M. Hill, D. Chyun, R. T. Klar, L. L. McCreary, T. L. Hughes, M. Moen, V. Meeks, B. Barrows, M. E. Durieux, C. D. McClain, A. Bunts, F. J. Calland, B. Hedt-Gauthier, D. Milner, G. Raviola, S. E. Smith, M. Tuteja, U. Magriples, A. Rastegar, L. Arnold, I. Magaziner, and A. Binagwaho. 2018. Health professional training and capacity strengthening through international academic partnerships: The first five years of the Human Resources for Health Program in Rwanda. International Journal of Health Policy and Management 7(11):1024–1039.
Capacity Project. 2007. Voices from the capacity project: Developing the health workforce—training future nurses and midwives in Rwanda. https://www.intrahealth.org/features/ voices-capacity-project-developing-health-workforce%E2%80%94training-futurenurses-and-midwives (accessed November 26, 2019).
Capacity Project. 2009. Planning, developing and supporting the health workforce: Results and lessons learned from the Capacity Project, 2004-2009. Washington, DC: United States Agency for International Development.
Chan, D. M., R. Wong, S. Runnels, E. Muhizi, and C. D. McClain. 2016. Factors influencing the choice of anesthesia as a career by undergraduates of the University of Rwanda. Anesthesia and Analgesia 123(2):481–487.
Costello, A., and A. Zumla. 2000. Moving to research partnerships in developing countries. BMJ 321(7264):827–829.
Cubaka, V. K., P. Kyamanywa, M. Flinkenflögel, M. Schriver, E. Ngabire, and P. Cotton. 2015. Lessons from the field: Transforming health professionals’ education in Rwanda—contributions from social and community medicine. Rwanda Journal 2:93.
Ewing, H., R. Pyburn, D. Asingizwe, B. A. Asiimwe-Kateera, K. Njunwa, and M. J. Kim. 2017. Facilitating health professional scholarship through a workshop series at the University of Rwanda. Rwanda Medical Journal 74(2):12–16.
Flinkenflögel, M., V. K. Cubaka, M. Schriver, P. Kyamanywa, I. Muhumuza, P. Kallestrup, and P. Cotton. 2015a. The desired Rwandan health care provider: Development and delivery of undergraduate social and community medicine training. Education for Primary Care 26(5):343–348.
Flinkenflögel, M., P. Kyamanywa, B. Asiimwe-Kateera, S. Musafiri, P. Kayumba, M. Irakoze, P. Cotton, and A. Hibble. 2015b. Umuganda for improved health professions education in Rwanda: Past, present and future in the training of health professionals at the University of Rwanda. Rwanda Journal 2:96.
Flinkenflögel, M., P. Kyamanywa, V. K. Cubaka, and P. Cotton. 2015c. The next generation of Rwandan physicians with a primary health care mindset. African Journal of Primary Health Care and Family Medicine 7(1).
Frenk, J., L. Chen, Z. A. Bhutta, J. Cohen, N. Crisp, T. Evans, H. Fineberg, P. Garcia, Y. Ke, P. Kelley, B. Kistnasamy, A. Meleis, D. Naylor, A. Pablos-Mendez, S. Reddy, S. Scrimshaw, J. Sepulveda, D. Serwadda, and H. Zurayk. 2010. Health professionals for a new century: Transforming education to strengthen health systems in an interdependent world. Lancet 376(9756):1923–1958.
Gahutu, J. B. 2010. Physiology teaching and learning experience in a new modular curriculum at the national University of Rwanda. American Journal of Physiology—Advances in Physiology Education 34(1):11–14.
Habineza, H., C. Nsanzabaganwa, N. Nyirimanzi, C. Umuhoza, K. Cartledge, C. Conard, and P. Cartledge. 2019. Perceived attitudes of the importance and barriers to research amongst Rwandan interns and pediatric residents—a cross-sectional study. BMC Medical Education 19(1).
Harerimana, A., and N. G. Mtshali. 2018. Implementing e-learning in resource-constrained nursing education institutions in Rwanda. Journal of Nursing and Health Sciences 4(1).
Harerimana, A., N. G. Mtshali, H. Hewing, F. Maniriho, E. Kyamusoke, A. Mukankaka, E. Rukundo, S. Gasurira, D. Mukamana, and J. Mugarura. 2016. E-learning in nursing education in Rwanda: Benefits and challenges. An exploration of participants’ perceptives. IOSR Journal of Nursing and Health Science 5:64–92.
Health Professional Councils of Rwanda. 2013. Continuing professional development (CPD) policy for health professional councils in Rwanda, August. Kigali, Rwanda: Health Professional Councils of Rwanda.
Hill, W., D. Ellis, C. Kress, S. Washington, D. Kitessa, S. Erber, C. C. Hays, S. Rulisa, and P. Bagame. 2015. The role, opportunities, accomplishments and implications worldwide of the obstetrician-gynecologist in the Human Resources for Health (HRH) Program in Rwanda. International Journal of Gynecology and Obstetrics 131:E100.
Kabakambira, J. D., R. Mugeni, J. Ho, A. N. Sofair, and T. Walker. 2017. Improvement of medical education through the establishment of a medical chief resident position in Rwanda. Rwanda Medical Journal 74(2):17–20.
Kabeza, A. B., N. George, M. Nyundo, and A. C. Levine. 2013. Development of emergency medicine in Rwanda. African Journal of Emergency Medicine 3(3):103–109.
Kelly, E., V. Doyle, D. Weakliam, and Y. Schönemann. 2015. A rapid evidence review on the effectiveness of institutional health partnerships. Global Health 11(1):48.
Matsika, A., K. Nathoo, M. Borok, T. Mashaah, F. Madya, S. Connors, T. Campbell, and J. Hakim. 2018. Role of faculty development programs in medical education at the University of Zimbabwe College of Health Sciences, Zimbabwe. Annals of Global Health 84:183–189.
Mbanjumucyo, G., E. DeVos, S. Pulfrey, and H. M. Epino. 2015. State of emergency medicine in Rwanda 2015: An innovative trainee and trainer model. International Journal of Emergency Medicine 8(1).
McCall, N., C. Umuhoza, C. O’Callahan, T. Rogo, D. Stafford, A. Kanyamuhunga, and P. T. Cartledge. 2019. Measuring change in knowledge acquisition of Rwandan residents: Using the American Board of Pediatrics International In-Training Examination (I-ITE) as an independent tool to monitor individual and departmental improvements during the Human Resources for Health Program: An observational study. BMC Medical Education 19(1).
MOH (Ministry of Health). 2011a. Human Resources for Health strategic plan. Kigali, Rwanda: Ministry of Health.
MOH. 2011b. Rwanda Human Resources for Health Program, 2011-2019: Funding proposal. Kigali, Rwanda: Ministry of Health.
MOH. 2016. Rwanda Human Resources for Health Program midterm review report (2012-2016). Kigali, Rwanda: Ministry of Health. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)
Mukamana, D., G. Uwiyeze, and A. Sliney. 2015. Nursing and midwifery education in Rwanda: Telling our story. Rwanda Journal 2(2).
Mukamana, D., V. Dushimiyimana, J. Kayitesi, G. Mudasumbwa, J. Nibagwire, F. Ngendahayo, M. J. Tuyisenge, O. G. Uwambajimana, and W. Rosa. 2016a. Nephrology nursing in Rwanda: Creating the future through education and organizational partnership. Nephrology Nursing Journal 43(4):311–315.
Mukamana, D., G. R. Karonkano, and W. Rosa. 2016b. Advancing perioperative nursing in Rwanda through global partnerships and collaboration. AORN Journal 104(6):583–587.
Mukamana, D., A. Niyomugabo, and W. Rosa. 2016c. Global partnership advances medical-surgical nursing in Rwanda. Med-Surg Nursing Connection, June.
Mullan, F., S. Frehywot, F. Omaswa, E. Buch, C. Chen, S. R. Greysen, T. Wassermann, D. E. Elgaili Abubakr, M. Awases, C. Boelen, M. J. M. I. Diomande, D. Dovlo, J. Ferro, A. Haileamlak, J. Iputo, M. Jacobs, A. K. Koumaré, M. Mipando, G. L. Monekosso, E. O. Olapade-Olaopa, P. Rugarabamu, N. K. Sewankambo, H. Ross, H. Ayas, S. B. Chale, S. Cyprien, J. Cohen, T. Haile-Mariam, E. Hamburger, L. Jolley, J. C. Kolars, G. Kombe, and A. J. Neusy. 2011. Medical schools in sub-Saharan Africa. Lancet 377(9771):1113–1121.
Munyemana, G. 2012 (unpublished). Harmonizing capacity building and work responsibilities of Rwandan nurses through e-learning. Kigali, Rwanda: National University of Rwanda.
Muraraneza, C., and G. N. Mtshali. 2017. Conceptualization of competency based curricula in pre-service nursing and midwifery education: A grounded theory approach. Nurse Education and Practice 28:175–181.
Musabyimana, C., J. N. Mukankusi, T. Nyandwi, J. Mugarura, and A. Collins. 2019. Clinical learning environment and supervision: Satisfaction levels of University of Rwanda students. Rwanda Journal of Medicine and Health Sciences 2(2).
Ndayambaje, A., R. Anderson, C. M. Yoder, H. Ewing, and D. R. Thomson. 2017. Human Resources for Health (HRH) midwives mentoring program and episiotomy rates at Muhima Hospital, Rwanda: A retrospective cross-sectional study. Rwanda Medical Journal 74(2):7–11.
Ngure, K. P., and E. Waiganjo. 2017. Factors influencing retention of health workers in the public health sector in Kenya: A case study of Kenyatta National Hospital. International Journal of Scientific and Research Publications 7(5):811–846.
Novak, S. 2012. Hands-on medical education in Rwanda. The New York Times. https://www.nytimes.com/2012/05/14/world/africa/14iht-educlede14.html?_r=2 (accessed January 31, 2020).
NYU (New York University). 2017. Rwanda Human Resources for Health (HRH) Program. https://nursing.nyu.edu/global-initiatives/rwanda-human-resources-health (accessed October 14, 2019).
Quinn, J. W., B. Powell, and R. Riviello. 2015. A paradigm shift in global surgery training: Rwanda’s Human Resources for Health (HRH) Program. Boston, MA: Harvard Business Publishing. http://www.globalsurgery.info/wp-content/uploads/2014/09/HRH-Program-Rwanda.pdf (accessed March 25, 2020).
RCE-VIHSCM (Regional Centre of Excellence for Vaccines, Immunizations and Health Supply Chain Management). 2018. EAC trainers join together with train the trainer facilitators from the Regional Centre of Excellence, HERA and UNICEF. Kigali, Rwanda: University of Rwanda.
RCE-VIHSCM. 2019. Master in health supply chain management. https://hscm.ur.ac.rw/?q=node/203 (accessed October 14, 2019).
RECA (Rwanda Emergency Care Association). 2019. About RECA. https://www.recaonline.org/home/sample-page (accessed October 24, 2019).
Relf, P. M., S. Isabelle, V. Linda, B. Linda, Y. Claire McKinley, and V. Michael. 2015. Duke University School of Nursing’s impact on nursing and midwifery education in Rwanda. Rwanda Journal 2(2).
Rickard, J., R. Ssebuufu, P. Kyamanywa, and G. Ntakiyiruta. 2016. Scaling up a surgical residency program in Rwanda. East and Central African Journal of Surgery 21:11.
Rogo, K. O. 2019. Overview of health professional education regulatory oversight for Rwanda and the region, edited by National Academies staff. September 17, 2019. (Available by request from the National Academies Public Access Records Office [firstname.lastname@example.org] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)
Ryamukuru, D., J. Mukantwari, L. Omondi, G. R. Karonkano, L. Uzikwambara, E. Munyaneza, E. Karera, T. Uwera, F. Mbonera, C. Ntakirutimana, P. H. Nzamurambaho, J. P. Nsekambabaye, O. Umuhoza, E. Sindayigaya, M. Mukeshimana, and D. Mukamana. 2018. Clinical experiences of perioperative nursing masters students in selected Rwandan referral hospitals. Rwanda Journal of Medicine and Health Sciences 1(1).
Sewankambo, N., J. K. Tumwine, G. Tomson, C. Obua, F. Bwanga, P. Waiswa, E. Katabira, H. Akuffo, K. E. Persson, and S. Peterson. 2015. Enabling dynamic partnerships through joint degrees between low- and high-income countries for capacity development in global health research: Experience from the Karolinska Institutet/Makerere University partnership. PLoS Medicine 12(2):e1001784.
Stevens, C., L. Forbush, and M. Morse. 2015. Social medicine reference toolkit. Social Medicine Consortium. https://static1.squarespace.com/static/5666e742d82d5ed3d741a0fd/t/5900ae6fa5790a1226d0e330/1493216881354/Social+Medicine+Toolkit+.pdf (accessed November 8, 2019).
Tvedten, I., A. Byabagambi, J. Lindstrom, and M. Tedre. 2018. Evaluation of the SIDA-supported research capacity and higher education development program in Rwanda, 2013–2017. Stockholm, Sweden: Swedish International Development Cooperation Agency.
UGHE (University of Global Health Equity). 2017. Strategic plan FY18-FY22: Executive summary. Kigali, Rwanda: University of Global Health Equity.
University of Rwanda. 2019. Policies and procedures on academic staff appointments and promotions. Kigali, Rwanda: University of Rwanda.
Uwizeye, G., D. Mukamana, M. Relf, W. Rosa, M. J. Kim, P. Uwimana, H. Ewing, P. Munyiginya, R. Pyburn, N. Lubimbi, A. Collins, I. Soule, K. Burke, J. Niyokindi, and P. Moreland. 2018. Building nursing and midwifery capacity through Rwanda’s Human Resources for Health Program. Journal of Transcultural Nursing 29(2):192–201.
Walker, T., V. Dusabejambo, J. J. Ho, C. Karigire, B. Richards, and A. N. Sofair. 2017. An international collaboration for the training of medical chief residents in Rwanda. Annals of Global Health 83(2):339–346.
WHO (World Health Organization). 2013. Transforming and scaling up health professionals’ education and training. Geneva, Switzerland: World Health Organization.
WHO and GHWA (Global Health Workforce Alliance). 2008. Scaling up, saving lives: Task force for Scaling Up Education and Training for Health Workers. Geneva, Switzerland: World Health Organization.
Yi, S., O. F. Umuhire, D. Uwamahoro, M. Guptill, and G. N. Cattermole. 2017. Developing and implementing a global emergency medicine course: Lessons learned from Rwanda. Education for Health: Change in Learning and Practice 30(3):203–210.
Yumbya, D. 2019. East African medical and specialist accreditation organizations—for U.S. National Academy of Medicine evaluation committee. Washington, DC. September 16, 2019. (Available by request from the National Academies Public Access Records Office [email@example.com] or via https://www8.nationalacademies.org/pa/managerequest.aspx?key=HMD-BGH-17-08.)
This page intentionally left blank.