2

Evaluation Design, Methods, and Limitations

This chapter details the evaluation’s operational design and methodology, building on the discussion of the theoretical framework in Chapter 1 and followed by a discussion of the limitations encountered.¹

DESIGN

To address the Statement of Task, the evaluation applied a retrospective, concurrent mixed-methods design with embedded in-depth examinations and contribution analysis. The committee approach focused on the Human Resources for Health (HRH) Program’s potential contributions to observed outcomes by understanding how the Program and its components were implemented and by examining the contextual factors that may have enhanced, moderated, or otherwise influenced outcomes (Moore et al., 2014).

Mixed-methods designs provide the flexibility to capture trends regarding what results have occurred, while enabling a deeper understanding of how gains were achieved and why change has (or has not) happened (Creswell and Plano Clark, 2007). Such designs also provide insight into how different populations might have experienced the intervention. Drawing on multiple data sources and approaches, this evaluation yields an understand-

___________________

¹ The protocol for this evaluation underwent a scientific and technical review by the U.S. Centers for Disease Control and Prevention, Office of the Associate Director of Science, and was approved on April 25, 2019. Ethical approvals for this evaluation were also provided by the Rwanda National Ethics Committee (April 3, 2019) and the U.S. National Academies’ Committee to Review Studies on Human Subjects (April 23, 2019).

ing of both breadth (via quantitative data) and depth (via qualitative data). Using mixed methods and drawing on data across diverse data sources is of critical importance when evaluating complex interventions for which the pathway between activities and outcomes is nonlinear (Creswell and Plano Clark, 2007; Patton, 1987).

This evaluation draws from the following types of data:

• Literature and document review
• In-depth examinations of the University of Rwanda and a facility microsystem
• Qualitative interview data
• Quantitative secondary data

Figure 2-1 illustrates the link between the evaluation objectives and the design elements (e.g., document review, in-depth examinations, qualitative interviews, and secondary quantitative data) used to address the objectives. The lower part of the figure shows that although the design was concurrent, some elements occurred sequentially. The document review preceded the concurrent qualitative interviews, in-depth examinations, and secondary quantitative data. Integrated interpretation was the final phase. Integrated analysis and interpretation occurred during July, August, and September 2019.

The data sources had complementary uses for the evaluation. The document review drew on policy and program documents, reports, and published literature. It situated the HRH Program within the broader HRH and HIV context in Rwanda and across the globe to understand the context

and landscape in which the Program was designed and implemented and to inform an understanding of the extent to which the Program implemented activities and produced results as planned. Data from interviews with key respondents with specialized knowledge of the HRH Program—its inception, implementation, management, and transition after the close of funds from the President’s Emergency Plan for AIDS Relief (PEPFAR)—were used to further address the evaluation objectives. These data provided insight into the Program’s implementation and achievements, and into its perceived impact on HRH capacity and HIV service delivery. These data also facilitated the interpretation of findings from the document review and from secondary quantitative data for HRH and HIV outcome indicators, which were analyzed for trends over time.

The two in-depth examinations were conducted at the University of Rwanda and a “facility microsystem,” a facility that receives referral patients from a lower-level facility. These deeper examinations provided a more holistic understanding of the effects of the HRH Program on the capacity to produce a workforce of sufficient quantity and quality to meet the needs of the Rwandan population, and of the HRH Program’s role in affecting health care management and the provision of HIV and other health services. In an effort to assess the potential causal impact on HIV outcomes, the intention was to treat HRH Program graduates as an intervention, characterizing each district’s dose based on quantity and type of graduate. The committee could then estimate the Program’s pooled effect on the HIV outcomes of interest. However, unavailability of data at the required level of detail hindered the committee’s ability to perform this type of analysis.

Evaluation Team

A team of expert evaluators from EnCompass LLC collaborated with the committee and study staff in the design phase, carried out the primary qualitative and secondary quantitative data collection and initial analysis, and provided synthesized findings and initial interpretations to the committee. See Appendix A for more information about the EnCompass evaluators.

Sample

A mixed-methods design involves collecting data from diverse populations using an appropriate sampling methodology.

Literature and Document Review

The evaluation team conducted a thorough search to identify the documents for review, including HRH Program reports; relevant global and

Rwandan policies, guidelines, plans, and strategies; and peer-reviewed and gray literature on topics related to HRH and HIV service delivery in Rwanda, the region, and worldwide. Key respondents shared additional documents during the qualitative data collection.

In-Depth Examination—University of Rwanda

The University of Rwanda, as the primary institution responsible for educating health professionals, was purposively selected to understand the institutional capacity for health professional education. The evaluation team identified and invited key respondents to participate in interviews. Key respondents were individuals with knowledge of the HRH Program design and implementation and with experience with the Program in the university, including administration, leadership, faculty, and students across relevant schools (medicine, nursing and midwifery, and public health).

In-Depth Examination—Facility Microsystem

The intention of the facility microsystem analysis, which included at least a district hospital and a teaching/referral hospital, was to construct a bridge between the outputs of the HRH Program (i.e., a larger and well-trained health workforce, including greater availability of specialists for referral services) and patient-level outcomes for HIV. The evaluation aimed to designate facilities by degrees of exposure to the Program, as determined by having or not having HRH trainees on staff. However, as the evaluation team learned more about the Program, it became evident that all facilities had HRH trainees on staff, including hospital administrators, physician specialists, nurses, and midwives.

It was also important to gather experiences and perceptions from HRH trainees and frontline health workers working outside of Kigali, prompting the selection of the Centre Hospitalier Universitaire de Butare (CHUB), which is a university teaching hospital in Butare in the Huye district (Southern Region), and Bushenge Hospital in the Nyamasheke district (Western Region). These districts were chosen because they had similar HIV prevalence to the 2014 to 2015 Rwandan national average of 3 percent (NISR et al., 2016), making them typical cases with respect to HIV-related indicators. The Nyamasheke district had higher antiretroviral therapy (ART) coverage (85 percent) than the 2015 national average (74 percent) (PEPFAR Rwanda, 2016),² while the Huye district was comparable to the national average in both categories (see Table 2-1).

___________________

² While more recent national-level estimates of ART coverage are available and indicate greater coverage; for selection purposes, it was necessary to use coverage estimates from the same year.

TABLE 2-1 HIV Prevalence and ART Coverage in Facility Microsystem Districts

NOTE: ART = antiretroviral therapy.

SOURCE: PEPFAR Rwanda, 2016.

Qualitative Interviews

Interviews (qualitative and in both of the in-depth examinations) involved a predominantly purposive sampling approach, mixed with snowball sampling, in which information-rich respondents were selected during the first round. These respondents were then invited to suggest other potential respondents. Respondents were selected from the following categories:

• HRH Program administration, both within and outside the Government of Rwanda
• Faculty from U.S. institutions that were members of the HRH Program’s academic consortium
• Professional associations and people living with HIV (PLHIV) groups
• Other key stakeholders, such as representatives and staff of other donors and international organizations working in HRH and HIV in Rwanda

Current employees of the U.S. Centers for Disease Control and Prevention (CDC) and PEPFAR at both the Rwanda country team and headquarters levels declined to participate as interview respondents, owing to a determination made by CDC’s Associate Director for Science that there could be an actual or perceived conflict of interest.

The final sample of interview respondents across the three evaluation components (facility microsystem in-depth examination, University of Rwanda in-depth examination, and qualitative interviews) included 87 interviews (see Table 2-2).

Several respondents fell into multiple stakeholder types (e.g., a student who had graduated and gone on to serve as a university faculty member, or a university faculty member who also held a leadership position in a professional association).

TABLE 2-2 Interview Respondent Sample

NOTE: CHUB = Centre Hospitalier Universitaire de Butare/University Teaching Hospital, Butare; NGO = nongovernmental organization; PLHIV = people living with HIV.

Data Collection

The evaluation team conducted a literature and document review and collected publicly available quantitative data between October 2018 and October 2019. Primary qualitative and requested secondary quantitative data were collected between May and September 2019.

Literature and Document Review

The evaluation team conducted a thorough literature and document review of materials on the HRH Program and related topics, as well as documents from other projects and programs related to HRH in Rwanda. The evaluators gathered publicly available documents from relevant websites dating from 1996 through 2019, beyond the end of PEPFAR’s investment in the HRH Program. This time frame includes health system rebuilding following the genocide against the Tutsi, decentralization of the health system, and initiation of the HRH Program in 2012, and it extends beyond the implementation and closeout of PEPFAR’s support of the HRH Program in 2017. The review comprised 4,267 documents:

• Program documentation: Request for proposals, proposals, designs, regular reporting to CDC, financial information provided by the Ministry of Health (MOH), and closeout documents

• Peer-reviewed, gray, and unpublished literature on the HRH Program, health workforce, health professional education, and the HIV epidemic in Rwanda more broadly, as well as HRH labor market analyses
• Government of Rwanda documents, including policies, strategies, plans, and guidelines on HRH and HIV service delivery at the national and subnational levels; labor market reports and other documentation on labor market dynamics; and insurance scheme documentation
• Performance-based and results-based financing documents
• Global documents, including policies, priorities, strategies, plans, and guidelines on HRH and HIV service delivery from sources such as PEPFAR; the Global Fund to Fight AIDS, Tuberculosis and Malaria; the World Health Organization; and the Joint United Nations Programme on HIV/AIDS (UNAIDS)
• Program documents and publications from other HRH-related programs in Rwanda, including those from donors and implementing partners

Qualitative Data Collection

All qualitative data collection conducted as part of the two in-depth examinations and other qualitative interviews used Appreciative Inquiry, an asset-based approach that captures current strengths and the most significant changes identified by respondents, as well as desired realities and steps needed to reach those realities. Appreciative Inquiry actively engages respondents and increases the likelihood of obtaining rich information about sensitive topics (Preskill and Catsambas, 2006). Data were gathered via semistructured individual interviews and group discussions.

In keeping with the evaluation’s appreciative and utilization-focused principles, interview guides emphasized collecting participants’ views of what worked well in the HRH Program,³ factors that facilitated or inhibited success, and opportunities for continued success in HRH in Rwanda. The guides also focused on areas where respondents saw potential for improvement in future HRH and health systems strengthening activities, programs, and initiatives. These approaches enabled the evaluation team to

___________________

³ Rwanda-based staff reviewed the interview guides for translation errors in Kinyarwanda and English and to ensure a clear understanding by all those interviewed. The semistructured interview guides were adjusted for relevance to respondent type. For trainee respondents, probes were adjusted as relevant. However, as is the standard when using semistructured interview approaches, the key questions were kept consistent to reveal trends across respondent groups.

gather a variety of perspectives on HRH Program successes and challenges and to ensure relevant, realistic findings and recommendations.

A central tenet of qualitative methods is flexibility; in this case, the evaluators adapted and adjusted lines of inquiry and interview guides as new information and insights emerged. Data collection and analysis was iterative; data were coded and initial analyses were performed during the data collection phase to inform subsequent data collection. During regularly scheduled meetings, the evaluation team collectively determined new insights to examine more thoroughly and how to gather this information. This process involved minor modification to the interview guides to better understand the nuances of the HRH Program and context, but these changes were not substantive, in that they did not involve new topics or respondent groups.

Initial analysis of the transcripts was another means of evaluating and enhancing the quality of the data. Transcripts were evaluated for fidelity to the data collection instruments, appropriateness and depth of probing, detail of descriptive notes from the data collectors, and quality of the rapport between data collectors and key respondents. When issues arose, data collectors were retrained, focusing on improving areas of weakness. Data quality assessment took place throughout the data collection phase.

Preference was for individual interviews to provide the space and confidentiality to examine individual experiences and potentially sensitive knowledge. All but one interview was individual. Respondents in Rwanda were invited to participate in person in Kigali and other locations; if scheduling did not allow for this, the data collection team offered a virtual interview option. Data were gathered in English, French, or Kinyarwanda, according to the respondent’s preference. Respondents based in the United States or a third country were invited to participate in virtual interviews. Virtual data collection was conducted in English by U.S.-based evaluation team members using Zoom or Skype, depending on the quality of the connection and the respondent’s preference.

All interviews and group discussions (in-person and virtual) were audio recorded, transcribed verbatim, and, when necessary, translated to English for analysis. Virtual interviews were recorded using an external device, not through the virtual platform’s recording option, to avoid a third party retaining a copy of the conversation.

In-Depth Examination—University of Rwanda

Interview guides for the in-depth examination with the University of Rwanda were tailored to those in administrative roles, those in faculty

roles, and current and former students. Interviews explored the following topics:

• Experience with the administration of the HRH Program
• Process of, and experience with, building faculty capacity to train health care workers and managers, including via the twinning program and curriculum development
• Perception of systemwide effects of the HRH Program on health care service delivery in Rwanda: production, distribution, and management of HRH, including HIV service; planning and management at health facilities; and referrals between facilities
• Perception of the HRH Program’s impact on the health education system infrastructure at the university

In-Depth Examination—Facility Microsystem

One Rwandan data collector made three site visits to collect data from the two facilities in the facility microsystem. Interviews were conducted with health care workers and health managers, those who had undergone training under the HRH Program and those who had not. Interviews explored the following topics:

• Experience with health professional education training (pre-service and in-service)
• Professional career path
• Job satisfaction and belief in their own capacity, morale, and motivation
• Perception of their job as an important, viable career
• (When applicable) Perception of systemwide effects of the HRH Program on health care service delivery in Rwanda: production, distribution, and management of HRH, with a focus on HIV service; planning and management at health facilities; and referrals between facilities
• (When applicable) Perception of the HRH Program’s impact on the quality of HIV and other health services

Qualitative Interviews

Qualitative interview guides were developed according to initially identified relevant topics and then adapted iteratively, based on respondent type and what had been learned in previous data collection activities, following the approach described above. Table 2-3 lists the detailed interview topics by respondent type.

TABLE 2-3 Interview Topics by Respondent Type

NOTE: HRH = human resources for health; PEPFAR = President’s Emergency Plan for AIDS Relief; PLHIV = people living with HIV.

Quantitative Data

The evaluation team sent an informal request for quantitative data to the MOH and Rwanda Biomedical Center (RBC) on May 7, 2019, followed by a formal request, sent on May 17, 2019. The RBC issued an approval letter on May 20, 2019, but with the stipulation that study findings be shared with the RBC before publication or any dissemination. To meet this condition, the National Academies of Sciences, Engineering, and Medicine (the National Academies) proposed that the MOH conduct a technical review of the analysis of MOH-provided data and that a predissemination briefing be held with the MOH. The Ministry’s agreement to this proposal was received on August 20, 2019, at which point the evaluation team began working directly with key individuals in the MOH to obtain these data. The following data were requested:

• HIV-related indicators at the facility level
• HRH Program trainee data: who was trained under the Program, in what specialty, graduation dates, and current place of employment
• Equipment procured under the HRH Program
• HRH Program budget and expenditures

The evaluation team did not receive facility-level HIV data from the MOH, leaving the committee to rely on publicly available data, predominantly from UNAIDS. Per communication with the MOH, there was not a readily available database of HRH Program trainees and their current places of employment; thus, it was not possible to understand which facilities had been exposed to the HRH Program. The MOH and the University of Rwanda did provide elements of the data on trainees alongside data from

professional councils responsible for licensures and other publicly available data from Government of Rwanda sources, such as the Master Facility List. However, these data varied in some instances, confounding interpretation. Because of the timing of receipt of the data, the committee was not able to reconcile these variations across sources.

Data Analysis

Literature and Document Review

The evaluation team conducted text analysis of literature and documents using a Microsoft Excel spreadsheet set up to capture and organize data according to topics represented in the evaluation objectives and questions and reflecting the theoretical causal pathway discussed in Chapter 1. The team generated a time line to more readily observe the events that preceded and were concurrent to HRH Program implementation, including after the end of PEPFAR funding for the Program. Comparison matrices were generated to explore convergence and divergence between what was planned as described in the design documents and what was executed, as described in HRH Program reports through 2017. Text and policy analyses were performed to examine key elements of the context and the Program,⁴ in keeping with the theoretical causal pathway.

In-Depth Examinations and Qualitative Data

The evaluation team employed a utilization-focused and iterative data collection and analysis process, with initial data analysis occurring as data were still being gathered to ensure quality, refine questions, and identify new lines of inquiry. Any additional data added to the dataset were coded appropriately, with new findings integrated into existing findings and tested to confirm or disconfirm cases.

A combination of deductive (theory-driven) and inductive (data-driven) coding approaches were applied to the qualitative data. The evaluation team developed an initial coding scheme, based on the evaluation objectives, and included the topics in the data collection instruments. Complementary to the deductive approach, transcripts were read to enable open coding and identification of topics outside the content of the guides. Four members of the evaluation team led the coding, using an iterative process, reapplying additions or changes to the coding scheme to the entire dataset.

___________________

⁴ “Policy analysis” describes a type of analysis that examines economic, social, or other public issues through the formulation, adoption, and implementation of a principle or approach to address a problem. As this examination was of existing policies, it is primarily descriptive.

Frequent meetings with the evaluation team members participating in coding facilitated a shared understanding of the coding scheme. Intercoder reliability testing was conducted to ensure at least 90 percent agreement across coders. Qualitative transcripts were coded in Dedoose, an online qualitative data analysis platform the team selected to allow multiple team members to code in the same platform and check and correct any discrepancies in real time. Dedoose enables encryption, so only those participating in data collection or analysis had access to these data. Coded data were synthesized into categories and themes, informed by the evaluation objectives. The team produced interpretive data summaries, reflecting the initial insights generated in each category or theme and, where appropriate, links with other categories or themes.

The evaluation team performed comparative analysis to identify points of convergence and divergence across key respondent groups. Comparative matrices were prepared by theme to generate visual representations of the points of convergence and divergence. Findings generated from these data sources were combined with insights generated through the comparative matrices to enrich understanding of the HRH Program’s planned activities and what had been implemented.

The evaluation team conducted two main phases of data analysis, with the first round in preparation for the fourth committee meeting in July 2019. Insight and guidance from the committee was integrated into subsequent data collection, coding, and analysis. The second phase of data collection was completed in August 2019. Following data collection, the evaluation team conducted an internal, integrated 2-day data analysis, triangulation, and interpretation session.

Quoted material throughout this report is redacted to protect respondents’ confidentiality and anonymity, in alignment with the ethical guidance approved in the evaluation protocol. Direct quotations from respondents included in this report have been edited for clarity.

Quantitative Data

The evaluation team used a time series approach to examine publicly available HRH and population-level data for trends over time. Data were plotted in run charts and annotated with events and other descriptions to help interpret what was observed. Where appropriate, a maximum likelihood event count time series analysis was performed. A maximum likelihood event count time series model assesses the probability of a maximum parameter value in an ordered sequence of observations through time. It is generally a stochastic method, and when analyzing the variability of HRH program data, this approach was useful to assess whether the intervention had an effect significantly greater than the underlying trends in HRH and

population-level data. Wherever possible, this analysis was performed at national and subnational levels. First, data was collated and cleaned in Microsoft Excel and subsequently analyzed in Stata 14 using the estimation function.

Analysis of financial data was descriptive, exploring changes in the level of funding for the HRH Program across sources. Other CDC and donor funds in HRH and health systems strengthening in Rwanda were also examined for change over time, as a means of understanding the financial landscape.

Analysis and Interpretation with the National Academies’ Committee

To ground findings and conclusions in utilization-focused approaches, following initial independent analysis of data by type, the evaluation team used a participatory process for integrating and synthesizing the data. This was conducted at three key points in the data collection and analysis period. First, during the fourth meeting of the National Academies’ committee (July 9–10, 2019), the EnCompass team presented initial insights into the data that had been collected by that time. The committee worked through an affinity analysis process, in which they organized insights into categories and then queried the data for what was missing, what needed further data collection, and what needed additional data analysis.

The evaluation team also held a 2-day internal data analysis, triangulation, and interpretation workshop on August 20–21, 2019. This workshop included only those on the evaluation team who had collected or analyzed data. Participants examined evidence across data sources, shared emerging themes, and developed initial findings and interpretations. The result of the workshop was an evidence-based narrative addressing the first three evaluation objectives, as well as thematic matrices that presented qualitative and quantitative data side by side, facilitating observation of convergence and divergence and explanation of evidence that supported each finding and conclusion.

The evaluation team then supported a process of further analysis and interpretation by the committee in September 2019. After providing committee members with draft findings and interpretations, backed by the thematic matrices, the evaluation team facilitated a participatory, appreciative session with the committee to further examine, interpret, and validate findings and to arrive at consensus on conclusions and recommendations.

Ethical Considerations

Approvals for this evaluation were provided by the Rwanda National Ethics Committee (April 3, 2019), the U.S. National Academies’ Commit-

tee to Review Studies on Human Subjects (April 23, 2019), and the CDC Office of the Associate Director for Science (April 25, 2019). Permissions were obtained from the necessary national, subnational, and institutional authorities to carry out the in-depth examinations at the University of Rwanda and the facility microsystem.

All potential respondents for in-depth examinations and interviews received a written information sheet and a certificate of consent in English or Kinyarwanda, as appropriate. The information sheet described the purpose of the evaluation; how the respondent(s) had been selected; the data collection method, procedure (individual interview or group discussion), and duration; risks and benefits; reimbursements; confidentiality; planned sharing of findings; and voluntariness (right to refuse or withdraw). Respondents also received contact information for the Rwanda National Ethics Committee. Signed consent was the preferred method of documentation. Virtual interview respondents received the consent information during the recruitment and scheduling process, and their electronic signatures were obtained.

LIMITATIONS

In an effort to address the first two objectives, which focused on describing the HRH Program and the context in which it was implemented, the evaluation team conducted qualitative interviews with individuals who were involved in the administration of the Program. These included respondents from the Government of Rwanda and other organizations involved in the design and early implementation. However, CDC’s Associate Director for Science determined that the participation of current staff would present a conflict of interest, and staff from CDC, PEPFAR, and the Office of the U.S. Global AIDS Coordinator declined invitations for interviews. The absence of PEPFAR’s perspective, as the funder of the portion of the Program under evaluation, presents a significant gap in the data and has likely resulted in skewed findings. The donor’s perspective would have provided balance and nuance to findings, conclusions, and recommendations around program design, management, and sustainability. The absence of clear conclusions regarding the donor is the result of these missing data and should not be interpreted as an indication of donor performance.

Not all specialties under the HRH Program are represented equally in the qualitative interviews. The sample is skewed toward pediatrics, obstetrics and gynecology, and nursing, facilitating deeper examination of these specialties relative to others, such as surgery, internal medicine, and midwifery.

With respect to the third objective of the evaluation, described in more depth in Chapter 1, it was not possible to design an evaluation that assessed the attributable impact of the Program on HRH outcomes or HIV-

related morbidity and mortality. The lack of facility-level HIV indicator data decreased the committee’s ability to assess meaningful differences in HIV care and outcomes between facilities with and without HRH trainees. However, even if the committee had these data, identifying an appropriate comparator—which would be the most appropriate approach for determining attribution—was not possible. Instead, this evaluation uses the lens of contribution analysis and presents regional data alongside Rwandan indicator data to situate Rwanda’s progress in context, without making analytical comparisons.

Drawing conclusions about plausible contribution was also challenging, because of the multitude of interacting factors and concurrent programs that could reasonably be expected to contribute to the same outcomes of interest specified in the Statement of Task. Furthermore, the timing of this evaluation, just 2 years following the end of PEPFAR funding, also presented an obstacle to determining changes in HIV-related mortality and morbidity, as well as other HRH outcomes. Producing a specialized health workforce that has entered the health system and is providing high-quality services for PLHIV takes years, and observing any patient-level effects would require even more time. This challenge was similarly important for evaluating the Program’s sustainability and degree of institutionalization.

In addition, the integration of financial investments from diverse sources in the Rwandan government’s administration and management of the HRH Program meant it was difficult to disentangle PEPFAR’s investments and related activities from other sources (Government of Rwanda and the Global Fund). It was also challenging to assess the plausible contribution of the activities that were predominantly funded by PEPFAR without more detailed disaggregation of Program expenditures, which would have enabled a more thorough description and analysis of the relative contribution of different programmatic activities.

Finally, the request for data on where HRH trainees were placed following graduation had to be extracted from multiple sources, as Rwanda’s Human Resources Information System did not include information on health worker training. This placed a significant burden on the MOH for extracting and generating the information. Much of the data used in this analysis was therefore extracted from publicly available data, which was generally aggregated data that did not allow for effective subnational analysis. Other data were not up to date; for example, the most recent Annual Health Statistics Booklet published by the MOH includes data up to 2016. Finally, the data that were received from the MOH raised questions when triangulating and interpreting them alongside data received from other sources.

A technical review was conducted in October 2019, in which the MOH was provided with tables and figures, as well as descriptive text (methods

applied and description of data sources), but excluding findings, interpretations, or any information that could be interpreted as committee deliberations. The purpose of this review was to seek guidance from the MOH and prevent misinterpretations that were technical in nature. In accordance with the National Academies’ policy, comments and written feedback provided by the MOH as part of this technical review are included in the study’s Public Access File. Documents and other information provided for the committee’s consideration are available upon request from the National Academies’ Public Access Records Office (paro@nas.edu).

REFERENCES

This page intentionally left blank.