National Academies Press: OpenBook

Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary (2015)

Chapter: 4 Reviewing the Evidence for Different Quality Improvement Methods

« Previous: 3 Six Widely Used Methods to Improve Quality
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

4

Reviewing the Evidence for Different Quality Improvement Methods

Key Points Made by Individual Speakers

  • An exhaustive systematic review on the effect of different quality improvement strategies to change provider performance was unable to make any conclusions about the effectiveness of COPE®, SBM-R, or accreditation because of insufficient information. (Rowe)
  • Training and supervision have modest positive effects on provider performance. The two strategies in combination may work better than either one alone. (Rowe)
  • The data suggested a somewhat larger effect for improvement collaboratives, but high risk of bias in these studies prevents any firm conclusion. (Rowe)
  • There is a need for more head-to-head comparisons of quality improvement tools, but it is not clear who should be charged with such research. Implementing organizations may have a conflict of interest in evaluating their own work. (Broughton, Mate, Rowe)
  • Electronic data management systems can improve the efficiency of the health system and, when properly managed, can provide the data necessary to establish the link between quality programs and improved health. (Agins)
  • Monitoring quality of care in a country depends on accurate clinical and death registries and the ability to link patient data across registries. (Klazinga)
  • Cost-effectiveness analysis requires information on the costs and health consequences of every outcome a quality of care program could have on a system, and it depends on epidemiological modeling,
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
  • the assumptions of which are always debatable. But the complexity of such research is no excuse not to do it. (Broughton)

  • Cost-effectiveness research is neglected, an omission that could undermine the field of quality improvement. (Broughton)
  • More randomized trials of improvement interventions could yield a wealth of information for policy makers. (Broughton)
  • Early planning for program evaluations could help make them more of a priority. (Barker, Broughton, English)

A consistent thread in the January 28 discussion was the need for more evidence, but the workshop aimed to glean as much insight as possible from the evidence that already exists. In the next session, participants considered an exhaustive systematic review on the effects of different quality improvement strategies on provider behavior. They also discussed challenges and opportunities in measuring quality of care and how to establish the cost-effectiveness and feasibility of quality improvement programs.

WHAT WORKS?: THE RESULTS OF A SYSTEMATIC REVIEW

Alexander Rowe of the CDC presented preliminary results of the Health Care Provider Performance Review, a systematic review of quality improvement methods to change the behavior of health care providers (hereafter, providers), funded by The Bill & Melinda Gates Foundation, the CDC, and the World Bank.

The systematic review included any strategy for improving provider performance, with providers being defined rather broadly to include private health workers, pharmacists, and drug shopkeepers. Both published and unpublished studies were included, and there were no language restrictions on inclusion. Pre- and post-intervention studies with a comparison group were eligible, as were post-intervention only studies with randomized controls, and interrupted time series studies with at least three data points before and after the intervention. Rowe’s team identified studies from 15 electronic databases; this search was finished in 2006. Next, they reviewed personal libraries, searched document inventories of 30 organizations, and asked colleagues for references and unpublished studies, a phase of the review that ended in 2008. They also conducted a hand search of bibliographies from 510 previous reviews and other studies. Over the project’s last years, 17 investigators sent in new reports from their research. Two people reviewed each study report, corresponding with authors when details of their study or strategy were not clear. Through the extensive literature review and verification process, the final database came to contain more information than the published reports.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

In preparation for the IOM workshop, Rowe and his colleagues reviewed their database for information about the effectiveness of the six quality improvement methods being discussed. The database has about 150 variables that separately code each component of a strategy. For example, training and supervision are separate components of quality improvement strategies. Because there is no universal taxonomy of strategies to improve provider performance, all the definitions in the database are working definitions created for the analysis. Table 4-1 shows how each strategy was defined. Rowe conceded that some experts might object to the analytic organization. The accreditation category, for example, includes licensing, certification, accreditation, or registration programs. The proprietary methods of COPE® and SBM-R also posed an analytic challenge. As the database had no studies specifically involving COPE® or SBM-R, Rowe

TABLE 4-1 Definitions of the Six Strategies Using Component Variables from the Health Care Provider Performance Review (HCPPR) Database

Strategy Definition
High-intensity training only Training >5 days (or ongoing training) with ≥1 interactive method (i.e., clinical practice, interactive sessions, or role play). No other components.a
Low-intensity training only Any training that is not high-intensity training. No other components.a
Supervision only Supervision. Excludes strategies that resemble supervision (e.g., audit and feedback). No other components.
Accreditation only A strategy with only the component: “licensing, certification, accreditation, or registration.”
Improvement collaborative only Improvement collaborative, as defined by authors of the report. No other components.
Client-oriented, provider-efficient (COPE)® methodb only A “COPE®-like” strategy was defined as having all the following components: provider self-assessment, continuous quality improvement (includes team-based problem solving), and peer review. No other components.
Standards-based management and recognition (SBM-R)b only An “SBM-R-like” strategy was defined as having all the following components: standard health facility specifications were introduced, health facility received recognition after meeting certain criteria, health care provider self-assessment, team-based problem solving, supervision, and low-intensity training (according to HCPPR’s definition, which also includes informal education by a peer).

a Excludes academic detailing and informal education by a peer. Also, training is allowed to have job aids or printed educational materials for health care providers.

b No studies involving COPE® or SBM-R were included in the HCPPR database. Using variables from the database, COPE®- and SBM-R-like strategies were constructed for the analysis.

SOURCE: Rowe, 2015.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

constructed COPE®-like and SBM-R-like variables based on consultation with Carmela Cordero and Edgar Necochea.

Studies on provider performance use a wide range of outcomes. Some researchers look at the effect of their program on mortality rates, others on taking a patient history; such disparate outcomes cannot be compared in the same analysis. Therefore, the analysis presented included only outcomes on the process of care expressed as a percentage (e.g., the proportion of patients correctly assessed, diagnosed, or treated). The analysis was also stratified by two main groups of providers. Rowe reasoned that lay health workers are different from health professionals (e.g., doctors, nurses, pharmacists) in important ways, and studies on these populations should be dealt with separately.

Rowe used methods based on the Cochrane Effective Practice and Organisation of Care recommendations for determining risk of bias (Higgins et al., 2008). In this system, risk of bias is a function of study design, the number of clusters in each arm, data completeness, between-group comparability at baseline, the outcome’s reliability, concealment of allocation for studies randomized at the patient level, the likelihood that the intervention could change data collection, and having fewer than six data points before or after an intervention for interrupted time series. Studies were coded as having low, moderate, high, or very high risk of bias.

Analytic Strategy

Estimates of effect size were expressed in terms of absolute percentage point change using the equation:

size = (Follow UpBaseline)intervention – (Follow UpBaseline)control.

Interpreting the difference of differences calculation is straightforward: if the intervention group sees the proportion of patients correctly treated rise to 50 percent from 20 percent, that is a 30 percentage point improvement. If the control group sees an increase of 5 percentage points, then the difference of differences is 25 percentage points; so, for every 100 patients seen, 25 are correctly treated in a way attributable to the intervention. This calculation has an added intuitive appeal because positive values indicate an improvement. (For studies that were designed to show a decrease in certain outcomes, the calculation was flipped.) For interrupted time series data, the investigators used a similar approach, but with values derived from segmented regression modeling.

The analysis mainly considered comparisons between a particular strategy and some kind of control; head-to-head comparisons of different methods were not included. If a study reported more than one primary

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

outcome, the median effect size was the statistic of interest. Investigators compared median effect size distributions using interquartile ranges and a weight calculated as:

(1 + ln (the number of providers or sites)).

For strategies with fewer than five studies, investigators used the unweighted median.

Exploratory analysis in the larger database and a priori knowledge of the topic suggested possible confounding effects, so the investigators attempted to adjust all the results to a partly standard context, the result that might have been observed if all studies had a similar baseline. To this end, they adjusted the analysis for two main effect modifiers: baseline performance and a public health facility setting. Rowe explained that, regardless of the strategy, better providers or clinics have less room for improvement; for every 10 percentage point increase in baseline performance, the observed effect size decreased by 2 percentage points on average. Therefore, for every 10 percentage points over the average baseline performance, investigators added 2 percentage points to the effect size estimate.1 Similarly, the mean effect was 8 percentage points higher in a public facility than in any other setting, regardless of strategy. As about half of the effect sizes were from studies with a public facility–only setting, the adjustment subtracted about 4 percentage points from effect sizes of studies with a public facility–only setting and added 4 percentage points to effect sizes from other settings.

For strategies that appeared to have the greatest effectiveness, the analysis checked for confounding by limited variability, the chance that the observed effect came from an idiosyncrasy of study design—that is, a setting unusually well suited to a strategy. When this confounder was a concern, the investigators broadened the definition of a strategy to include more studies with the same basic strategy components. If the adjustment brought about a large decrease in the estimate of effect size, confounding is likely.2

Rowe also briefly discussed his plans for secondary analyses. In the future, the group will attempt different adjustments for sample size and ways to summarize effect estimates. Network meta-analysis, a relatively

_________________

1 For further clarity, the average baseline performance in the database is 41 percent, meaning that for every 100 patients supposed to be treated or diagnosed, 41 get the service as it is meant to be done. If the baseline in a particular study were 10 percentage points above the mean, or 51 percent, then the adjustment increased the effect size by 2 percentage points.

2 For instance, Rowe gave a hypothetical example of three studies of licensing suggesting an effect size of 50 percentage points, but, by broadening the definition of the licensing strategy to include studies of licensing with other components, the median effect size would fall to only 17 percentage points. In such a case, it would be prudent to conclude the effect of licensing is about 17 percentage points.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

TABLE 4-2 Number of Studies in the Health Care Provider Performance Review Database Comparing an Active-Strategy Group with a No-Intervention or Historical Control Group

Strategy Number of comparisons
High-intensity training only 9
Low-intensity training only 36
Supervision only 7
Accreditation only 0
Improvement collaborative only 7
COPE®-like strategy only 0
SBM-R-like strategy only 0
High-intensity training + supervision 4
Low-intensity training + supervision 5
Low-intensity training + improvement collaborative 3

SOURCE: Rowe, 2015.

new analytic technique, will allow for the inclusion of head-to-head comparisons. Explaining why this was not the primary analytic strategy, Rowe raised concerns with the validity of the sample sizes and, therefore, with analytic weighting, especially in the older studies. CONSORT3 guidelines require investigators to report a fair amount of detail on the design and conduct of randomized trials, such as the number of subjects per cluster, the number of clusters per study arm, the unit of randomization, and the inter-cluster correlation coefficient. Many of the studies in Rowe’s database pre-date such reporting requirements, making the more conservative approach preferable.

Preliminary Results

From the more than 105,000 citations screened and 824 reports included in the database, 66 were eligible for the analysis presented at the IOM workshop. Table 4-2 shows the number of comparisons in Rowe’s database on the six strategies discussed at the workshop. After removing the strategy groups not mentioned in the database, only 11 percent (n = 7) of the studies had low risk of bias, 23 percent (n = 15) had moderate risk

_________________

3 CONSORT, or Consolidated Standards of Reporting Trials, is a set of guidelines for reporting the results of trials that came into common use in the mid-1990s, with an extension for cluster randomized trials coming into use in the early 2000s.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

of bias, 36 percent had high risk of bias (n = 24), and 30 percent (n = 20) had very high risk of bias. Table 4-3 shows the breakdown of risk of bias by study strategy.

Thirty-three countries were represented in this analysis, 56 percent of which were low-income countries. Figure 4-1 gives more detail on the geographic breakdown of the data. Almost half of the studies were randomized

TABLE 4-3 Breakdown of the Risk of Bias in the Strategy Studies

Strategy   Risk of bias
Number of comparisons Low/Moderate High/Very high
High-intensity training only   9   4   5
Low-intensity training only 36 15 21
Supervision only   7   3   4
Improvement collaborative only   7   0   7
High-intensity training + supervision   4   0   4
Low-intensity training + supervision   5   2   3
Low-intensity training + improvement collaborative   3   0   3

SOURCE: Rowe, 2015.

image

FIGURE 4-1 Breakdown of the strategy studies by region.

SOURCE: Rowe, 2015.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

controlled trials (see Table 4-4); most studies were conducted in the 1990s and 2000s.

Rural, urban, peri-urban, and mixed settings accounted for similar proportions of the 66 studies, and there was fairly wide representation of different types of facilities (see Table 4-5), providers (see Table 4-6), health conditions (see Table 4-7), and methods of data collection (see Table 4-8).

Figure 4-2 shows the weighted median adjusted median effect sizes (MES) and interquartile range (IQR) for the strategies being discussed. The combination of improvement collaborative and low-intensity training showed unusually high effectiveness. The effectiveness, and the small number of studies informing the comparison, caused the investigators to suspect confounding by limited variability. After broadening the strategy inclusion criteria somewhat, the effectiveness estimate for the strategy declined from

TABLE 4-4 Breakdown of the Strategy Comparisons by Study Design

Study design Number of studies (%)
Pre-post study with randomized controls 27 (41)
Pre-post study with non-randomized controls 22 (33)
Interrupted time series 13 (20)
Post-only with randomized controls 4 (6)

SOURCE: Rowe, 2015.

TABLE 4-5 Breakdown of the Strategy Studies by Setting

Setting Number of studies (%)
Urban or peri-urban area only 20 (30)
Rural area only 15 (23)
Mixed setting 19 (29)
Public or governmental only 40 (61)
Any private sector 14 (21)
Other (e.g., household) 12 (18)
Outpatient health facilities 39 (59)
Hospital outpatient departments 20 (30)
Hospital or health facility inpatient wards 15 (23)
Community settings 8 (12)

SOURCE: Rowe, 2015.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

TABLE 4-6 Breakdown of the Strategy Studies by Health Care Provider Type

Type of provider Number of studies (%)
Nurse or midwife 36 (55)
Physician 35 (53)
Nurse or midwife aide 26 (39)
Pharmacist/lab worker 10 (15)
Paramedic 9 (14)
Lay health worker 8 (12)
Health educator 6 (9)

SOURCE: Rowe, 2015.

TABLE 4-7 Breakdown of the Strategy Studies by Health Condition

Health topic Number of studies (%)
Multiple (or all) health conditions 27 (41)
Acute respiratory infections 10 (15)
Diarrhea 10 (15)
Pregnancy 8 (12)
HIV/AIDS +/− other sexually transmitted diseases 8 (12)
Newborn health 4 (6)
Malaria 3 (5)
Malnutrition 3 (5)
Reproductive health (not pregnancy) 2 (3)
Tuberculosis 1 (2)

SOURCE: Rowe, 2015.

60 percent (IQR: 30 to 76 percent) to 11 percent (IQR: 6 to 60 percent) (see Table 4-9). The sharp drop in the effectiveness estimate suggests the studies informing the 60 percent estimate have low generalizability.4 Rowe also pointed out the risk of bias in the data. The strategies with the highest

_________________

4 After the discussion, participants asked if broadening the definition of a strategy changed the effect estimates for other strategies. Rowe said they have observed something similar, though not on the same scale, in the larger database when considering the group-based problem solving and training with supervision strategies.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

TABLE 4-8 Breakdown of the Strategy Studies by Method of Data Collection

Method Number of studies (%)
Record review 46 (70)
Interview with patient or caretaker 17 (26)
Interview with health care providers 14 (21)
Observation of health care provider–patient interaction 13 (20)
Questionnaire for health care provider 10 (15)
Simulated client 4 (6)
Physical exam of patient 2 (3)

SOURCE: Rowe, 2015.

image

FIGURE 4-2 Weighted median adjusted median effect sizes (MES) and interquartile range (IQR) for selected strategies. For the three strategies with greatest MES, all studies had high or very high risk of bias (see Table 4-3).

SOURCE: Rowe, 2015.

effect estimates come from 14 studies whose risk of bias was classified as high or very high (see Figure 4-2).

In an effort to determine where different strategies work best, Rowe stratified the results by country income level. This analysis, though con-

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

TABLE 4-9 Broadening the Definition of Improvement Collaborative with Low-Intensity Training to Adjust for the Probable Confounding of Limited Variability Sharply Reduces Estimates of Effectiveness

Strategy group Number of comparison Median adjusted MES (IQR)
Original definition 3 60 (30, 76)
Improvement collaborative + low-intensity training
Broadened definition 6 11 (6, 60)
Group problem solving + low-intensity training
+/− other components

SOURCE: Rowe, 2015.

ceptually useful, was practically prevented by small sample sizes. Similarly, in attempting to determine the strategies that work best with lay health workers, Rowe identified only four comparisons, all from low-income countries and with a high or very high risk of bias.

A post-hoc analysis aimed to identify factors associated with the effectiveness of training and supervision. The analysis used mixed linear regression modeling and drew on all training and supervision studies in the database (not just the 66 studies identified as relevant to the IOM workshop). The data indicate that, for trainings on one topic, the length of training is not associated with its effectiveness. When several topics are covered in the same training, however, the training’s effectiveness increases by 1 or 2 percentage points for each added day. Only after 5 days do the trainings on multiple topics reach the same effectiveness as a single-day training on one topic. Regarding supervision, strategies emphasizing feedback to the provider appeared to be about 11 percentage points more effective than other types of supervision. Rowe cautioned against over-interpreting these results, however. The models had serious problems with missing data and, therefore, substantial risk of bias and confounding.

Study Limitations

In discussing the limitations of the study, Rowe mentioned the lack of studies on COPE®, SBM-R, and accreditation in the database and how attempts to make COPE®-like study and SBM-R-like study variables failed. He also cited limitations in the studies themselves, such as lack of detail on strategy and context, lack of standardization, difficulty in assessing study precision and strength of implementation, and high risk of bias.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

Discussion

Because of insufficient information about COPE®, SBM-R, and accreditation, it was not possible to compare the effectiveness of the six target strategies. Rowe expressed interest in including gray literature from EngenderHealth and Jhpiego in future additions to the database. As for accreditation, it might simply be a strategy that is not implemented without other components, so its effectiveness should be studied this way.

Results indicated modest effectiveness for both training and supervision. For facilities with a baseline performance of about 40 percent, training and supervision of providers can boost performance to about 50 percent. Combining the two strategies seems to work better than employing either alone, though conclusions about the combination of high-intensity training and supervision should be guarded because of the high risk of bias in these studies. Similarly, improvement collaboratives showed somewhat larger effect estimates, but such results should be interpreted cautiously because of the high risk of bias and limited variability in the data.

Rowe concluded his comments with a request for more studies of rigorous design. He recognized USAID’s need for more information about which strategy is the best investment and encouraged the agency to fund the research that could help determine that, particularly head-to-head comparisons of different strategies. Still, it is not clear who should be responsible for such research. Implementing organizations may have a conflict of interest in evaluating their own programs. Rowe expressed some regret that the database does not include information on the study funder, as people often ask how that might influence the results.

In the session that followed the presentation, participants discussed the study methods. Concerning the adjustment for baseline performance, one participant asked if the higher effectiveness in places with poor baseline performance might be a reflection of the effectiveness of the strategy rather than the low baseline. Rowe agreed, pointing out that this raises a larger question of how to handle contextual factors in analysis. One approach would be to adjust for them until all the comparisons are between similar strategies. Another would be to stratify the analysis by important contextual factors and see how the data vary within strata. These are the kinds of analyses the team plans to undertake in the future.

Rowe was also asked whether his team had considered the scale of the intervention, if the effectiveness changes when 10 providers are involved versus 100 or 1,000. He explained some possible analyses that could address this question, using the number of providers or facilities as a proxy for study scale. The data indicate that, as the study scale gets larger, effectiveness gets smaller. This analysis is subject to risk of confounding, however, as certain strategies tend to be implemented at larger scale than others.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

Several speakers asked Rowe how to distinguish between bias and suitability of strategy to setting. He explained that working in a place friendly to a particular strategy is not itself a bias, but it does limit the generalizability of the results. The contextual factors that improve a place’s receptivity to quality improvement are difficult to measure. Identifying these factors is another important question for implementation research.

HOW DO WE KNOW IT WORKS?: MEASURING CHANGES IN QUALITY

Bruce Agins of New York State Department of Health AIDS Institute started the next session with a brief review of the role for information systems in measuring performance. He described a goal of using real-time data in clinics for continuous improvement. This would allow managers to link process improvements at clinics or in certain regions with corresponding changes in health. Better attention to data collection can also build the health system’s capacity for quality improvement. To illustrate these points and to give an example of the feasibility even in a poor country, Agins shared a case study of a program to improve information technology in the Haitian ministry of health. The President’s Emergency Plan for AIDS Relief (PEPFAR) and the Global Fund5 support the program, both with direct contributions to the ministry and through local provider networks.

The CDC funded two Web-based electronic records systems for the Haitian PEPFAR program: the Monitoring, Evaluation and Surveillance Interface, or MESI, and an electronic medical records system used in 80 percent (n = 144) of the Haitian antiretroviral clinics. In the early days of the program (2004–2005), the monitoring and evaluation system was used primarily for collecting data to report to donors. Then in 2005, I-TECH, a health systems development organization run by the University of Washington and the University of California, San Francisco, with support from PEPFAR and the CDC, developed iSanté, an electronic medical records system that supports both individual and population health. iSanté gave clinicians a way to manage longitudinal data and to make data easily accessible to the ministry.

iSanté is an open-source system developed using Linux OS, Apache Web server, MySQL database, and PHP scripting language. When the program was new, the main servers were kept in Seattle, but eventually they were transferred to Haiti. As of April 2015, more than 100 sites—including government clinics, teaching hospitals, mission hospitals, and nongovernmental organizations—use iSanté to manage more than 500,000 patient records. Eighty-seven of the clinics have local servers, eliminating

_________________

5 Officially, the Global Fund to Fight AIDS, Tuberculosis and Malaria.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

image

FIGURE 4-3 The timeline for developing the iSanté electronic medical records (EMRs) system.

SOURCE: Agins, 2015.

reliance on slow Internet connections and allowing for automatic backup to a central data repository.

Rolling out the iSanté system took about 1 year (see Figure 4-3) and required input from an electrical engineer, a database specialist, a programmer, an analyst, and a network specialist. Ten full-time staff operate the system, and health care workers need a 2-day orientation before they can use it. The cost of implementing the system was $400,000 plus an additional 10 percent annually for ongoing maintenance and technical assistance. Agins acknowledged that the low cost can be attributed in part to idiosyncrasies of the Haitian labor market: Haiti has a proportionately large number of workers qualified for technology jobs, and pay scales in Haiti are much lower than in most other PEPFAR countries. But, even considering the additional ongoing costs, iSanté was inexpensive and good value for the money.

The iSanté system allows the health worker to make different kinds of retrospective or prospective reports, generate case lists, or set reminders. This applies at the level of the patient in clinic, but also at the district or national level. Agins chose one indicator, the enrollment of eligible patients on antiretroviral therapy (ART), and shared it as an example of how electronic medical records can be used to improve quality.

Clinic managers used iSanté to generate their baseline data on ART enrollment. Then, using plan-do-study-act cycles, the clinic staff were able to test the effectiveness of three strategies to improve enrollment, with the goal of increasing enrollment of eligible patients by 20 percentage points. The iSanté system afforded the clinic managers confidence in their data and

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

the ability to see the effect of their changes almost instantly. When they found that reducing the amount of time patients spend waiting to start ART had the best effect on enrollment, they were able to give more attention to that strategy.

Haiti’s national quality advisory board also made use of the electronic medical records system. The real-time data allowed them to set goals, measure their performance, and give feedback to the district and local health offices about their progress. So, in fiscal year 2012, when the national target was to add 10,000 patients to ART, every participating clinic was given a specific goal for enrollment. Analysts used iSanté data to identify the main problems preventing new enrollment and to test different solutions; there was no need to marshal a separate data collection and analysis program. As of September 2014, iSanté data indicated that 83 percent of eligible patients were on ART.

Agins then briefly introduced MESI, the Haitian electronic monitoring and evaluation system. Like iSanté, MESI was a relatively inexpensive program to set up: including recurring maintenance and technical assistance, the platform cost about $200,000. Both systems link local, district, and regional data and enable mandatory case reporting and mandatory reporting of aggregate epidemiological data. Because the value of the electronic systems was quickly evident to the Haitian government and donors, there has been enthusiasm for broadening the platforms to include primary and chronic disease care.

The Haitian example underscores the promise of electronic management systems to improve quality in low-income countries; its success might be replicated, assuming there is political commitment for change. Ministries of health are complicated organizations; in Haiti, the senior leadership was willing to cooperate across departments and allocate staff to the project. Though relatively modest, the additional staffing demands can surpass the workforce capacity in the least developed countries. Agins emphasized that the investment in electronic systems can contribute to sustainability because the platform helps improve efficiency and integrate different work streams, ultimately establishing the link between quality improvement programs and improved health.

In the next presentation, Niek Klazinga of the Organisation for Economic Co-operation and Development (OECD) built on Agins’s closing remarks. He stressed the importance of an information infrastructure for health systems strengthening and reminded the audience to be mindful of how their programs contribute to developing this infrastructure.

Measuring the performance of health systems has been a priority for the OECD for many years. Since 2005, they have used a framework guided by the IOM report Crossing the Quality Chasm (see Figure 4-4). This framework recognizes four functions of the health system: effectiveness,

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

image

FIGURE 4-4 The OECD health system performance assessment framework.

NOTE: HCQI = Health Care Quality Indicators.

SOURCE: Arah, O. A., G. P. Westert, J. Hurst, and N. S. Klazinga. 2006. A conceptual framework for the OECD Health Care Quality Indicators project. International Journal for Quality in Health Care 18 (Suppl 1):5–13. Reproduced by permission of Oxford University Press and the International Society for Quality in Health Care. Adapted from Kelley and Hurst, 2006. As presented by Klazinga on January 28, 2015.

safety, patient-centeredness, and accessibility. The OECD tracks hundreds of indicators that give information about different functions of the health system; to simplify their work for the audience, Klazinga pulled out the set of key indicators most often used to track quality (see Table 4-10). By tracking these indicators over time, the OECD analysts have determined that, for example, survival of patients with acute myocardial infarctions is improving in all its member countries. While richer countries clearly have an advantage, the data show an association between better health outcomes and the amount of policy attention a country gives to quality improvement.

The same information about health outcomes is not readily available in low- and middle-income countries. The joint OECD/World Health Organization (WHO) report Health at a Glance: Asia/Pacific aims to present similar data for all countries in the Asia-Pacific region. Many countries in this region are trying to measure more than the minimum (e.g., infant mortality and vaccination rates) and therefore are investing in improved cancer registries, death registries, and electronic administrative databases; the growing momentum for universal coverage has driven some of these improvements. But tracking the 30-day case fatality rates of chronic disease remains challenging. The ability to link patient data across registries is an essential prerequisite to monitoring most of the OECD’s key indicators.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

TABLE 4-10 Types of OECD Health Systems Indicators

Infectious diseases
  • Vaccination among children
  • Flu vaccination among the elderly
Mortality
  • Infant mortality
  • Maternal mortality
Acute care
  • 30-day case fatality following acute myocardial infarction
  • 30-day case fatality following stroke
Primary care
  • Hospital admission for chronic conditions (diabetes, asthma/ chronic obstructive pulmonary disease, chronic heart failure)
  • Prescribing of antibiotics
Cancer care
  • Screening
  • Mortality
  • 5-year survival
Mental health
  • Excess mortality in persons with severe mental health problems
Patient safety
  • Post-operative complications (sepsis, deep vein thrombosis/ pulmonary embolism)
  • Obstetric complications
Patient experiences
  • Respect
  • Autonomy
  • Communication

SOURCES: Klazinga, 2015. Drawn from OECD, 2013, and Carinci et al., 2015.

Klazinga recommended that the first priority for low- and middle-income countries be developing clinical registries and administrative databases.

Over the past 5 years, the OECD quality experts have been asked to analyze the national quality strategies for a growing list of the non-OECD countries. When they give input, the OECD experts look at the way the health system is governed—for example, the methods used to ensure health worker competence, hospital quality, and the safety of medicines and equipment. The evaluation gives some attention to the country’s efforts at standardization of processes, the use of guidelines, consideration for patients’ rights, and the prominence of a safety strategy. Klazinga reminded the audience that all quality improvement strategies, including the six being discussed at the workshop, complement the national quality improvement architecture. What he described as the three management strategies (COPE®, improvement collaborative, and SBM-R) help develop the overall management culture in a country. Training and supervision can accompany the continuing education and professional development plans. Accreditation is a part of the safety strategy and a way to promote standard guidelines. The OECD’s initial survey data have established that many of

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

the 26 countries in the Asia-Pacific region have a basic quality architecture in place (see Table 4-11); the task is to build on it.

Klazinga closed his presentation with points on ensuring successful project implementation. Management interventions, for example, generally have a certain overall philosophy; it is important to determine if that philosophy meshes well with local management styles. He encouraged training and supervision programs designed to complement local incentive and education structure. Involving policy makers and local experts, even in the program’s pilot stages, helps ensure long-term success.

Following the presentations, participants shared their own experiences with medical information systems. William Tierney of the Indiana University School of Medicine mentioned national programs that are now under way in Bangladesh, Kenya, Mozambique, the Philippines, Rwanda, and Tanzania. There is an opportunity to use these systems—all at different stages of development—to build national quality improvement programs, though Tierney expressed concern that connections between the new electronic systems and quality managers were somewhat haphazard. Mike English built on this point, saying that the electronic medical records system he worked with in Kenya is not as sophisticated as the example Agins shared from Haiti. He found that the HIV care system was far superior, but parallel, to the national health information systems. Problems of interoperability, which Western countries are struggling with too, are vastly aggravated in Kenya, where various small entrepreneurs develop software packages and sell them to hospitals.

The participants also discussed the promise of electronic medical records systems, as well as some risks to control. Klazinga had concerns that most of the attention to information technology was in hospitals. Many OECD

TABLE 4-11 Number and Percentage of 26 Asia-Pacific Countries Reporting Selected Components of a Quality Improvement System

Component of national quality improvement system Number of countries (%)
Mandatory continuing medical education or continuing professional development 16 (61.5)
Mandatory hospital accreditation 8 (30.8)
Voluntary hospital accreditation 13 (50.0)
Technology assessment studies on medicines 15 (57.7)
Standards on safe blood use 23 (88.5)
Pharmacovigilance system 21 (80.8)

SOURCES: Klazinga, 2015; OECD and WHO, 2014.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

countries are now at a point where poor information about primary care is limiting their ability to monitor quality of care. It is not possible to know, for example, if someone is hospitalized for a chronic condition without integrating primary care and hospital databases. The problem could be avoided in low- and middle-income countries with attention to computerized information systems in primary care. DHIS 2, a health information system used in 47 low- and middle-income countries and 23 international organizations, has the ability to link hospital and primary care data and to control the burden of data collection on health workers (DHIS 2, n.d.).

There was also discussion of the best ways to measure quality in the poorest countries. Information technology, though full of promise, is going to remain out of reach for the poorest patients, many of whom are treated in their homes or in rural clinics. A participant pointed out that only half of the world’s newborns are even weighed at birth and asked how, in such settings, we can ensure reliable information on chronic disease. Klazinga agreed and suggested that the most realistic first steps would be developing basic death registries and then cancer registries. He cited success from South America where cancer registries have developed rapidly over the past 5 years.

AT WHAT COST?: WHAT MAKES A PROGRAM COST-EFFECTIVE

In the last panel discussion of the day, Edward Broughton of URC and Dinesh Nair of the World Bank discussed cost-effectiveness and feasibility in quality improvement. Broughton opened his presentation with a simple example of cost-effectiveness in health. If a person knows his or her likelihood of getting sick, weighing the costs of treatment against foregone wages is relatively simple. But in quality improvement, although the basic principle is the same, the analysis gets complicated quickly. For example, if the USAID mission to Liberia did a quality improvement program in connection to the Ebola response, the epidemiology of Ebola would not be the only thing changing in the health system. Clinics that treat Ebola patients presumably treat others as well; attention to these conditions would improve, suffer, or stay the same (see Figure 4-5). Cost-effectiveness analysis needs information on the cost and health consequences of all those variables. Usually, as Figure 4-5 shows, the analysis accounts for costs and consequences associated with improving or not improving treatment. Accounting for changes in mortality requires sophisticated epidemiological modeling. Modeling depends on assumptions that are invariably debatable, leaving the final conclusions open to criticism.

For one thing, it is difficult to say how long the effects of a quality improvement intervention last. The initial intervention might go on for 6 months to 2 years. Modeling must make assumptions about how the im-

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

provements will continue after the technical experts leave. While it is probably not sensible to assume processes return to baseline after the project, it is also imprudent to suppose that all gains are sustained. Sometimes the local host agency expands the program after the USAID implementing partner leaves. Estimating cost-effectiveness requires making assumptions about how effective the implementation will be with different technical staff in charge. Finally, Broughton stressed that cost-effectiveness analysis generally takes the perspective of the costs to the funder—in the case of his organization that means the cost to USAID. A stronger model would consider the costs to society and thereby give the host country ministry the information it needs to determine cost-effectiveness.

After describing the factors that make health systems research difficult, Broughton advised that the only justifiable course of action was to accept the inherent challenges of such research and get on with it. The complexity of the research is not, he maintained, an excuse for not doing it. A quick review of cost-effectiveness studies indexed to PubMed found 2,962 articles about the cost-effectiveness of quality interventions, but only 10 (0.34 percent) of those papers actually published results for cost-effectiveness analysis (see Figure 4-6).

Of the 10 papers that published economic analysis of quality interventions, half were from the USAID Applying Science to Strengthen and Improve Systems (ASSIST) project. None of the analyses, including those from the ASSIST collaboratives, looked explicitly at the effectiveness of the different brand names being discussed at the workshop. In seven of the studies, researchers found the quality changes to be cost-effective; in the

image

FIGURE 4-6 A quick literature review of cost-effectiveness studies indexed to PubMed found only 10 papers included cost-effectiveness analyses.

SOURCE: Broughton, 2015.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

remaining three, the interventions were found to be cost-saving, meaning the change not only improved health but also saved money.

Broughton was frank about the limitations of these studies. None of the USAID-funded research used control groups. There were also problems with generalizability and short time frames. BMJ guidelines for cost-effectiveness analysis ask investigators to report certain information about the method of modeling, confidence intervals on key estimates, and sensitivity analyses; the guidelines give considerable attention to describing uncertainty in the data, threats to the study validity, and the assessment of outcomes. The 10 studies of cost-effectiveness of quality interventions did not rank highly on these criteria and would probably not have met publication standards for BMJ and similar journals. The fact that all 10 studies found positive results raises questions about publication bias. Because all the studies were evaluated by people close to the implementation, the risk of confirmation bias also seems high.

Broughton concluded his talk by observing that cost-effectiveness research is a serious weakness in quality improvement. He sees this weakness as threatening to undermine the whole quality improvement field and echoed a sentiment expressed in earlier presentations about the need for more randomized trials of improvement interventions.

Nair built on similar themes in his presentation on feasibility and cost-effectiveness in quality strategies. He agreed that economic evaluation of quality strategies—research weighing the costs and benefits of correcting inefficiency—is neglected. The few studies published in peer-reviewed journals tend to look at a narrow group of maternal and child health outcomes. More importantly, cost-effectiveness depends on the comparisons shown in Figure 4-7. This analysis can fail to balance costs against feasibility.

Nair felt each of the six methods being discussed at the workshop had feasibility issues. For example, the standards-based methods can face problems with inconsistencies in the national guidelines. Many of the methods have requirements for foreign technical support, decreasing the feasibility. Nair emphasized the value of evaluating the methods under discussion, and all quality improvement strategies, against a strong counterfactual.

In the final discussion of the day, the participants reflected on the limitations of the data. One participant brought up the possibility that the problem of poor quality data is not likely to change and suggested that global health researchers might do well to anticipate this, moving to Bayesian approaches. Classical statistics need a kind of data that quality improvement simply may not be able to provide. On the other hand, classical statistics command an authority in medicine where there is wide acceptance that decisions should be evidence based. Although health systems researchers in developing countries do not have the same resources available to them as

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

image

FIGURE 4-7 General structure of economic evaluations.

NOTE: QALY = quality-adjusted life year.

SOURCE: Favato, 2008, as presented by Nair on January 28, 2015.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

other kinds of medical research, there is still a need for convincing evidence and an obligation to provide it. If the case can be made, with objective data, that quality improvement can actually save a country money, then ministers of health and finance will be eager to support it.

The discussion also touched upon implementation research with some participants acknowledging the constraints of working with foreign hosts. A host country agency may request, for example, that a project take place in 100 clinics in 2 districts; they often choose the districts and set some of the terms of the project. Some guests suggested that the implementers could try to persuade their hosts to implement the project in two phases, randomly assigning clinics to begin the intervention at different times in a step-wedge design. While not the strongest design, such models would be an improvement on what is often done now.

Participants highlighted opportunities for future quality improvement work to confront the weaknesses discussed at the workshop and improve on them. Studies can be made stronger by articulating clear aims and tying those aims to the improvement strategy being used. Project evaluations can be given better priority by planning early, allocating time and money for evaluation from the start. Other participants questioned the relevance of the experimental research design for quality improvement given the importance of context. Some of their colleagues responded that one way to address this concern is for researchers to describe the context and limitations of their work, so that future work can weigh the likely relevance of a given study to their setting.

A slight contradiction was identified between the first sessions of the day, where experts on the six strategies emphasized the importance of data collection to each of their methods, and the last sessions, where there was a consistent concern that no one has data about quality improvement. Although the morning speakers explained how quality improvement programs collect considerable data, Broughton made the point that quality improvement research requires information about a comparable control group. Jishnu Das of the World Bank pointed out that publicly available data, such as the Demographic and Health Surveys (DHS), can be valuable in closing the data gap. Triangulating controls off DHS data could help solve the problem of the poor control group, though DHS data is sometimes not de-aggregated below the district level.

CLOSING REMARKS

After the discussion, Kedar Mate of the Institute for Healthcare Improvement gave closing remarks on the day’s proceedings. He described the workshop’s main theme as improving the science of quality in low- and middle-income countries. Implementation experts, policy makers, academics,

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

and researchers may come at the problem from different perspectives, but the goal is the same.

Over the course of the day, participants heard a good deal about accreditation, COPE®, collaboratives, SBM-R, supervision, and training, but less about the wider quality improvement movement. Mate echoed Ashish Jha’s judgment that the six methods are more similar than different, and that the ultimate success or failure of a strategy has as much to do with its suitability to a particular environment as its technical merits. At the same time, it is imprudent to be too confident about the value of any method, as some of the most promising results come from the weakest study designs. Several presenters had brought up the need for more head-to-head comparisons of different strategies. There was similar demand for independent evaluation of quality improvement work, with the distance between evaluator and implementer allowing for more confidence in the results.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×

This page intentionally left blank.

Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 35
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 36
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 37
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 38
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 39
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 40
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 41
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 42
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 43
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 44
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 45
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 46
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 47
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 48
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 49
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 50
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 51
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 52
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 53
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 54
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 55
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 56
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 57
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 58
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 59
Suggested Citation:"4 Reviewing the Evidence for Different Quality Improvement Methods." National Academies of Sciences, Engineering, and Medicine. 2015. Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary. Washington, DC: The National Academies Press. doi: 10.17226/21736.
×
Page 60
Next: 5 Synthesizing Evidence, Identifying Gaps »
Improving Quality of Care in Low- and Middle-Income Countries: Workshop Summary Get This Book
×
Buy Paperback | $54.00 Buy Ebook | $43.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Quality of care is a priority for U.S. Agency for International Development (USAID). The agency's missions abroad and their host country partners work in quality improvement, but a lack of evidence about the best ways to facilitate such improvements has constrained their informed selection of interventions. Six different methods - accreditation, COPE, improvement collaborative, standards-based management and recognitions (SBM-R), supervision, and clinical in-service training - currently make up the majority of this investment for USAID missions. As their already substantial investment in quality grows, there is demand for more scientific evidence on how to reliably improve quality of care in poor countries. USAID missions, and many other organizations spending on quality improvement, would welcome more information about how different strategies work to improve quality, when and where certain tools are most effective, and the best ways to measure success and shortcomings.

To gain a better understanding of the evidence supporting different quality improvement tools and clarity on how they would help advance the global quality improvement agenda, the Institute of Medicine convened a 2-day workshop in January 2015. The workshop's goal was to illuminate these different methods, discussing their pros and cons. This workshop summary is a description of the presentations and discussions.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!