National Academies Press: OpenBook

Medicare's Quality Improvement Organization Program: Maximizing Potential (2006)

Chapter: 9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer

« Previous: 8 Technical Assistance for Quality Improvement
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 230
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 231
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 232
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 233
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 234
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 235
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 236
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 237
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 238
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 239
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 240
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 241
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 242
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 243
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 244
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 245
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 246
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 247
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 248
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 249
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 250
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 251
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 252
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 253
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 254
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 255
Suggested Citation:"9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 256

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

9 Impact of Technical Assistance for Quality Improvement and Knowledge Transfer CHAPTER SUMMARY This chapter reviews the literature base for quality improvement and knowledge transfer, two concepts important to the provision of technical assistance to help health care providers improve the quality of care that they provide. To understand the effectiveness of the technical assistance provided through the Quality Improvement Organization (QIO) program or the extent to which the program goals have been achieved, it is necessary to first understand how similar quality improvement efforts have fared in the larger health care environment outside of the QIO program. This chapter pre- sents discussions on the following topics related to quality improve- ment: quality improvement interventions in general and in QIOs, approaches to quality improvement, and an overview of knowl- edge transfer and its impact within the QIO environment. QUALITY IMPROVEMENT The Quality Improvement Organizations (QIOs) seek to achieve qual- ity improvement through the use of various interventions to enhance the efficiency and effectiveness of care received by Medicare beneficiaries. This section examines the impacts that QIOs' quality improvement interventions have on the delivery of health care. With specific reference to the QIOs, the Centers for Medicare and Medicaid Services (CMS) defines interventions as activities adopted by providers, beneficiaries, or the QIOs to facilitate change to improve health care delivery processes, structures, or behaviors (CMS, 2005b). To assess and explain the impacts of quality interventions by the QIOs, the impacts of these interventions throughout the health care industry must be reviewed. Thus, this discussion first examines the litera- 230

IMPACT OF TECHNICAL ASSISTANCE 231 ture regarding quality improvement in general and then assesses the litera- ture on QIO quality improvement interventions. A literature review conducted for this project provides an overview of evidence on improving health care quality. (For details on each study, see Table A.1 in Appendix A.) The studies included in the literature review were categorized by study design; the findings of studies with more rigorous methodologies were considered more heavily (see Table A.2 in Appendix A). The studies reviewed paint an inconclusive picture of the effectiveness of quality improvement programs, whether they are conducted by QIOs or other organizations, for both Medicare and nonMedicare services. Although health care quality improvement interventions have been dis- cussed for decades, the emerging evidence base supporting their effective- ness remains sparse and therefore difficult to use as a basis for making policy decisions. Comprehensive studies of specific types of interventions are limited in part because of the many different methods of approaching quality improvement. Quality improvement resulting from specific inter- ventions, however, is difficult to measure because many of the impacts of the interventions are qualitative, it often takes more time than is allowed by the study to demonstrate measurable improvements, and the interventions themselves are not described at a level of detail that allow them to be replicated. Quality improvement interventions tend to target multiple components of complex organizations, all of which are subject to many internal and external influences, making evidence of effectiveness almost impossible to detect (Ovretveit and Gustafson, 2003). If improvement has been made, attribution of this success cannot be determined because of the wealth of players often involved in enhancing the quality of care. Conversely, the reason why quality improvement interventions fail also remains inconclu- sive, despite qualitative studies suggesting that specific organizational cul- ture characteristics play a role (Bradley et al., 2001). These limitations in the assessment of quality improvement overall, as well as the assessment of specific quality improvement interventions, re- sult in various types of study designs and different levels of reliability of the study results. Few of the studies reviewed contained true control groups, and even fewer were the more stringently devised randomized control trials. The majority of the studies measured improvement as the change from the baseline in the group receiving the intervention and were either prospective or retrospective in design. One research method used to temper some of the limitations and control for changes in the environ- ment is to stagger implementation of the intervention. The intervention is put into practice twice: once in the original study intervention group and once again in the designated control group after the conclusion of the original study (Chu et al., 2003). Analyses assessing time trends are espe-

232 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM cially important in evaluations of quality improvement interventions, as the desired changes often take longer to achieve than the lengths of the interventions themselves and because a potential disparity exists between short- and long-term achievements. Impact of Health Care Quality Interventions As mentioned above, many different approaches have been tried to achieve quality improvement in health care. Although some studies that the committee examined found no change in the quality of care delivered as a result of the selected interventions; most cited some level of improvement ranging in levels of statistical significance. Most interventions approach quality improvement by targeting three aspects of health care: structure, process, and outcome. Structure refers to the characteristics of a care set- ting, including material resources, human resources, and organizational structure. Process describes what and how care is actually provided and received. Outcome denotes the impact of health care services on health sta- tus and patient satisfaction. In theory, improvement in structure drives a good process, which in turn drives a good outcome (Donabedian, 1988). The following discussion assesses the literature on interventions that target processes and outcomes and then discusses the literature on other interven- tions that focus on structure and audit-feedback. Process Measures One common finding in the literature was the demonstrated improve- ment in process measures due to the implementation of clinical practice guidelines. This conclusion was determined from the findings of both ran- domized controlled trials and cohort studies (Ornstein et al., 2004; Joseph et al., 2004; Halm et al., 2004). Clinical practice guidelines are evidence- based recommendations developed to direct decision making for the provi- sion of care. An example of a guideline for a process measure is the percent- age of providers who document taking a patient's blood pressure during an office visit. The use of process guidelines generally led to increases in the documentation of care processes for a variety of conditions (cardiovascular diseases, pneumonia, and tobacco use) and a variety of care settings (physi- cians' offices and hospitals). However, the resulting levels of statistical sig- nificance varied, with studies citing significant improvement in only one of many measures (one study looked at 14 process measures) (see Table A.1 in Appendix A). One randomized controlled trial, however, evaluated guide- lines in the context of a larger, more systemwide intervention, and found only marginal change in physician adherence (Ornstein et al., 2004). The more systemwide effects of the intervention were not separated from those

IMPACT OF TECHNICAL ASSISTANCE 233 of the guidelines themselves, complicating any conclusions that guidelines alone enhance quality. The provision of guidelines also does not necessarily lead to the dissemination of new knowledge regarding care practices (Centor et al., 2003). Outcomes Another theme detected in the literature review was a lack of demon- strated improvement in health outcomes (such as patient health status) as a result of the use of treatment guidelines for desired outcomes. For example, a desired outcome derived from the observance of diabetes care guidelines is control of hemoglobin A1c levels in diabetes patients to less than 8 per- cent mg/dL. The same studies that failed to demonstrate improvement in process measures were used to evaluate outcomes. These studies were de- signed as randomized controlled trials and cohort studies. The studies evalu- ated multiple conditions and care settings. Outcomes based on treatment guidelines did not change significantly during the study periods, with use of the guidelines having from no impact to only a marginal impact (Ornstein et al., 2004; Joseph et al., 2004; Halm et al., 2004) (see Table A.1 in Ap- pendix A). The impacts of interventions on outcomes may, however, take longer to identify than the duration of the study, thereby resulting in false conclusions about an intervention. Changes in outcomes may also be influ- enced by patient behaviors, over which providers have limited control. Other Interventions Conclusions about other types of interventions cannot be drawn be- cause of a lack of a robust evidence base and inconsistent results. For in- stance, research gaps exist concerning structural issues and audit-feedback (Jamtvedt et al., 2004; Coleman et al., 2004; Mark et al., 2004). Although improved performance on the process and outcome measures presented in studies are a good starting point for obtaining improvements in quality, performance should ultimately keep getting better. Structural issues such as nurse staffing levels were addressed in a cohort study. Im- provements in mortality rates were found in association with increases in the numbers of registered nurses on staff, but the improvements could not be solely ascribed to those increases. Also, a diminishing marginal effect of increased staff members was found on improvements of mortality rates. Although the evidence base for improvements in health care quality attrib- utable to structural changes is emerging, it is sparse (Mark et al., 2004). Provider and organizational characteristics are also important struc- tural issues to be considered for the achievement of continuous improve- ments. One qualitative study used interviews to evaluate whether cor-

234 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM relations between provider characteristics and quality improvement exist (Bradley et al., 2001). The researchers identified provider characteristics and measured quality improvement in terms of the percentage of patients who had received beta-blockers at discharge. Hospitals with greater amounts of improvement had four similar characteristics: shared goals throughout the institution, strong administrative support, high levels of physician leadership, and high-quality feedback. The levels of innovation were not, however, found to correlate with high or low levels of perfor- mance (Bradley et al., 2001). To prepare an organization to be receptive to change, researchers iden- tified the following dimensions of success: strategy, culture, technique, and structure. The improvement mechanism must target strategic conditions and processes within the organization. The organization must foster a culture that supports the mechanism, and it must ensure that staff are properly trained and given the necessary tools to implement the intervention techni- cally. The last dimension is structure, which refers to the mechanisms used to adopt and spread better practices throughout the organization. All four dimensions must be present for successful change to occur (Shortell et al., 1998; Heller and Arozullah, 2001). However, it is very difficult to develop the strategic and cultural dimensions if the organization does not already have these attributes. Technique and structure are somewhat easier to de- velop. For example, the organization can purchase expertise, but it takes longer to develop a supportive culture. In health care settings, audit and feedback refers to the process in which provider performance is evaluated and reported back to the provider, which allows the provider to make improvements. The committee's present review found that the audit and feedback method inconsistently provides signifi- cant improvements. The coupling of these efforts with other types of im- provement interventions also did not yield better results. A major limitation to the committee's systematic review was the lack of high-quality studies, as it was noted that the participants in many studies had low levels of compli- ance at the baseline and the studies had small sample sizes. Improved re- porting of the details about the actual intervention was also found to be needed (Jamtvedt et al., 2004). Impact of QIO Quality Improvement Interventions with Providers The evidence about the impact of QIO quality interventions compared with the impact of other health care quality interventions is mixed. This conclusion is most prominently derived from studies in which researchers found that the trends for hospital systems and hospitals that did and did not participate in QIO interventions were similar (Ellerbeck et al., 2000; Dellinger et al., 2005).

IMPACT OF TECHNICAL ASSISTANCE 235 Process Measures Studies using process measures to evaluate the effects of QIO efforts to improve quality used a variety of designs: randomized controlled trial de- signs, quasiexperimental designs, cohort study designs, and cross-sectional study designs. These studies looked at the use of practice guidelines for the care processes for multiple conditions (diabetes, cardiovascular disease, and pneumonia) and were conducted in hospitals, academic medical centers, and physicians' offices. Echoing the findings from the quality improvement intervention literature of studies assessing non-QIO-related interventions, improvements in quality were found as a result of the QIO-related interven- tions (Marciniak et al., 1998; Ellerbeck et al., 2000; Holman et al., 2001; Kiefe et al., 2001; Sheikh and Bullock, 2001; Sutherland et al., 2001; Luthi et al., 2002; Gould et al., 2002; Chu et al., 2003; Burwen et al., 2003; Berner et al., 2003; McClellan et al., 2003; Massing et al., 2003; Daniel et al., 2004a). Outcomes Many studies looked at the impact that QIO interventions have on outcomes. Those studies examined a variety of outcomes in patients with diabetes, cardiovascular disease, and pneumonia seen in the hospital and the physician's office settings and used quasiexperimental and cohort meth- odological designs. As has been found in the broader literature on quality improvement, interventions by QIOs to improve outcomes have not yet been demonstrated to result in significant change. Although some studies found improvements in quality, the results of many of them are inconclu- sive. The limitations described in the broader literature on quality improve- ment are the same, such as a limited ability to control patient behaviors (Marciniak et al., 1998; Ellerbeck et al., 2000; Sheikh and Bullock, 2001; Sutherland et al., 2001; Holman et al., 2001; Luthi et al., 2002; Gould et al., 2002; Chu et al., 2003; McClellan et al., 2003; Massing et al., 2003; Daniel et al., 2004a) (see Table A.1 in Appendix A). Only one study evaluated in the committee's literature review focused on patient behaviors. That study examined patient adherence to a health maintenance organization's guidelines for the frequency of mammography upon receipt of one of three types of reminders. This QIO-related ran- domized control trial found that only telephone reminders coupled with the option to schedule an appointment were effective; publicity campaigns encouraging screening and mail reminders were not effective (Barr et al., 2001).

236 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Other Interventions Studies of other types of interventions that have been performed by QIOs or that use improvement tools developed by QIOs have yielded in- conclusive results. For example, the impact of educating second-year medi- cal students on the use of process guidelines and reminders provided by QIOs generated some significant improvement (Gould et al., 2002). A study conducted by a QIO audit and feedback in hospitals could not conclusively find methods to yield improvements for the following five conditions: acute myocardial infarction, heart failure, pneumonia, stroke, and atrial fibrilla- tion. Consistent with the general limitations of quality intervention studies, a true causal relationship could not be drawn because of the lack of con- trols in that study (Schade et al., 2004). National Evaluations of QIO Technical Assistance Efforts Various reviews have evaluated various elements of the technical assis- tance provided by the QIO program, but none could ascribe the improve- ments directly to the efforts of the QIOs. One study compared the national and state-level improvements in 22 QIO measures of quality in the hospital and the physician's office settings between the time periods of 1998­1999 and 2000­2001 (from the end of the 6th scope of work [SOW] to the beginning of the 7th SOW). That cross- sectional study built upon earlier efforts that found that the quality of care provided by the states was inconsistent (Jencks et al., 2000). The study found that care for fee-for-service beneficiaries on the whole improved dur- ing this time period, as the national and the state averages for 20 of the 22 measures increased. In general, states with lower baselines yielded higher rates of absolute improvement in the quality of care. The study also ranked states on the appropriate provision of care on the basis of the same 22 measures. The result was a geography-specific pattern of care, with better care being delivered in the northern states than in the southern states. The levels of improvement followed a similar pattern. The improvements cited in that study could not be attributed directly to the QIO program, however, for many of the reasons discussed above. However, that study was designed to look at quality trends and not to assess the impact of the QIO program (Jencks et al., 2003). Another study randomly surveyed 105 hospital directors of quality of care across the nation to determine their perceptions of QIO effectiveness. That cross-sectional study found that 60 percent of quality directors be- lieved that certain QIO interventions, such as the provision of performance data and education materials, were helpful or very helpful. However, the same survey disclosed that in the absence of QIO interventions, only 25 per-

IMPACT OF TECHNICAL ASSISTANCE 237 cent of directors thought that the quality of the care delivered would have been worse. Although the study found that QIOs were mostly viewed as partners for improving care, as opposed to their previous role as adversar- ies, the researchers noted the need to engage senior-level support (both phy- sicians and hospital management) to further the QIOs' impacts. These find- ings were based on the opinions of those interviewed and not on quantitative measures of actual impacts (Bradley et al., 2005). A recent study evaluated the effectiveness of QIO interventions on im- proving the quality of care in hospitals. That quasiexperimental study com- pared the differences in care in five clinical areas (atrial fibrillation, acute myocardial infarction, heart failure, pneumonia, and stroke) delivered by participating (n = 199) and nonparticipating (n = 142) hospitals in Mary- land, New York, Nevada, Utah, Washington state, and Washington, D.C. The researchers collected data on 15 indicators at the baseline (1998­1999) and at the follow-up (2000­2001). The data revealed that nonparticipating hospitals were generally for profit and smaller than the participating hospi- tals. (The researchers defined "participating hospitals" as those that either collected measurement data or implemented changes in their procedures as a result of efforts made by their respective QIOs.) At the baseline, data for 5 of 15 indicators differed significantly, with nonparticipating hospitals performing better than participating hospitals on 2 of these 5 indicators; participating hospitals performed better on the other three indicators. At the follow-up, the data showed significant differences on four indicators, with the participating hospitals performing better on all of them. However, at the baseline, differences between participating and nonparticipating hos- pitals were found on only two of these four indicators. When the baseline performance and the follow-up performance were compared, researchers found significant differences (P < 0.05) between participating and nonpar- ticipating hospitals for only 1 of the 15 indicators (the patient was screened for pneumococcal immunization or given the pneumococcal vaccine [P = 0.005]). They concluded that, overall, hospitals participating with QIOs were no more likely to improve than nonparticipating hospitals. Impor- tantly, a general trend of improvement was found for both participating and nonparticipating hospitals (Snyder and Anderson, 2005). The findings of that study concur with those of other studies that show that quality improvements have been made over time, but the study con- cludes that the improvements cannot be directly attributed to the activities of the QIOs (Jencks et al., 2003). Although that study has strength because of its use of a comparison group, it also has limitations. In particular, the distinction between participating and nonparticipating hospitals is very broad, and precise descriptions of the interventions are lacking. A lack of descriptive details also makes it difficult to discern the exact roles that the QIOs played. The effects of other quality interventions that were concur-

238 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM rently under way in the hospitals included in the study are unknown. Each participating hospital did not necessarily participate in QIO activities or focus on improving all 15 indicators at the same time, which biased the impacts of the interventions and thus does not allow comparisons of the improvements on specific indicators within a clinical area. Additionally, the study examined only a limited subset of the hospitals in the country and only one portion of the QIO program. Moreover, the data reflect the results from the 6th SOW; evaluated changes thus do not reflect the major strate- gic shifts made by the QIO program in the 7th SOW and is not indicative of the effectiveness of the current program. Although the studies described above cannot ascribe improvements in the quality of the care delivered directly to the efforts of the QIOs, this is not to say that QIO interventions have been unsuccessful. Some studies find that the efforts of QIOs have had positive impacts. One of the building blocks for promoting quality improvement through the QIO program is the Cooperative Cardiovascular Project (see Chapter 1). The Cooperative Car- diovascular Project was part of the initial movement in 1992 to shift the strategy of CMS to providing technical assistance to providers (Jencks and Wilensky, 1992). The Cooperative Cardiovascular Project pilot lasted from 1992 to 1995 and assessed quality improvement activities for acute myo- cardial infarction patients in Alabama, Connecticut, Iowa, and Wisconsin. The quality of care was evaluated on the basis of improvements in 26 mea- sures of processes and outcomes. The researchers found that not all eligible patients received treatments either at all or in a timely manner. Despite this finding, the Cooperative Cardiovascular Project successfully attained sig- nificant documented improvements in process measures (Ellerbeck et al., 1995; Marciniak et al., 1998; Holman et al., 2001; Burwen et al., 2003), thereby promoting the use of quality improvement efforts. In addition to the Cooperative Cardiovascular Project, other studies of QIO activities support the effectiveness of QIO interventions (Sutherland et al., 2001; Gould et al., 2002; Chu et al., 2003; Daniel et al., 2004b). While these studies all targeted physicians, no other commonalities were readily identified in the articles. The positive effects were limited, and do not demonstrate the impact of the QIO program overall. Because of the QIO program's voluntary nature, its goal of the provi- sion of a public good, and the QIOs' method of involving many partners in each intervention program, it is difficult to attribute in a causal manner the activities of QIOs to quality improvement, as these activities cannot be eas- ily distinguished from those of other organizations also working to improve quality (Jencks et al., 2000). This does not mean that the QIO program is ineffective; rather, it is difficult to measure its effect separately, as is the case with quality improvement efforts in general. Although other organizations work to enhance the quality of care, their efforts can work in tandem with

IMPACT OF TECHNICAL ASSISTANCE 239 the efforts of the QIOs. For example, a demonstration project in Texas found that the efforts of a state-sponsored program did not duplicate those of a QIO that provided technical assistance (Cortes, 2004). Given the limited research available, it remains unknown what drives both successful and unsuccessful quality interventions by QIOs. More evi- dence is needed to identify these potential drivers. While the QIO program looks to further develop its quality improvement activities, some lessons can be gleaned from the rest of the industry. APPROACHES TO QUALITY IMPROVEMENT Many approaches to managing quality improvement focus on improv- ing systemwide processes within provider settings; these approaches are not limited to use within health care settings, however, and are often adapted from other industries for use by health care organizations. As many health care organizations work toward improving quality and decreasing costs, these approaches have become increasingly more relevant to understand. If QIOs are to successfully work in collaboration with some of the hospitals, physicians' offices, and health care plans that have turned to these methods, the QIOs need to be informed. When these processes are implemented, it is important for all players (providers) to agree to participate as well as to keep the customers (patients) as the focus for improvement. The approaches described below represent a mixture of tools, methodologies, and goals for improving quality; they are not independent of each other, as some focus on streamlining processes within individual organizations, whereas others tar- get large organization or multi-organization systems. In addition, process and systems are often combined to achieve higher quality: · Baldrige criteria. Baldrige criteria are indicators of organizational performance excellence used to evaluate organizations in different indus- tries. The seven criteria are leadership; strategic planning; customer and market focus; measurement, analysis, and knowledge management; a focus on human resources; process management; and results. Organizations that apply and exemplify these criteria are awarded the Baldrige National Qual- ity Award, which symbolizes excellence in quality and performance. The United States Congress established the Baldrige National Quality Award in 1987, and it is presented annually by the President of the United States. Awards are given for five sectors: manufacturing, service, small business, education, and health care. In any given year, awards may be given to one or more organizations in any or all of the five sectors. Recent award recipi- ents for the health care sector include: Saint Luke's Hospital of Kansas City, Kansas City, MO (2003); Baptist Hospital, Inc., Pensacola, FL (2003); Robert Wood Johnson University Hospital Hamilton, Hamilton, NJ (2004);

240 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM and Bronson Methodist Hospital, Kalamazoo, MI (2005) (NIST, 2005). Individual states have also created similar awards on the basis of the criteria presented above. · Collaborative methodology. The model for improvement has been extrapolated to the collaborative methodology, which refers to a semi- structured gathering of providers from various health care organizations to improve a common process of care by sharing experiences, best practices, and lessons learned with each other (Ovretveit et al., 2002). The methods used often include meetings, web-based conferences, and teleconferences (see Chapter 8 for further discussion). · Human factors. Human factors is a tool used to redesign processes and systems to better use the attributes controlled by both the physical and the cognitive abilities of the people involved in the delivery of care. By understanding the human aspects of why errors occur, processes and sys- tems can become more safe, effective, efficient, and patient centered (NAE and IOM, 2005; Qualis Health, 2005c). · International Standards Organization Standard 9000 (ISO 9000). ISO 9000 is a standard that provides process guidelines and requirements for quality management. On the basis of an external audit, an organization can be certified to ISO 9000, which signifies that it has demonstrated the ability to adhere to processes of quality improvement. The ISO 9000 family includes guidelines for quality management, quality management systems, and quality assurance. · Lean principles. Lean principles aim to streamline processes with the goal of reducing waste and eliminating zero-value-added tasks and re- sources. Lean principles can also be used to simplify systems. For example, by tracking patients across the care system, services can become more effec- tive and efficient in improving quality and minimizing costs. · Plan, Do, Study (Change), Act (PDSA) Cycle. The PDSA Cycle is a method for continuously improving the quality of processes: plan for a change, do a trial of the planned change, study the results, and act to imple- ment the next steps based on the results. In the PDSA Cycle, change refers to implementing change in the process. · Six Sigma. Six Sigma is a data-driven problem-solving methodology that is used to minimize variations in processes and builds on statistical process control. Individuals can build Six Sigma competencies through train- ing and continuous practice with application of the method in various projects. The competency levels include green belt (entry level), black belt (middle), and master black belt (expert). · Root-cause analysis. Root-cause analysis is an approach taken to understand the reasons why an event has occurred. The recurrence of ad- verse events can be eliminated by looking systematically at the managerial

IMPACT OF TECHNICAL ASSISTANCE 241 processes behind the series of actions that lead up to an event (American Society for Quality, 2005). By learning and implementing the skills required for the use of these ap- proaches, organizations can streamline their own resources as well as be- come drivers of improvement by teaching these methods to others who pos- sess the necessary infrastructure support. QIO Support Centers (QIOSCs) and QIOs train staff on many of these approaches and instruct both other QIOs and providers to use them within their organizations (personal com- munication, L. A. Baseflug, August 19, 2005) (see Chapter 7 for a discus- sion of the accreditations and awards held by QIOs and the organizations holding QIO contracts). The Process Improvement QIOSC conducted the Health Care Quality Improvement Project Improvement Methodologies Survey in September 2004 (the results are presented below) to measure QIOs' familiarity with and use of six of the methodologies for quality improvement described above: Baldrige criteria, collaborative methodology, human factors, ISO 9000, lean principles, and Six Sigma (Qualis Health, 2005c). In total, 99 respondents from 41 states completed the surveys. The respondents (75 per- cent of whom were project managers, project coordinators, or departmen- tal directors) were the most familiar with and used the collaborative meth- odology. This is expected, because the QIO program strongly promoted the collaborative methodology during the 7th SOW. The survey responses showed familiarity with all methodologies, although familiarity with ISO 9000 and lean principles scored the lowest (49 and 45 percent, respectively). Even though, on average, half of the respondents stated their familiarity with the six methodologies, an average of only 28 percent of the respon- dents noted that they actually used the methodologies. The respondents remarked that the methodologies that they would most like to receive train- ing on are human factors and lean principles (76 and 74 percent, respec- tively) (Qualis Health, 2004). Training in human factors can be very rel- evant in impacting patient safety (Gosbee, 2002; Silver et al., 2004), while lean principles are useful in adding value to processes (Toussaint, 2005). Two of the methodologies described above are particularly pertinent to the QIO program: the collaborative methodology and lean principles. In the 7th SOW, CMS heavily promoted the use of collaboratives, as described in Chapter 8. In the 8th SOW, CMS identified the use of lean principles as a strategy for promoting transformational change for case review (CMS, 2005c). Although this strategy will be most closely tied to making case review more efficient, the lessons learned from lean principles will be ap- plied where applicable. The use of quality improvement collaboratives is beginning to be evalu- ated. Like quality improvement interventions, the designs of collaboratives

242 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM vary widely. A randomized controlled trial used the collaborative method to assess physician adherence to guidelines for the treatment of stroke and cardiovascular disease but found only small levels of improvement (Ornstein et al., 2004). Other prospective and retrospective studies provided mixed results for a variety of conditions (Sheikh and Bullock, 2001; Holman et al., 2001; Halm et al., 2004). Another study assessed the effect of a controlled national collaborative on the outcomes of care for human immunodefi- ciency virus­infected patients. That controlled pre- and postintervention study concluded that the outcomes did not improve significantly (Landon et al., 2004). One study evaluated a QIO-run national demonstration project that used the Institute for Healthcare Improvement collaborative method and was part of the National Surgical Infection Prevention Project, a project that was integrated into the 7th SOW and that was designed to improve the safety of surgical care (Dellinger et al., 2005). From April 2002 to February 2003, researchers aggregated data on changes for three process measures and one outcome measure from 44 of 56 volunteering hospitals represent- ing 50 QIO jurisdictions and 35,543 surgical cases; some hospitals also elected to work and report on additional process measures. The study used a preintervention-postintervention design without control groups and yielded significant improvements in performance (P < 0.05) for the three process measures required by the study. The researchers calculated the dif- ferences in performance on measures between the beginning and the end of the study by conducting paired differences tests to adjust for confounding variables. Changes in the outcome measures were not significant from quar- ter to quarter, although comparison of the results at the end of the study period with those at the beginning yielded significant differences. In their summary, the investigators questioned the sustainable effects of quality im- provement on outcome measures. That study provides evidence for the positive effects of efforts toward improving patient safety in surgical care processes and suggests that QIOs can have an effect on quality improvement (Dellinger et al., 2005). Never- theless, a limitation to the study was that not all hospitals assessed the same surgical procedures because of different case mixes or other reasons; there- fore, direct hospital-to-hospital comparisons could not be made. The sustainability of these results and their ability to be spread to other provid- ers also remain unanswered by this study. Although the interventions of process measures were well described and these processes could have pro- duced the given outcomes, the lack of randomization remains a barrier to attribution because all participants were interested in improving. There- fore, this study is unable to discern whether these positive effects are di- rectly attributable to the QIO program, the collaborative methodology, a combination thereof, or other variables.

IMPACT OF TECHNICAL ASSISTANCE 243 Collaborative studies that have used the chronic care model have also recently been assessed. The chronic care model focuses on a system in which the patient is the manager of his or her health care. Patient preferences, evidence-based guidelines, and persistent follow-up are emphasized (IOM, 2003).These studies evaluated the collaborative methodology, with the fo- cus being to improve the quality of care for particular disease states, such as asthma and chronic heart failure, by using quasiexperimental study designs. The participant groups showed significant improvements in the process measures compared with the improvements for the control groups, but they did not show improvements in outcomes. These results are in concert with the findings presented in the quality improvement literature (Shortell et al., 2004; Cretin et al., 2004; Schonlau et al., 2005; Asch et al., 2005). The difficulty of sustaining the transfer of best practices to other providers--a key component to providing continuous improvement--has also been identified (Kosseff and Niemeier, 2001). Comparison of signifi- cant results from these studies is difficult, as it is not in the nature of col- laboratives to determine statistical significance (Daniel et al., 2004a). In addition, the effectiveness of the collaborative model has not been estab- lished, as not enough evidence is available to affirm or deny the effective- ness of collaboratives in general or in the QIO environment in particular (Leatherman, 2002; Mittman, 2004; Greenhalgh et al., 2004). In telephone interviews with QIO chief executive officers (CEOs), the CEOs did not mention collaboratives until they were directly asked about them. Although the Performance Improvement QIOSC trains QIOs on how to implement the Institute for Healthcare Improvement collaborative model, many states use other collaborative designs. In addition, although the QIOs are interested in learning about various approaches to improving quality, they often do not implement other methods for a variety of reasons, such as the cost of training, the intensity of work required for their success, and acceptance by the providers themselves. KNOWLEDGE TRANSFER The transfer of knowledge is difficult to achieve in any field, including health care. The terminology in the literature is not yet well defined and is inconsistent. Nuances exist as a result of the use of different terms, such as "dissemination," "sharing of best practices," and "knowledge transfer." In the context of this report, the term "knowledge transfer" refers to collective exchanges of ideas on how to best promote or provide high quality. Knowl- edge transfer is not limited to interactions between researchers and provid- ers or decision makers. In the QIO program, the participants in knowledge transfer include CMS, the QIOSCs, the QIOs, practitioners, administra- tors, and beneficiaries. By affecting the components of health care delivery,

244 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM such as the care processes, organizational structures, and systems in which care are delivered, the QIOs can work toward changing behaviors to pro- mote higher-quality health care and, ultimately, better health. Knowledge transfer is thus an important function of the QIO program and is especially relevant to the QIOSCs, because despite variations in health and health care delivery systems, common fundamental components of the provision of high-quality care may exist. The next section discusses knowledge transfer in the general health care environment through an assessment of the litera- ture, followed by a discussion of the multiple methods in which ideas are translated within the QIO program. Knowledge Transfer in the Literature As is the case for the health care­related quality improvement interven- tion literature, the evidence base for knowledge transfer in health care is limited (Heller and Arozullah, 2001). Although no single technique for the best way to transfer knowledge has been identified, many methods have been tried, with some appearing to be more successful than others. Most of the literature tends to be descriptive and observational and, thus, is based on inference and extrapolation (Berwick, 2003). Existing studies demon- strating knowledge transfer in health care are often cited as being of poor design and as containing methodological flaws (Greenhalgh et al., 2004; Mittman, 2004; Fleuren et al., 2004). Internal and external factors affect knowledge transfer. One important aspect is the development of new ideas internally within an organization. Rogers' diffusion of innovations theory (see Chapter 8) discusses five orga- nizational characteristics that impact how quickly organizational behaviors change: relative advantage, compatibility, complexity, triability, and ob- servability. In addition to these characteristics, organizational commitment and readiness for change are necessary elements for successful knowledge transfer. Support must come from the organization leadership (Greenhalgh et al., 2004; Wang et al., 2004; Mills and Weeks, 2004). The organizations contributing to the literature typically want to improve and thus already have some important organizational support in place. To operationalize the adoption of quality improvement, a clear plan of how to implement change is necessary. In health care, the value of interpersonal influence among pro- viders should not be underestimated, as the literature suggests that the pres- ence of clinical leaders and champions promotes knowledge transfer, despite a lack of conclusive evidence of their effectiveness (Berner et al., 2003; Shortell et al., 2004; Jamtvedt et al., 2004; Greenhalgh et al., 2004; Thomson O'Brien et al., 2005). Knowledge transfer may also be influenced by external factors, such as financial incentives and politics. Depending on the design of the payment

IMPACT OF TECHNICAL ASSISTANCE 245 structure, financial incentives such as pay for performance could inhibit or expedite knowledge transfer. If competition to improve arises, providers may not want to exchange ideas about best practices and lessons learned; however, if such exchanges are rewarded, knowledge transfer may be pro- moted. Political factors outside of an organization can also affect an organization's readiness for change (Greenhalgh et al., 2004). The literature discusses many barriers to successful knowledge transfer, some of which are mentioned below. Within an organization, issues such as competing priorities, resource allocation, and delayed acceptance by key stakeholders are among the numerous difficulties. Another barrier is the lack of willingness to share due to fear that competitors will fare better upon collaboration. Other issues, such as physician autonomy, make be- havioral changes in the health care industry particularly complicated. Be- cause physicians are considered experts because of their highly specialized knowledge of the complex field of medicine, it can be difficult for non- physicians to evaluate the work of physicians and be heeded. The presence of many small group practices adds to the insularity of health care provid- ers, making the widespread transfer of knowledge challenging. Transfers of knowledge between different health care settings are often of inconsistent quality and thus an additional barrier to knowledge transfer. The largely decentralized nature of health care research is another barrier. With the copious amounts of new ideas and technologies that are con- stantly being introduced, researchers need to develop a filter for use among providers that will allow providers to differentiate good and bad ideas. Guidelines for good ideas are difficult to develop, however, because of is- sues such as a provider's need to alter interventions to fit local needs, as well as the time required to elicit measurable changes in outcomes. Wide- spread knowledge transfer of health care practices is challenging and is an area requiring increased attention and research. Knowledge Transfer Within the QIO Program With the multitude of players in the health care delivery system, knowl- edge is gained on many fronts and can be transferred in many directions, as displayed in Figure 9.1. In the QIO program, sharing occurs mainly be- tween CMS and the QIOs through the QIOSCs and between the QIOs and the providers. However, ideas are also exchanged between QIOs, among providers, from providers to QIOs, from QIO to QIOSCs, and from QIOSCs to CMS. Beneficiaries also play an integral role in this process through beneficiary education (the transfer of knowledge from QIOs and providers to beneficiaries) and through assessments of beneficiaries' per- spectives of care (from beneficiaries to QIOs and providers). In combina-

246 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM tion, these multiple paths of knowledge transfer can help strengthen the communication and effectiveness of the QIO program. QIO Support Centers QIOSCs provide technical assistance to QIOs, just as the QIOs offer technical assistance to providers, to achieve improvements in care, as dis- cussed in Chapters 7 and 8. QIOSCs are critical to the sharing of informa- tion among QIOs. Unlike the core QIO contracts, which are based on a point scale (see Chapter 10), CMS assesses QIOSCs on how satisfactorily their deliverables are met on the basis of the judgment of the Government Task Leader in charge of each QIOSC (see Chapter 13). CMS's evaluations of the QIOSCs in the 7th SOW led to a redesign of the QIOSC system for the 8th SOW (see Chapter 8) (personal communication, J. Taylor, April 29, 2005). With recognition of the unique feedback that QIOs can provide to the QIOSC system as the customers of the QIOSCs, a survey that assessed QIO satisfaction with the QIOSCs was added to the 8th SOW as part of the redesign of the SOW. Evaluation of QIOSC effectiveness by CMS will there- fore be a function of satisfaction from both the Government Task Leaders and the QIOs. One approach used to transfer knowledge in the QIO program is the Process Improvement QIOSC, which was created in the 7th SOW and which has been renamed the Performance Improvement QIOSC in the 8th SOW. Qualis Health led this QIOSC in both the 7th and the 8th SOWs. The goal of this QIOSC is to ensure the efficiency and effectiveness of QIO processes by creating a "culture of shameless stealing" to exchange best practices (Qualis Health, 2005b). To achieve this goal, this QIOSC trains both QIOs and providers on running and facilitating collaboratives following the de- sign of the Institute for Healthcare Improvement Breakthrough Series Col- Beneficiaries QIO Provider CMS QIOSCs QIO Provider FIGURE 9.1 Knowledge transfer in the QIO program.

IMPACT OF TECHNICAL ASSISTANCE 247 laborative (see Chapter 8). Although the evidence base for collaborative initiatives remains inconclusive, as discussed earlier in this chapter, in the 7th SOW, Qualis Health assisted with the implementation of more than seven collaboratives, each of which focused on different aspects of technical assistance tasks. Many of the providers involved in collaboratives made improvements, such as increasing the rate of antibiotic use from 78 to 91 percent among patients with pneumonia in 14 critical access hospitals in an 8-month collaborative in Idaho (Qualis Health, 2005a). However, be- cause of the limitations of measuring and attributing improvement, the im- pact of the QIO intervention cannot be separated from those of other possible interventions or factors. Beyond instilling a culture of quality im- provement through collaboration, the leaders of these initiatives hope that the providers involved transfer the knowledge that they have gained to oth- ers in their communities. On the basis of the interviews with QIOSC CEOs and staff, the QIOSCs mentioned the following as barriers to their missions: imprecise evaluation methods, the rigidity of CMS oversight, a lack of contract flexibility, the timing of the QIOSC contract (which is aligned with the start of the SOW, and thus does not allow the QIOSCs time to develop materials before the beginning of the SOW), and the timing of the approval and distribution of the tools developed by the QIOSCs. In telephone interviews with 20 QIO CEOs, the QIOSCs received mixed reviews in terms of both expertise and timeliness. The following are some of the criticisms of the QIOSCs from the overall assessments of the QIOSCs by the QIO CEOs: · QIO task-related materials were not made available in a timely enough manner, at times forcing QIOs to produce their own. · The focus or the target of the materials was not always applicable to all states because of differences in state sizes, regional influences, and popu- lation demographics. · The innovativeness of the interventions and the materials did not necessarily lead to significant changes in areas where the QIOs needed support. · The flow of information was backwards, in that the QIOs offered more support to the QIOSCs than the other way around. QIO-to-QIO Knowledge Transfer The development and alteration of interventions at the state level but with maintenance of the core attributes of the interventions that can result in improvement are key to successful quality improvement within each state

248 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM because of differences in local environments. The QIOs can develop and alter interventions independently or partner with other public or private organizations. Interventions often develop in the QIO program as part of pilot tests or special studies. An example of a successful pilot program was the Cooperative Cardiovascular Project, which began as a four-state pilot project in 1995 that was eventually implemented on a national scale. An 11-state pilot project that was under way at the time of this writing and that was expected to conclude in July 2005 is testing the ability of providers to assess specific process and outcome measures to reduce preventable hospi- talizations in the home health setting (AHQA, 2005). If the pilot is success- ful, the 11 states in the pilot will be able to pass on the lessons learned and help other QIOs implement the intervention. In telephone interviews with the QIO CEOs, 8 of 20 CEOs commented that pilot testing is beneficial because it allows experience with the task at hand to be obtained before all QIOs are required to do the task. In the telephone interviews, the CEOs also identified non-QIOSC QIOs as sources of information and assistance. This is echoed by responses to a question in the web-based data collection tool that asked the respondents to name the top three QIOs that they would turn to for help with technical assistance tasks. Fifty-seven percent of all the responders (88 of 155) said that they would approach a non-QIOSC QIO for support. Best Practice Methods Special Study Innovations created at the state level may not, however, be applicable to all QIOs because of variations among the states. A special study entitled the Best Practice Methods Special Study, run by the Process Improvement QIOSC, was a two-part study of the 7th SOW for determination of how successful interventions and lessons learned can best be transferred among the QIOs. The first part of this special study distinguished high-performing QIOs from low-performing QIOs on the basis of statistically significant comparisons of statewide performance with national performance on the 12 hospital measures used in both the 6th and the 7th SOWs. Using this categorization of QIOs as a platform, the QIOSC administered surveys to the QIOs in the high-performing group (eight QIOs) and low-performing group (nine QIOs). With the recognition of limitations because of recall bias and the relatively small sample size, the study identified the organiza- tional characteristics of high-performing QIOs that resulted in the high- quality performance: staff empowerment, low staff turnover, flexibility, and staff in place at the beginning of the SOW. The QIOs with good reputations among providers scored higher. Greater CEO and board involvement were also associated with higher-performing QIOs. Standardized ("one-size-fits- all") interventions were associated with lower-performing QIOs; the higher-

IMPACT OF TECHNICAL ASSISTANCE 249 performing QIOs were able to customize interventions to meet their local needs. Differences in patient age, population education, and the frequency of provider interactions were not found to be associated with either high- or low-performing QIOs. The second part of the Best Practice Methods Special Study examined the portability of the best practices identified in the first part of the study. Fifty hospitals participated in the second part of the study, 10 hospitals from each of the five participating QIOs from the states of Arizona, Colo- rado, Maryland, South Carolina, and Washington. Upon the implementa- tion of similar processes of care for smoking cessation counseling and dis- charge planning, the special study attempted to determine whether certain QIO characteristics or interventions are transferable to other states, which would delineate the ability to transfer knowledge of best practices among QIOs in various environments (CMS, 2005a). The Medicare Quality Improvement Community (MedQIC) is a pub- licly available web-based resource that primarily serves as an interface among CMS, QIOs, and Medicare providers. Support is available in the forms of tools such as fact sheets, templates, slides, presentations, specific information on process measures and guidelines for their collection, litera- ture, and success stories for both clinical and consumer education. The QIOs and providers may also find contact information on the website for staff at the appropriate QIOSC. Providers can rank the tools available, as well as suggest new tools. A forum within MedQIC called QNet Quest is a data- base of answers to frequently asked questions. Through this database, pro- viders can directly ask the QIOs for support (see Chapter 13 for a further discussion of MedQIC). CMS-to-QIO Knowledge Transfer CMS uses a variety of methods to communicate with QIOs, including memos, e-mails, face-to-face meetings, and various information and com- munications technology tools (see Chapter 13). In the telephone interviews, the QIO CEOs also cited CMS working groups and meetings with CMS Regional Offices as methods of knowledge transfer. CMS uses QIONet, an intranet site provided under the auspices of CMS's Standard Data Process- ing System available only to QIOs and CMS groups, to share information and tools for the purpose of improving program management and achiev- ing program goals (see Chapter 13). QIO-to-Provider Knowledge Transfer The transfer of knowledge from QIOs to providers stemmed from the widespread sharing of materials beyond the identified groups of partici-

250 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM pants through the inclusion of a broader group of providers in meetings and through the inclusion of stakeholder groups that could then communicate with their memberships. Participants are provided with materials so that they can try to make changes on their own without receiving additional technical assistance from the QIO. In the telephone interviews with the QIO CEOs, the CEOs stressed the importance of champions and the use of stakeholder organizations. Additionally, the CEOs often mentioned how the electronic age aids with the transfer of knowledge within their states, particularly when providers are dispersed geographically or have limited time to spend away from their offices. QIOs use on-site meetings and tech- nology like webex and other video teleconferencing methods. Some QIOs develop and send out compact discs containing quality improvement­ related presentations to those providers who do not attend their meetings. Many QIOs hold statewide quality forums for the presentation of best prac- tices for providers within their states and give out awards for achievement. Other QIOs publish articles in their state medical journals as well as in national medical journals. In telephone interviews, one QIO CEO men- tioned the possibility of tying some quality improvement efforts to continu- ing medical education credits for physicians as a way to increase their participation. Knowledge Transfer with Beneficiaries Beneficiaries are an integral part of knowledge transfer. Input on how care is received is important in CMS's and QIOs' evaluations of the QIO program and the performance of individual QIOs. Beneficiaries also relate their perceptions of care to providers, who can then provide feedback to the QIOs. Knowledge is indirectly transferred from beneficiaries to CMS through the QIOs. Examples of the means of knowledge transfer from ben- eficiaries to QIOs include consumer advisory councils, representation of consumers on QIO boards, and beneficiary surveys on their satisfaction with the mediation of beneficiary complaints (see Chapters 11 and 12). It is also important for beneficiaries to receive information from CMS, QIOs, and providers on the quality of care being delivered and what benefi- ciaries themselves can do to promote better health. CMS publicly reports data on the quality of care for individual nursing homes, home health agen- cies, and hospitals through its Compare websites (see Chapter 11). For ex- ample, CMS requires QIOs to maintain help lines to assist beneficiaries. QIOs sometimes also supply providers with fact sheets to distribute to patients. How well knowledge has been transferred between beneficiaries and CMS, QIOs, and providers is not well documented. In the 7th SOW, benefi-

IMPACT OF TECHNICAL ASSISTANCE 251 ciary satisfaction surveys covered only mediation activities; in the 8th SOW, however, beneficiaries will be surveyed on all the care that they receive. The telephone interviews disclosed that many QIO CEOs found benefi- ciary education to be valuable for quality improvement. For example, these activities may help QIOs build rapport with providers. It is also unknown whether consumers effectively use the data presented on the CMS Compare websites or call in to the QIO help lines (see Chapter 11). The actual im- pacts that these efforts have on beneficiaries and the quality of care that they receive remain unclear. Other Knowledge Transfer Mechanisms Other mechanisms of spreading knowledge among the QIOs have de- veloped, such as those carried out through the American Health Quality Association (AHQA), which is the trade organization for the QIOs. Infor- mal, unofficial groups of CEOs often come together on the basis of their geographic region or state size, such as the Coral Initiative, which started among five QIOs in the Midwest. Through AHQA and these other coali- tions, QIOs meet periodically; learn about the latest program changes; and share successes and failures via telephone conference calls, listserves, e-mail, newsletters, etc. These groups foster a culture of sharing among the QIOs. As described by the QIO CEOs in the telephone interviews, "The problem is not the sharing but sifting out the scientifically based from the noise and self-promotion," and "Already we have free flow of information. Perhaps we have too much free flow because the focus is not on rigor in documenta- tion. Lots of assertions are made about what works." The CEOs praised the specialized groupings of QIOs for their effectiveness in sharing and solving problems as well as the sharing that they perform through AHQA commit- tees and conferences. Many QIOs have been innovators of quality improvement, as interven- tions are often adapted to best fit the needs of each state or jurisdiction. However, if the development processes and successes of these variations are not shared with other QIOs, opportunities for improvement may be lost. Although some QIOs share their stories of innovation with others, an "owner" of this function who can make the information widely available is lacking. Although the QIOs are encouraged to publish articles regarding their work, QIO evaluation formulas did not emphasize those efforts in the 7th SOW. In the 8th SOW, CMS added acceptance for publication in peer- reviewed journals as an extra-credit point to the Hospital Payment Moni- toring Program but not for other aspects of the SOW. Therefore, as QIOs are under performance-based contracts with limited time and resources, CMS provides little incentive to contribute to the literature.

252 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM SUMMARY This chapter has discussed issues related to the impact of technical as- sistance for quality improvement and knowledge transfer both in the health care environment in general and in the QIO program in particular. The following are some of the main themes of this chapter, which are reflected in the findings and conclusions presented in Chapter 2: · Quality improvement is difficult to achieve. · The evidence for the impact of quality improvement interventions is mixed. Conclusions drawn from the literature base for quality improve- ment show that the quality improvements resulting from interventions in health care in general and the QIO program in the particular are similar. · Quality improvement interventions were able to consistently pro- duce significant improvements in process measures; other interventions, such as those focusing on outcomes and structure, were not found to yield improvements. There are, however, many limitations to the methods by which quality improvement interventions are documented and evaluated. · Although the quality of care received by Medicare beneficiaries has improved somewhat, researchers have been unable to attribute these changes to the QIO program. This can be the result of various limitations, such as how QIO interventions are currently evaluated or the fact that QIO inter- ventions do not improve quality. · A variety of approaches to quality improvement are being tried in many industries, and QIOs are learning from these approaches. · Collaboratives were often used to incite quality improvement in the 7th SOW. However, evaluations of collaboratives in the literature--both in the general health care environment and in the QIO program--have pro- vided inconclusive results on their impact on improving quality. · On the basis of the information in the literature, it cannot be determined what drives knowledge transfer in the general health care environment. · Knowledge transfer in the QIO program is not well documented, making it difficult for the committee to find evidence for how effectively the QIOs achieve it. There are many paths for the transfer of knowledge in the QIO program; these could all be leveraged to achieve increased quality. REFERENCES AHQA (American Health Quality Association). 2005. Home health pilot project to provide model for SOW8 work. AHQA Matters 6(8):12­13. American Society for Quality. 2005. American Society for Quality. [Online]. Available: www.asq.org [accessed September 7, 2005].

IMPACT OF TECHNICAL ASSISTANCE 253 Asch SM, Baker DW, Keesey JW, Broder M, Schonlau MP, et al. 2005. Does the collaborative model improve care for chronic heart failure? Medical Care 43(7):667­675. Barr JK, Franks AL, Lee NC, Antonucci DM, Rifkind S, Schachter M. 2001. A randomized intervention to improve ongoing participation in mammography. American Journal of Managed Care 7(9):887­894. (Erratum, 7(12):1116.) Berner ES, Baker SC, Funkhouser E, Heudebert GR, Allison JJ, et al. 2003. Do local opinion leaders augment hospital quality improvement efforts?: A randomized trial to promote adherence to unstable angina guidelines. Medical Care 41(3):420­431. Berwick DM. 2003. Disseminating innovations in health care. Journal of the American Medi- cal Association 289(15):1969­1975. Bradley EH, Holmboe ES, Mattera JA, Roumanis SA, Radford MJ, Krumholz HM. 2001. A qualitative study of increasing beta-blocker use after myocardial infarction: Why do some hospitals succeed? Journal of the American Medical Association 285(20):2604­2611. Bradley EH, Carlson MDA, Gallo WT, Scinto J, Campbell MK, Krumholz HM. 2005. From adversary to partner: Have quality improvement organizations made the transition? Health Services Research 40(2):459­476. Burwen DR, Galusha DH, Lewis JM, Bedinger MR, Radford MJ, et al. 2003. National and state trends in quality of care for acute myocardial infarction between 1994­1995 and 1998­1999: The Medicare Health Care Quality Improvement Program. Archives of In- ternal Medicine 163(12):1430­1439. Centor RM, Allison JJ, Weissman NW, Canto J, Heudebert G, et al. 2003. Diffusion of tropo- nin testing in unstable angina patients: Adoption prior to guideline release. Journal of Clinical Epidemiology 56(12):1236­1243. Chu LA, Bratzler DW, Lewis RJ, Murray C, Moore L, et al. 2003. Improving the quality of care for patients with pneumonia in very small hospitals. Archives of Internal Medicine 163(3):326­332. CMS (Centers for Medicare and Medicaid Services). 2005a. Best Practice Methods Special Study: Report of First Year Scan of QIO Inpatient Practice. Unpublished. Baltimore, MD: Centers for Medicare and Medicaid Services. CMS. 2005b. MedQIC Glossary. [Online]. Available: http://medqic.org/content/Common Items/Article/glossary.jsp?g11n.enc=ISO-8859-1&topicID=&search=yes [accessed Janu- ary 5, 2005]. CMS. 2005c. 8th Statement of Work (SOW). [Online]. Available: http://www.cms.hhs.gov/ qio [accessed April 9, 2005]. Coleman EA, Smith JD, Frank JC, Parry CP, Kramer AM. 2004. Preparing patients and caregivers to participate in care delivered across settings: The care transitions interven- tion. Journal of the American Geriatrics Society 52(11):1817­1825. Cortes LL. 2004. The Impact of Quality Improvement Programs in Long Term Care: Are State and Federal Quality Improvement Initiatives for Nursing Homes Redundant? Aus- tin: Texas Department of Human Services Division of Long Term Care. Cretin S, Shortell SM, Keeler EB. 2004. An evaluation of collaborative interventions to im- prove chronic illness care framework and study design. Evaluation Review 28(1):28­51. Daniel DM, Norman J, Davis C, Lee H, Hindmarsh MF, et al. 2004a. Case studies from two collaboratives on diabetes in Washington state. Joint Commission Journal on Quality and Safety 30(2):103­108. Daniel DM, Norman J, Davis C, Lee H, Hindmarsh MF, et al. 2004b. A state-level application of the chronic illness breakthrough series: Results from two collaboratives on diabetes in Washington state. Joint Commission Journal on Quality and Safety 30(2):69­79. Dellinger EP, Hausmann SM, Bratzler DW, Johnson RM, Daniel DM, et al. 2005. Hospitals collaborate to decrease surgical site infections. American Journal of Surgery 190(1):9­15.

254 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Donabedian A. 1988. The quality of care. How can it be assessed? Journal of the American Medical Association 260(12):1743­1748. Ellerbeck EF, Jencks SF, Radford MJ, Kresowik TF, Craig AS, et al. 1995. Quality of care for medicare patients with acute myocardial infarction: A four-state pilot study from the cooperative cardiovascular. Journal of the American Medical Association 273(19):1509­ 1514. Ellerbeck EF, Kresowik TF, Hemann RA, Mason P, Wilblin RT, Marciniak TA. 2000. Impact of quality improvement activities on care for acute myocardial infarction. International Journal for Quality in Health Care 12(4):305­310. Fleuren M, Wiefferink K, Paulussen T. 2004. Determinants of innovation within health care organizations. Literature Review and Delphi Study. International Journal for Quality in Health Care 16(2):107­123. Gosbee J. 2002. Human factors engineering and patient safety. Quality and Safety in Health Care 11(4):352­354. Gould BE, Grey MR, Huntington CG, Gruman C, Rosen JH, et al. 2002. Improving patient care outcomes by teaching quality improvement to medical students in community-based practices. Academic Medicine 77(10):1011­1018. Greenhalgh T, Robert G, Macfarlane F, Bate P, Kyriakidou O. 2004. Diffusion of innovations in service organizations: Systematic review and recommendations. Milbank Quarterly 82(4):581­629. Halm EA, Horowitz C, Silver A, Fein A, Dlugacz YD, et al. 2004. Limited impact of a multicenter intervention to improve the quality and efficiency of pneumonia care. Chest 126(1):100­108. Heller C, Arozullah A. 2001. Implementing change: It's as hard as it looks. Disease Manage- ment and Health Outcomes 9(10):551­563. Holman WL, Allman RM, Sansom M, Kiefe CI, Peterson ED, et al., for the Alabama CABG Study Group. 2001. Alabama coronary artery bypass grafting project: Results of a state- wide quality improvement initiative. Journal of the American Medical Association 285(23):3003­3010. IOM (Institute of Medicine). 2003. Priority Areas for National Action: Transforming Health Care Quality. Washington, DC: The National Academies Press. Jamtvedt G, Young J, Kristoffersen D, Thomson O'Brien M, Oxman A. 2004. Audit and feedback: Effects on professional practice and health care outcomes. Cochrane Database of Systematic Reviews (4). Jencks SF, Wilensky GR. 1992. The health care quality improvement initiative. A new ap- proach to quality assurance in Medicare. Journal of the American Medical Association 268(7):900­903. Jencks SF, Cuerdon T, Burwen DR, Fleming B, Houck PM, et al. 2000. Quality of medical care delivered to Medicare beneficiaries: A profile at state and national levels. Journal of the American Medical Association 284(13):1670­1676. Jencks SF, Huff ED, Cuerdon T. 2003. Change in the quality of care delivered to Medicare beneficiaries, 1998­1999 to 2000­2001. Journal of the American Medical Association 289(3):305­312. Joseph AM, Arikian MJ, An LC, Nugent SM, Sloan RJ, Pieper CF. 2004. Results of a random- ized controlled trial of intervention to implement smoking guidelines in veterans affairs medical centers: Increased use of medications without cessation benefit. Medical Care 42(11):1100­1110. Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. 2001. Improving quality improvement using achievable benchmarks for physician feedback: A randomized controlled trial. Journal of the American Medical Association 285(22):2871­2879.

IMPACT OF TECHNICAL ASSISTANCE 255 Kosseff AL, Niemeier S. 2001. SSM health care clinical collaboratives: Improving the value of patient care in a health care system. Joint Commission Journal on Quality Improvement 27(1):5­19. Landon BE, Wilson IB, McInnes K, Landrum MB, Hirschhorn L, et al. 2004. Effects of a quality improvement collaborative on the outcome of care of patients with HIV infection: The EQHIV study. Annals of Internal Medicine 140(11):887­896. Leatherman S. 2002. Optimizing quality collaboratives. Quality and Safety in Health Care 11(4):307. Luthi JC, McClellan WM, Fitzgerald D, Krumholz HM, Delaney RJ, et al. 2002. Mortality associated with the quality of care of patients hospitalized with congestive heart failure. International Journal for Quality in Health Care 14(1):15­24. Marciniak TA, Ellerbeck EF, Radford MJ, Kresowik TF, Gold JA, et al. 1998. Improving the quality of care for medicare patients with acute myocardial infarction: Results from the cooperative cardiovascular project. Journal of the American Medical Association 279(17): 1351­1357. Mark BA, Harless DW, McCue M, Xu Y. 2004. A longitudinal examination of hospital regis- tered nurse staffing and quality of care. Health Services Research 39(2):279­300. Massing MW, Henley N, Biggs D, Schenck A, Simpson RJ Jr. 2003. Prevalence and care of diabetes mellitus in the Medicare population of North Carolina. Baseline findings from the Medicare Healthcare Quality Improvement Program. North Carolina Medical Jour- nal 64(2):51­57. McClellan WM, Millman L, Presley R, Couzins J, Flanders WD. 2003. Improved diabetes care by primary care physicians: Results of a group-randomized evaluation of the Medicare Health Care Quality Improvement Program (HCQIP). Journal of Clinical Epidemiology 56(12):1210­1217. Mills PD, Weeks WB. 2004. Characteristics of successful quality improvement teams: Lessons from five collaborative projects in the VHA. Joint Commission Journal on Quality and Safety 30(3):152­162. Mittman BS. 2004. Creating the evidence base for quality improvement collaboratives. Annals of Internal Medicine 140(11):897­901. NAE and IOM (National Academy of Engineering and Institute of Medicine). 2005. Building a Better Delivery System: A New Engineering/Health Care Partnership. Washington, DC: The National Academies Press. NIST (National Institute of Standards and Technology). 2005, July 13. Malcolm Baldridge National Quality Award. [Online]. Available: http://www.nist.gov/public_affairs/ factsheet/mbnqa.htm [accessed December 30, 2005]. Ornstein S, Jenkins RG, Nietert PJ, Feifer C, Roylance LF, Nemeth L, Corley S, Dickerson L, Bradford WD, Litvin C. 2004. A multimethod quality improvement intervention to im- prove preventive cardiovascular care: A cluster randomized trial. Annals of Internal Medi- cine 141(7):523­532. Ovretveit J, Gustafson D. 2003. Using research to inform quality programmes. British Medical Journal 326(7392):759­761. Ovretveit J, Bate P, Cleary P, Cretin S, Gustafson D, McInnes K, McLeod H, Molfenter T, Plsek P, Robert G, Shortell S, Wilson T. 2002. Quality collaboratives: Lessons from research. Quality and Safety in Health Care 11(4):345­351. Qualis Health. 2004. HCQIP Improvement Methodologies Survey Results. Seattle, WA: Qualis Health. Qualis Health. 2005a. Case Study: Collaborative Experience Project, Surgical Infection Pre- vention. [Online]. Available: http://www.qualishealth.org/qi/collaboratives/case-study- sipc.cfm [accessed September 13, 2005].

256 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Qualis Health. 2005b. Performance Improvement QIO Support Center (PI QIOSC). [Online]. Available: http://www.qualishealth.org/piqiosc/index.cfm [accessed November 15, 2005]. Qualis Health. 2005c. Improvement Methodologies. Seattle, WA: Qualis Health. Schade C, Cochran B, Stephens M. 2004. Using statewide audit and feedback to improve hospital care in West Virginia. Joint Commission Journal on Quality and Safety 30(3): 143­151. Schonlau M, Mangione-Smith R, Chan KS, Keesey J, Rosen M, et al. 2005. Evaluation of a quality improvement collaborative in asthma care: Does it improve processes and out- comes of care? Annals of Family Medicine 3(3):200­208. Sheikh K, Bullock C. 2001. Urban-rural differences in the quality of care for Medicare patients with acute myocardial infarction. Archives of Internal Medicine 161(5):737­743. Shortell SM, Bennett CL, Byck GR. 1998. Assessing the impact of continuous quality improve- ment on clinical practice: What it will take to accelerate progress. Milbank Quarterly 76(4):510, 593­624. Shortell SM, Marsteller JA, Lin M, Pearson ML, Wu SY, et al. 2004. The role of perceived team effectiveness in improving chronic illness care. Medical Care 42(11):1040­1048. Silver MP, Geis MS, Bateman KA. 2004. Improving health care systems performance: A hu- man factors approach. American Journal of Medical Quality 19(3):93­102. Snyder C, Anderson G. 2005. Do quality improvement organizations improve the quality of hospital care for Medicare beneficiaries? Journal of the American Medical Association 293(23):2900­2907. Sutherland JE, Hoehns JD, O'Donnell B, Wiblin RT. 2001. Diabetes management quality improvement in a family practice residency program. Journal of the American Board of Family Practice 14(4):243­251. Thomson O'Brien M, Oxman A, Haynes R, Davis D, Freemantle N, Harvey E. 2005. Local opinion leaders. Cochrane Database of Systematic Reviews (2) Toussaint J. 2005, May 5. Presentation at Pay for Performance Subcommittee Meeting. Un- published. Washington, DC. Wang A, Wolf M, Carlyle R, Wilkerson J, Porterfield D, Reaves J. 2004. The North Carolina experience with the diabetes health disparities collaboratives. Joint Commission Journal on Quality and Safety 30(7):396­404.

Next: 10 Evaluation of Quality Improvement Achieved by the QIO Program »
Medicare's Quality Improvement Organization Program: Maximizing Potential Get This Book
×
Buy Hardback | $81.00 Buy Ebook | $64.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Medicare's Quality Improvement Organization Program is the second book in the new Pathways to Quality Health Care series. Focusing on performance improvement, it considers the history, role, and effectiveness of the Quality Improvement Organization (QIO) program and its potential to promote quality improvement within a changing health care delivery environment that includes standardized performance measures and new data collection and reporting requirements. This book carefully examines the QIOs that serve every state as well as the national program that guides and supports them. In addition, it highlights the important roles that a national program with private organizations in each state can play in promoting higher quality care. Medicare's Quality Improvement Organization Program looks closely at the technical assistance role of the QIO program and the need to encourage and support providers to improve their performance. By providing an in-depth assessment of the federal experience with quality improvement and recommendations for program improvement, this book helps point the way for those who strive to create higher quality and better value in health care. Intended for multiple audiences, Medicare's Quality Improvement Organization Program is essential reading for members of Congress, the federal executive branch, the QIOs, health care providers and clinicians, and stakeholder groups.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!