National Academies Press: OpenBook

Medicare's Quality Improvement Organization Program: Maximizing Potential (2006)

Chapter: 2 Assessment of the QIO Program: Findings and Conclusions

« Previous: Part I--Introduction: 1 A Historical Perspective and the Current QIO Program
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 55
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 56
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 57
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 58
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 59
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 60
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 61
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 62
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 63
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 64
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 65
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 66
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 67
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 68
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 69
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 70
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 71
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 72
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 73
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 74
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 75
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 76
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 77
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 78
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 79
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 80
Suggested Citation:"2 Assessment of the QIO Program: Findings and Conclusions." Institute of Medicine. 2006. Medicare's Quality Improvement Organization Program: Maximizing Potential. Washington, DC: The National Academies Press. doi: 10.17226/11604.
×
Page 81

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

2 Assessment of the QIO Program: Findings and Conclusions CHAPTER SUMMARY This chapter describes the study's evaluation approach, summa- rizes the key findings discussed in detail in Part II, and presents the committee's conclusions about the impact of the Quality Im- provement Organization program. The committee concludes that although the evidence regarding the program's effects on quality of care is limited and inconclusive, the program has the potential to be a valuable asset as providers become more involved with public reporting and incentive payments from Medicare and other insurers. The historical overview of the Quality Improvement Organization (QIO) program in Chapter 1 describes a public program that has changed direction, functions, and methods frequently over its 35-year history. Given the absence of earlier evaluations of the program, uncertainties exist with regard to its potential roles in the future. As a base for the recommenda- tions for the future of the program presented in Chapters 4 and 5, this chapter details the committee's findings and conclusions concerning the program's impacts. Specifically, the chapter addresses whether: · There has been improvement in the quality of health care services provided to Medicare beneficiaries. · The QIO program has contributed to that improvement. · Certain components of the QIO program should be eliminated or strengthened. · There should be a continuing federal role in technical assistance for quality improvement and, if so, whether that role should be stronger. It becomes clear from the descriptive details presented in Part II that the 55

56 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM QIO program is richly varied. The organizations holding the QIO contracts for each state often differ dramatically in structure, in the services they provide, and in their quality intervention programs, perhaps reflecting dif- ferences in the health care services in their communities. This variation makes it difficult to generalize about all QIOs and the program at the na- tional level, as well as to determine what changes in structure might be of most benefit to the program and yield the greatest improvements in the quality of care. The historical overview presented in the previous chapter shows how the program has changed over time. Nonetheless, many of the organiza- tions holding state contracts and some of their key leadership have persisted with the program through much of its evolution and have demonstrated remarkable flexibility in adapting to each new statement of work. Through that evolution, the QIOs have built partnerships with the providers and other key stakeholders in their states, such as state departments of health and local chapters of the American Association for Retired Persons. The committee gathered anecdotal evidence of many favorable and some less favorable relationships between QIOs and providers, brought to the com- mittee's attention at national conferences, site visits, and informal discus- sions with providers and consumers. In addition, results of large national surveys of physicians, hospitals, nursing homes, and home health agencies reveal generally favorable attitudes toward their local QIO. (The provider surveys are discussed in detail in Chapter 10.) The committee finds these relationships and the QIOs' capacity for adaptability to be valuable assets. The remainder of this chapter begins with a discussion of the first two questions raised above: (1) whether care for Medicare beneficiaries has im- proved, and (2) whether the improvements, if any, can be attributed to the QIO program. Alternatives to the current QIO program, along with their advantages and disadvantages, are considered next. The chapter then ad- dresses program infrastructure at both the national and state levels. Next is a discussion of the main functions of the QIOs, followed by a review of structural issues, such as funding and board composition. Finally, the chap- ter addresses the oversight responsibilities of the Centers for Medicare and Medicaid Services (CMS). Detailed data and analyses supporting the com- mittee's findings and conclusions are presented mainly in Part II and the appendixes, relevant portions of which are referenced throughout the dis- cussion here. EVIDENCE OF QUALITY IMPROVEMENT IN MEDICARE The Medicare program has carefully tracked the growth in the numbers of beneficiaries, the expenditures made on their behalf, and the increasing variety and use of covered services over the history of the program; how-

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 57 ever, it has produced much less information on the quality of those services. The former director of the QIO program in CMS examined Medicare ser- vices by comparing data for 22 clinical inpatient and outpatient measures against state baseline performance reported through the QIO program from 1998­1999 to 2000­2001. It was found that the weighted national aver- age, as well as the state with the median performance, had improved on 20 of the 22 indicators (Jencks et al., 2003). A recent national study employing 9 of the same hospital quality indicators used by Jencks and colleagues (2003), along with publicly reported data on Medicare and all other pa- tients, found that the level of hospital performance had improved on all but 1 measure (Jha et al., 2005). Although these studies were less than ideal--in particular, the number of measures used in each study was limited, and the measures focused primarily on care provided within hospitals--their results indicate that health care for Medicare beneficiaries has improved. There are signs of improvement as well in clinical activities targeted as national pri- orities on which QIOs and other organizations have focused, such as mammograms and care for heart attacks and diabetes (Leatherman and McCarthy, 2005). Improvements in care for beneficiaries in managed care were also noted from 2000 to 2004 (NCQA, 2005). On the other hand, much evidence, cited in Chapter 1 of this report and in Performance Mea- surement: Accelerating Improvement (IOM, 2006), indicates that the qual- ity of care varies greatly from provider to provider by geographic location, race or ethnicity, and income of the beneficiary and that many people do not receive all the services they need, particularly appropriate preventive care. There is substantial room for further improvement. Can the gradual improvements in care that have been accomplished be attributed to the QIO program? Because of the nature of their evaluation designs, the studies mentioned above cannot be used to determine the cause of the improvements documented or attribute them to the QIO program. During site visits and focus group discussions conducted for this study, the committee heard frequent anecdotal evidence from providers and the QIOs regarding positive impacts on quality resulting from the multiple forms of technical assistance provided by the QIOs (see Chapter 8). Some studies of specific, limited QIO interventions or collaboratives have also documented improvements over time, but likewise cannot be used to conclude that the improvements were due to the QIOs' efforts (Marciniak et al., 1998; Kiefe et al., 2001; Gould et al., 2002; Chu et al., 2003; Daniel et al., 2004; Dellinger et al., 2005). In addition, preliminary data on several measures of the quality of care under the 7th scope of work (SOW) suggest that the QIOs may have had a positive impact on the care received from nursing home, home health agency, and hospital providers that participated inten- sively with their QIOs ("identified participants") in comparison with that received from all such providers statewide. The data also suggest that those

58 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM identified participants achieved greater improvements on measures related to the clinical area of their particular intervention relative to improvements achieved by participants who collaborated with QIOs on interventions un- related to those measures (personal communication, W. C. Rollow, July 8, 2005; personal communication, J. Kelly, September 8, 2005). The analyses were incomplete as of this writing, however, and it is unclear whether these apparent differences will be significant. Given the lack of consistent and conclusive evidence in scientific jour- nals and the lack of strong findings from the committee's analyses, it is not possible to determine definitively the extent of the impact of the QIOs and the national QIO infrastructure on the quality of health care received by beneficiaries. Many confounding factors make it difficult to attribute the results obtained thus far. Also, the literature does not address the QIO program as a whole; rather, it merely addresses the impacts of specific qual- ity improvement activities of individual QIOs or quality measures aggre- gated at the state or national level. Other aspects of the program, such as the impact of QIO case reviews on quality and the value of QIO Support Centers (QIOSCs), have received little or no scrutiny from evaluators. (See Chapter 8 for a discussion of QIOSCs.) One challenge to evaluating the QIO program is that QIOs recruit voluntary participants and generally partner with multiple stakeholder or- ganizations to conduct quality improvement interventions (see Chapter 8). It would be difficult to identify true control groups; random assignment has not been tried because of practical and political implications; studies often rely only on changes that have been observed compared with perfor- mance at the baseline; it is difficult to measure impacts within the time frame of the study; and it is nearly impossible to distinguish the impacts of a QIO from those of its partners and other environmental factors. Also, the nature of the interventions varies depending on the provider type and the QIO, as well as from one provider to another, because the intervention methods used are determined largely by each QIO. The voluntary nature of provider participation with QIOs introduces the possibility of bias in the self-selection of participants, which in turn limits the value of compari- sons of the rates of change for identified participant groups with the rates for the entire state. Also, because the QIO has the responsibility to raise the level of quality statewide as well as for identified participants, a large difference between the two rates of quality improvement may indicate a particularly successful program for identified participants but a very weak statewide program. These research limitations and inconclusive findings in the literature are not limited to studies of the QIO program, but also ham- per other studies of improvements in health care quality. In addition to studies related to the QIOs, the literature review per- formed for this study focused on more generic quality improvement studies

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 59 in health care. In addition to 23 studies focused specifically on the QIO program and various QIO-led quality improvement interventions, the com- mittee examined 9 articles, including 2 Cochrane reviews (the latter system- atically assessed a wide range of studies according to explicit, high stan- dards). (This literature review is discussed further in Chapter 9, and selected articles are summarized in Tables A.1 and A.2 in Appendix A.) The studies paint an inconclusive picture of the effectiveness of quality improvement programs, whether the interventions are conducted by QIOs or other orga- nizations, for both Medicare and non-Medicare services. In part, this lack of conclusive data results from the research challenges mentioned above. Few of the studies used randomized controlled trials or control groups. An examination of additional literature on the transfer of knowledge about various quality improvement methods also provided inconclusive findings on the effectiveness of these methods; no specific method has been identified as best. Studies do show the importance of commitment on the part of an organization's leadership, staff empowerment, and the develop- ment of a plan to promote quality improvement (Bradley et al., 2005; CMS, 2005a). Analysis of the management literature on effecting change to im- prove patient safety in health care organizations reveals that several human resource management practices are helpful in that process. In particular, to promote successful change and the adoption of new practices, it is impor- tant for management to promote "ongoing communication; training; use of mechanisms for measurement, feedback, and redesign; sustained attention; and worker involvement" (IOM, 2004:118). Knowledge transfer is considered an important part of the QIO pro- gram, although evidence that it occurs is meager. A significant portion of the functions of the QIOSCs relates to knowledge transfer among the QIOs: training the trainer, the establishment and maintenance of communities of practice, and the identification and promotion of best practices. During the site visits, telephone interviews, and focus group discussions conducted by the committee, the chief executive officers (CEOs) of the QIOs emphasized the need to include early adopters, leaders, and champions, in addition to middle and late adopters, in quality interventions to promote their uptake by practitioners and institutions beyond those involved in the interventions. On the other hand, focusing solely on providers that are lead- ers and champions and may be relatively easy to recruit to a QIO technical assistance program could limit the participation of providers that may need the QIO's assistance the most. The QIO program considers sharing among QIOs to be important and has established mechanisms for this purpose. For example, the shift in the 7th SOW from a relative failure rate definition for QIOs that must re- compete to an absolute target for automatic renewal meant that QIOs were more willing to share and help each other. They were no longer competing

60 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM to avoid the penalty associated with being in the bottom quartile; they were all trying to meet absolute targets and could cooperate without jeopardy. The 8th SOW maintains similar incentives. While the QIO contracts and QIO collaboratives are based on sharing and knowledge transfer, other forces in health care are pushing providers in the opposite direction, toward more competition. Under the pressures of public reporting, no providers will want to see their name on the bottom half of the quality-of-care list, which would indicate that the care they pro- vide is of below-average quality. Pay-for-performance schemes are likely to reinforce that competition, depending on the reward structure. Unintended consequences of public reporting and pay for performance should be exam- ined, but no evidence yet exists on what impacts, if any, these programs will have on competition, and it is unclear how QIO collaboratives will work as the environment becomes more competitive for providers. In some commu- nities, providers may welcome the use of public reporting and use it to benchmark and identify other providers from which they could learn. In any case, the program should be considering new options for QIO interven- tions and mechanisms, such as web-based training and self-guided study tools, to extend the QIOs' reach to more providers. Because of the limitations of the scientific literature, the committee at- tempted to use program-generated data, including the performance scores used by CMS to evaluate each QIO's contract performance, to identify high- and low-performing QIOs (see Chapter 10). CMS conducts separate per- formance assessments for each SOW task and subtask. The scores related to Task 1 (technical assistance for quality improvement) define "quality" for the program. The scores used by the committee are based on specified clini- cal measures for hospitals, nursing homes, home health care agencies, and physician offices during the 7th SOW through December 2004. To retain its QIO contract during the next SOW without competing, a QIO must attain a passing score for each task; no total or average score based on all tasks is assigned to each QIO. The committee's analyses revealed the following: · A QIO's score in one provider setting did not correlate with that in any other setting. · No correlation existed between spending per beneficiary in a state on the QIO's technical assistance in a particular setting and the QIO's per- formance measure for that setting. · No correlation existed between the state performance score in a par- ticular setting and measures of the provider's satisfaction with the QIO. · Some regional variations in the performance scores were noted, as were variations in contract rounds for the home health care provider scores (see Chapter 13 for a discussion of contract rounds).

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 61 The committee was unable to ascertain why one QIO might have a higher performance score than another in a particular setting. One might expect that many skills needed to carry out a quality improvement interven- tion would be transferable from one provider setting to another. They may in fact be transferable, but perhaps some QIO staffs are not well organized to effect that transfer of knowledge, or the provider communities and their stakeholder organizations may differ in ways that affect the success of the QIOs' interventions. Also, the role played by providers' voluntary partici- pation in the impacts of the QIO program is unclear. One study of hospital and ambulatory care measures showed that absolute improvement on 21 of the 22 measures tracked was greater for those states starting at a lower- quality baseline than for those starting at a higher-quality baseline (Jencks et al., 2003). While many inferences can be drawn about why a trend can- not be identified to link improvement and setting of care, it must be noted that some skills and expertise required to provide assistance in each setting are not necessarily the same. Given limited resources and the need to com- plete other tasks, such as communications and beneficiary protection, QIOs must choose how best to allocate their resources. The committee heard consistent reports of extensive variations among the QIOs: some are outstanding, while others are mediocre. These judg- ments appear to reflect an overall perspective on the organizations rather than the QIOs' performance on specific tasks. Yet there are no objective global measures, based on contract performance scores for quality improve- ment, that could indicate which QIOs belong in which category or even whether there are in fact significant differences in overall performance. For example, some QIOs score high on improving care in one setting but not in others. The complex and detailed formulas used by CMS to assess each state's performance on the core tasks of the SOW can be used to determine the QIOs' performance only on each task separately. For example, 6 of the 36 QIOs in the first two contract rounds failed on at least one assessment standard (task or subtask) and had to recompete for an 8th SOW contract. Those 6 QIOs included some that would not generally be viewed as belong- ing in the bottom tier on overall performance, some that had received sub- stantial contracts from CMS to conduct special studies or to serve as a QIOSC, and some that were named "best" at particular tasks by other CEOs according to the committee's web-based data collection tool. ALTERNATIVES TO THE CURRENT QIO PROGRAM In considering alternatives to the current QIO program, the committee decided it was necessary to step back and ask some fundamental questions. First, should the federal government exercise a stronger role in the pro- vision of technical assistance to health care providers to promote more rapid

62 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM improvements in health care quality? The evidence base for the committee's Performance Measurement report (IOM, 2006) and for the present study (see the introduction to this report) indicates that the health care received by Americans is seriously inadequate, and that the health care system needs to produce better-quality care that meets standards supported by the best scientific evidence. While these shortcomings have been in the headlines and discussed extensively in the provider community for many years, progress toward improved quality continues to be painfully slow. The com- mittee concludes that the public sector needs to play a substantial role in improving the quality of care for all Americans. This is especially so for those who depend upon federal programs, such as Medicare, Medicaid, the State Children's Health Insurance Program, the Department of Defense's TRICARE, and the programs of the Veterans Health Administration and the Indian Health Service, which provide coverage and care to roughly 100 million people (IOM, 2002). The creation of a performance measurement and reporting system and the implementation of payment systems that re- ward quality care are important steps in accelerating improvement, but they are insufficient to achieve the six quality aims outlined by the IOM (IOM, 2000, 2001). The committee believes the federal government is well suited to promote better-quality care for all Americans because it spends more than $513 billion annually on their care. The magnitude of this investment can generate positive changes in quality throughout the health care system (IOM, 2002). As the IOM concluded previously, "the federal government must assume a stronger leadership role to address quality concerns" (IOM, 2002:x). Second, given a role for the federal government in providing technical assistance for quality improvement, should that assistance be offered at no cost to providers, as is now the case through the QIOs, or should providers be expected to pay for the assistance? The following findings can help an- swer this question: · Many providers have resources that allow them to purchase techni- cal assistance from private organizations, such as those discussed in Chap- ter 3 and listed in Table B.1 in Appendix B. Also, many providers spend considerable sums to hire quality improvement experts and to conduct im- provement programs internally. While the extent and distribution of such expenditures are unknown, they are generally considered to be more com- mon among hospitals, large physician groups, and managed care organiza- tions than among medium and small physician practices and other provider settings. The committee does not wish to discourage such behavior. · At the same time, particular attention needs to be paid to disparities in access to and use of technical assistance resources within the provider community. Some providers are ready and willing to undertake the internal

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 63 changes needed to improve quality but lack the resources to purchase the technical assistance they need to achieve these reforms. · Continuing to achieve improvement at the current rate would be costly in lives and dollars and is therefore unacceptable. · Regardless of the current level of expenditures by providers for tech- nical assistance to improve quality, the end result is clear--improvements in quality nationally are progressing too slowly. While many providers may be willing to expend their own resources to improve quality when payment becomes based on such improvements, others are likely to be left behind. Some providers simply do not have sufficient resources to purchase the necessary services readily. The expansion of pay-for-performance systems to cover a greater portion of a provider's revenues is likely to worsen this situation. Providers with low levels of quality according to the given metrics may receive no increase or perhaps even a decrease in reimbursements. · In some areas, poor-quality providers may be the only ones available to a portion of the population, and their failure would further exacerbate disparities in health care access and quality. The committee concludes that it is important to improve the level of care offered by all providers, even if they cannot afford to purchase techni- cal assistance privately, and that some level of technical assistance should be available through the federal government as a public good. As providers improve their care processes for Medicare beneficiaries, it is likely that other patients will benefit as well. PROGRAM INFRASTRUCTURE Given the committee's limited ability to attribute quality improvements in Medicare directly to the efforts of the QIO program, it is necessary to consider whether the current program should continue. Although it may appear obvious, the committee believes the existence of 41 separate organi- zations holding QIO contracts dedicated to providing quality improvement services in every state, the District of Columbia, Puerto Rico, and the Virgin Islands is a significant asset. The cadre of trained experts in QIOs (see Chap- ter 7) is a potentially valuable resource for offering technical assistance in quality improvement and for helping hospitals, nursing homes, home health agencies, and outpatient physician practices collect data on their perfor- mance and aggregate and analyze those data to improve the care they deliver. The QIO program has created an infrastructure across the United States with staff trained and experienced in various quality improvement tech- niques. The QIO program serves as a focal point within CMS for assisting health care service providers in improving the care they offer to Medicare

64 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM beneficiaries and others. It is also the only program within CMS that ac- tively addresses health disparities (Jost, 2005). The following are examples of the QIOs' expertise and experience drawn from the committee's QIO and Regional Office site visits, telephone interviews, and web-based data collection tool: · All but seven QIO contracts are staffed with at least one employee who is a Certified Professional in Healthcare Quality (through the Health- care Certification Board, the National Association for Healthcare Quality, or a similarly recognized professional accreditation in quality improvement). Eleven QIOs have from 10 to 34 such certified staff. · Eighty-eight percent of QIO CEOs believe their leadership staff have substantial competencies for such functions as collaboration, relationship building, team development, and performance measurement. Many CEOs stated that their QIOs have individuals with other leadership skills avail- able to carry out the tasks of the 8th SOW. · A national survey of the QIOs by the Best Practices QIOSC (Qualis Health, 2004) showed that the quality improvement management staff of many QIOs were familiar at a minimum with various quality improvement techniques and programs, such as the Baldrige criteria, a collaborative meth- odology, human factors, International Standards Organization (ISO) 9000 criteria, Lean principles, and the Six Sigma program (see Chapter 7 for a discussion of QIO staff training and Chapter 9 for a description of each method). Some QIOs indicated having used some of those tools; only a small number of QIOs reported having staff certification for specific pro- grams, such as Six Sigma. Collaboratives were a major method used in the 6th and 7th SOWs, and many QIO staff were trained in its use. This is reflected in the fact that 98 percent of the QIOs reported being familiar with the method, and 95 percent of the QIOs reported having used it (per- sonal communication, J. Kelly, June 29, 2005). Almost half of the respon- dents were familiar with even the least common methods--those based on ISO 9000 criteria and Lean principles (Qualis Health, 2004). · The QIOs had a separate task in the 7th SOW to reduce disparities between an identified group, such as a particular underserved or rural popu- lation, and a reference group. In the 8th SOW, the disparities task was folded into all the quality improvement technical assistance tasks rather than remaining separate (see Chapter 8). · All QIOs gained experience in assisting hospitals with reporting mea- sures during the 7th SOW, when nearly all hospitals decided to participate voluntarily in public reporting to avoid a payment reduction. QIOs were also involved with the public rollout of comparative nursing home mea- sures (see Chapter 11).

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 65 Not only does each QIO have technical expertise in quality improve- ment, data management, and data analysis, but significant expertise in these areas also exists within CMS at the national and regional levels (see Chap- ter 13). The Central and Regional Office program staff may appear to be small (132 full-time equivalents) relative to the size of the program they oversee, the variety of tasks they perform, and their responsibilities for im- proving the quality of care for Medicare beneficiaries. Among the staff, however, are people with advanced training and experience in medicine, nursing, quality improvement, epidemiology, scientific research, and data systems (CMS, 2004; personal communication, J. Kelly, September 8, 2005). Although frustration and complaints with regard to the staff at CMS were voiced frequently during the committee's site visits to QIOs, telephone interviews, and other informal meetings, 85 percent of the QIOs rated their Project Officer "good" or "excellent" on expertise; the Scientific Officers received similarly high ratings on their performance (see Chapter 13). The QIO program's administrator reported that the program has sufficient staff and expertise to conduct the work required under the 8th SOW (personal communication, W. C. Rollow, July 8, 2005). In addition, the program has established potentially useful national communications networks and a data repository that currently serve both the QIOs and the public reporting of hospital data (CMS, 2004) (see Chapter 13 for further discussion of the data systems). The committee notes that at both the state and the national levels, the QIOs and CMS have established important working relationships with pro- viders, their professional associations, and various other stakeholder groups and convened parties around specific issues (CMS, 2004; Westat, 2005; National Health Policy Forum, 2004). The QIOs and CMS have also been major participants in such collaborative efforts as the Hospital Quality Al- liance and the Ambulatory Care Quality Alliance. Both the convening abil- ity of the QIOs and the clout of CMS to bring national organizations to the table are key ingredients for promoting widespread, coordinated quality improvement. The committee recognizes the expertise available within the QIO pro- gram and the enthusiasm, commitment, and dedication to quality improve- ment exhibited by staff and leadership from the Central Office at CMS, the Regional Offices, and all the organizations holding QIO contracts. The com- mittee concludes that the potential exists for the QIO program to have a measurable positive impact on improving the quality of care for Medicare beneficiaries, serving useful and important functions in the rapidly chang- ing world of Medicare.

66 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM FUNCTIONS OF THE QIOs QIOs have three main functions: (1) offering technical assistance to providers to help them improve the quality of care they deliver; (2) provid- ing education for beneficiaries and communications; and (3) protecting ben- eficiaries and the Medicare Trust Funds through the review of complaints, appeals, and other cases. Technical Assistance for Quality Improvement As discussed above and in detail in Chapters 9 and 10, the existing evidence concerning the impact of QIO quality improvement interventions is insufficiently robust to permit the attribution of improvements to QIO activities. This lack of evidence neither supports nor refutes the effective- ness of the QIO program, nor does the evidence make clear which QIOs are doing a better job of providing technical assistance for quality improve- ment. Over time, a QIO may show major improvements on statewide mea- sures in one provider setting but not in others. Even if a state excels on all the measures, the improvements cannot be attributed to the QIO with con- fidence with current data. Research concerning the various quality improve- ment methods used by QIOs and other organizations is also meager, is unclear about what methods work the best, and does not support the use or avoidance of any particular methods. Offering technical assistance to a broad variety of providers is a signifi- cant challenge for QIOs. In the 7th SOW, technical assistance was provided to nursing homes, home health agencies, physician practices, hospitals, and Medicare Advantage plans. The Medicare Prescription Drug, Improvement, and Modernization Act (P.L. 108-173) added more provider settings re- lated to the new drug benefit, beginning in the 8th SOW with specific projects related to physicians' medication therapy management and to phar- macies working with prescription drug plans. The committee concludes that the current variety of provider settings is sufficiently demanding of QIO staff, and indeed includes more providers than the QIOs are likely to be able to reach. Thus it would be imprudent to expand this task to encompass additional types of Medicare providers, such as clinical laboratories or therapists, at this time. The biggest challenge faced by the QIOs may be delivering technical assistance to the existing range of providers that helps improve quality across all settings, enhances patient transitions from one provider setting to the next, and promotes public health. Expanding the scope of technical assistance and including additional provider settings might be considered in the future should the QIOs achieve sufficient success with their current responsibilities and should there be a compelling need for technical assistance to improve quality in other areas.

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 67 During the committee's site visits and telephone interviews, most CEOs of QIOs stated a preference for working with providers considered early adopters, champions, or opinion leaders--or at least providers at a mix of readiness levels--since they believe early adopters are key to the increased diffusion of ideas (see Chapter 8). Concerning the recruitment of providers to QIO improvement projects, CEOs generally agreed that working mainly with the worst-performing providers would be more difficult, slower, and more costly and would result in less diffusion of improvements to other providers. They recommended against focusing QIO resources solely on poor performers because the stigma of being labeled the "worst" would create problems with attracting willing participants. The committee recognizes that focusing QIO technical assistance on a mix of providers--early adopters and leaders, as well as slower adopters-- may have certain benefits. Indeed, ideally there should be sufficient resources for QIOs to assist all providers requesting help. The committee considers it important for the QIO program to focus on assisting providers at a low quality level or those that would be unable to obtain assistance through the private market, particularly with the expansion of public reporting and pay- for-performance programs; however, those providers would not necessarily be unwilling, unready for change, or the slowest adopters of change. Some CEOs also noted that those providers starting a collaborative with the low- est quality measures often make the greatest improvement. Beneficiary Education and Communications The 7th SOW reduced funding for education functions, activities tar- geting beneficiaries and communications, and the 8th SOW shifts a greater proportion of funding to communications than to education. Nonetheless, under the 7th SOW the QIOs conducted communications campaigns to inform the media, hospital discharge planners, and beneficiaries about the rollout of Nursing Home Compare, Home Health Compare, and Hospital Compare,1 as well as to promote the use of quality information on provid- ers (see Chapter 11). Although some QIOs used communications to benefi- ciaries to promote the use of preventive services, most reported to the com- mittee that they are oriented more toward providers, and most do not perceive beneficiaries as their primary customers even though improved health outcomes for beneficiaries is the ultimate goal (CMS, 2004). 1CMS created websites, designed for beneficiary use, that include comparative data on each provider by name.

68 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Protection of Beneficiaries and Trust Funds The fact that QIOs conduct case reviews of local providers for Medi- care payment and coding purposes causes some providers to view them in a regulatory light; this perception can hinder QIOs' attempts to gain the co- operation of providers for quality improvement (NORC, 2004; Bradley et al., 2005). Although some anecdotal evidence suggests that QIO rela- tions with providers have improved substantially since the early days of the program, when the role of the Utilization and Quality Control Peer Review Organizations (PROs) was predominantly regulatory and led to the imposi- tion of significant sanctions, there appears to be lingering resistance among some providers to working with their QIO (National Health Policy Forum, 2004). CMS's surveys of quality improvement officers among providers who both did and did not work intensively with QIOs showed a generally high level of satisfaction (80 percent or better according to QIONet, CMS's internal Dashboard website) in most states and nationally among all pro- vider types except nonidentified nursing home participants (personal com- munication, M. G. Wang, July 7, 2005). However, the views of the CEOs may differ from those of their quality improvement officers. Indeed, the committee is concerned about the inherent conflict between the QIOs' regu- latory and quality improvement roles (see recommendation 4 in Chapter 4). Nearly all QIO CEOs consider case reviews to be a useful adjunct to the provision of technical assistance for quality improvement. The commit- tee finds that reviews of appeals requiring consideration of a patient's record should not require local physicians to review the record. Local physicians and their representatives may choose to be involved in the development of national measures, guidelines, and quality standards, but once those stan- dards have been set, a local interpretation would be superfluous. Since na- tional guidelines and quality standards should be used, the reviews could be conducted at a central point, although some beneficiary complaints involv- ing mediation would require a local presence. Various agencies and organi- zations in each state, such as the state medical board and the state depart- ment of insurance or health, carry out similar functions. If the bulk of case reviews were conducted at the regional or national level, aberrations at the state level could be perceived more clearly. Also, case reviews at the re- gional or national level would be more efficient because at present, each QIO must support a full range of on-call specialists 24 hours a day, 7 days a week for appeals requiring a rapid turnaround. It is not possible to determine the cost-effectiveness of the various cat- egories and types of case reviews given the available data. The abstracting of case records for payment reviews, the staffing of on-call specialists 24 hours a day, 7 days a week for urgent appeals, and the intensity of the labor required for all reviews are costly elements of the QIO review pro-

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 69 cess, whose value is not clear. CMS has not measured the impact of case reviews on the quality of care received by beneficiaries, nor has the pro- gram tracked the costs and savings associated with the QIO reviews through to the end of the process to determine the costs and benefits of the various categories of reviews throughout the Medicare program. Recent data show that QIOs receive relatively few complaints--about 1 for every 14,000 beneficiaries (Gaul, 2005). Additionally, only about one- fourth of these complaints are found to be justified as quality problems, and even fewer sanctions are imposed; today, an average of only 1 sanction per year is imposed (compared with an average of 31 sanctions per year two decades ago). CMS data on the effectiveness of complaint reviews are insuf- ficient for determining trends (see Chapter 12). Additionally, few analyses have been done to determine the effectiveness of payment reviews in terms of cost savings overall. STRUCTURAL ISSUES Program Funding The QIO program is funded through an apportionment from the Medi- care Trust Funds rather than an annual appropriation.2 (Structural issues are discussed in detail in Chapter 7.) The apportionment mechanism allows the U.S. Department of Health and Human Services (DHHS) and the fed- eral Office of Management and Budget to determine the program's needs and to draw on existing resources for that support. Yet the funding level for the QIO program has been very small relative to the Medicare program's level of spending on benefits, shrinking to approximately 0.1 percent in the 8th SOW. The committee was unable to find comparable data for Medicare Advantage plans or other private health plans. Under the 7th SOW, funding for the 53 QIO core contracts was ap- proximately two-thirds of the estimated total funding for the QIO program of $1.154 billion (see Chapter 7 for detailed discussion of funding). Tasks focused directly on technical assistance for quality improvement consumed more than half of the core contract amount--approximately $449 million from 2002 to 2005. The remaining one-third of the total apportionment went to contracts for the QIOSCs, special studies conducted by selected QIOs, and support for data systems as well as other CMS quality-related activities (personal communication, C. Lazarus, March 17, 2005). The stud- 2Much of the data on these issues is drawn from the committee's web-based data collection tool and telephone interviews.

70 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM ies conducted with these funds have not been systematically disseminated among the QIO community, and their value has not been assessed. Under the 8th SOW, technical assistance tasks are expected to receive a slightly smaller proportion of the total budget, although the number of such tasks has increased, because responsibilities and funding for case reviews have expanded. The 3-year budget for the overall QIO program increased to $1.265 billion in the 8th SOW, a 9.6 percent increase over the budget under the 7th SOW--roughly a flat budget plus inflation (personal commu- nication, D. Adler, May 23, 2005). QIO Board Composition, Function, and Structure Consumer Representation on QIO Boards The boards of the 41 organizations holding QIO core contracts sur- veyed for this study vary considerably in size and are heavily dominated by physicians and other clinicians and providers (see Chapter 7 for further discussion of QIO boards and their membership). As noted above, most QIO CEOs consider health care providers, not consumers, to be their main audience. Although consumer representation was mandated more than 20 years ago in the Peer Review Improvement Act (P.L. 97-248), QIO boards rarely have more than the one mandated consumer representative, who is identified primarily as a consumer or beneficiary. The Consumer Advisory Council in each QIO, established under the 7th SOW, is not a substitute for a stronger consumer­beneficiary presence on the board, since the council is only advisory to the board and has a limited focus on policy direction for consumer-related issues (CMS, 2002:35). The committee concludes that, given the growing role for patients in the management of their own care, the need for patients to choose caregivers on the basis of publicly reported quality measures, and an accepted definition of quality health care that includes patient-centeredness, a single consumer on a QIO board is unlikely to have sufficient influence on QIO activities. Also, cultural sensitivity to the diversity of the population is important if QIOs are going to create successful technical assistance programs for providers. Although the QIOs' focus on providers is appropriate and reflects their historical and current functions, greater consumer input on the boards is needed to help shift providers toward the provision of more patient-centered care. Other Board Issues Several other elements of the operations of QIO boards may affect their capacity to offer the QIO leadership and guidance. Four areas involving board oversight and direction would benefit from closer attention:

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 71 · Broader representation of various disciplines and expertise--The QIOs have diverse constituents, including hospitals, physician practices, nursing homes, and home health care agencies. However, the QIO boards appear to have relatively few members from health professions other than physicians. The committee found that physicians dominated nearly all the QIO boards, that two-thirds of all board members listed in the IOM's web- based data collection tool were physicians, and that all responding QIO CEOs (39 of 41) listed representatives of office-based practices on their boards. In particular, nurses (especially those in home health care), physi- cian assistants, pharmacists, and physical and occupational therapists ap- pear to be in short supply on QIO boards and are nonexistent on most boards, even though these types of health care professionals could play critical roles in improving the quality of care. Boards need to become more inclusive, encouraging participation from a variety of stakeholders and rel- evant experts. The growing responsibilities of QIOs during the 7th SOW included assisting stakeholders with data measurement, collection, aggrega- tion, analysis, and reporting as part of their provision of technical assis- tance for quality improvement. The committee expects those functions to increase in importance, and the minimal representation of QIO board mem- bers with expertise and experience in information technology management and oversight is a concern. Board members from outside the health care sector with experience in quality improvement, process measurement, and performance accountability could also contribute to the governance of QIOs. At the same time, when considering the addition of consumers, health professionals, and others to its board, a QIO must balance the need to keep the board's membership to a workable number. The creation of alternative mechanisms, such as advisory committees, may be necessary to obtain the varied input needed while keeping the board to a manageable size. · Board member development and assessment--Studies have shown that organizational leadership can benefit from the implementation of a systematic plan for individual board member development, an annual as- sessment of each board member's performance, and an evaluation of the board as a whole (Orlikoff and Totten, 2005). Such assessments are par- ticularly important in light of recent scandals involving both for-profit and not-for-profit health care entities; passage of the Sarbanes-Oxley Act of 2002 (P.L. 107-204), which brings greater accountability and transparency to the for-profit corporate world; and efforts in the not-for-profit sector by Board Source, Independent Sector, the Aspen Institute, and the Center for Healthcare Governance, which have recommended new requirements for board accountability. Just under a quarter of the 41 organizations holding contracts under the 7th SOW reported that they had formal mechanisms in place for evaluating the performance of individual board members; the same proportion of QIOs evaluate the overall performance of their boards.

72 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM · Financial oversight and strategic guidance--Only 23 of the 41 orga- nizations with QIO contracts have a board finance committee, only 15 re- ported that they have an audit committee, and only 8 reported that they have a strategic planning committee. Thus there appears to be a need for increased attention to these important areas of board responsibility to pro- vide adequate oversight and strategic direction to QIO management. · Overall responsibility--In addition to financial oversight and strate- gic guidance, the QIO board should be responsible for assessing the overall performance of the CEO and senior staff, as well as the accomplishment of defined goals and priorities. Physician-Access or Physician-Sponsored QIOs Outmoded Requirement The legislative requirement that QIOs be physician-access or physician- sponsored organizations was intended to assure physicians that their clini- cal work would be reviewed by their peers on the basis of local practice standards. Although certain clinical assessments and case reviews still re- quire medical expertise, the physicians need not be local because quality of care is now defined according to evidence-based national guidelines and standards rather than local patterns of practice. The committee concludes that this requirement is now outmoded and unnecessarily limits the compe- tition for QIO contracts from other entities. Limited Competition The committee finds that few entities other than out-of-state QIOs have been serious competitors for QIO core contracts in the past or are expected to compete for the 8th SOW, given the wide and complex assortment of required tasks and structural requirements. Other entities that are not physician-access or physician-sponsored organizations might have the ca- pacity to perform all or portions of that contract (see Chapter 3 for further discussion of "other entities"). Yet there has been a history of very limited competition for QIO contracts, and there have been few opportunities to replace QIOs that did relatively poorly or that failed on their previous contract: · All but two of the QIOs in the 7th SOW held their state's contract under the 6th SOW. · Fully 70 percent of the 7th SOW contractors were also contractors under the 1st SOW.

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 73 · No outside entities bid against the previous QIO for three of the seven state contracts opened for competition for the 7th SOW. · As of the end of the 7th SOW, five contracts had been recompeted and one was to be; there were no competitors for three of five contracts (personal communication, S. Pazinski, November 14, 2005). Conflicts of Interest Federal regulations for QIOs are designed to foster impartiality and to prevent conflicts of interest between members of the QIO board and the health care providers whose care is the focus of the QIO assessment through case review (see Chapter 7). These regulations contribute to a perception of neutrality and respect for the QIO, as well as acceptance of its quality im- provement and case review tasks, among the provider community and ben- eficiaries. They also mean that the QIO boards have derived limited benefit from the involvement of executives from the provider community, such as hospital CEOs whose hospitals might come under the review of a QIO. Under the 8th SOW, organizations that hold QIO contracts are not permitted to contract with providers in their state(s) for technical assistance or to review services for Medicare beneficiaries that are similar to the assis- tance or services they provide under the QIO contract. Although some pro- viders and insurers in some states may wish to purchase additional QIO services beyond those supported by Medicare funds, the QIO is not allowed to negotiate such a contract (personal communication, D. Schulke and T. Ketch, American Health Quality Association, June 30, 2005). This prohibi- tion appears to be based on a perception that lucrative contracts with the QIO could influence relations with certain providers and the review of their cases. The potential for such conflicts appears less likely now, however, given the decreased emphasis on case review and sanctions since the early days of the Professional Standards Review Organization program. There is clearly a market demand for the additional services of some QIOs, and under its current level of funding, the Medicare QIO program is unlikely to meet fully the future need for technical assistance with quality improve- ment. The committee finds that many QIOs have demonstrated a capacity to develop external revenues (not from CMS) and extensive relations with providers and provider organizations within their own states. At least 15 QIOs receive more than half of their revenues from sources other than the core contract. The committee concludes that if the case review functions were no longer maintained within each QIO, the need for this contracting prohibition might be eliminated. Reasonable controls, however, would be necessary to prevent favoritism in the QIOs' selection of recipients of their technical assistance with Medicare quality improvement.

74 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Confidentiality Restrictions The attitudes of the early predecessors to the QIOs and physician pres- sure have generated strict confidentiality restrictions on the QIOs' treat- ment of clinical data. The committee concludes that these restrictions pro- tect the privacy of providers and are not well suited to the current need for transparency, public reporting, and consumer access to information (Gaul, 2005). The restrictions also constrain the use of data on quality measures to support the quality improvement process. These restrictions were estab- lished mainly through regulations, and the secretary of DHHS could make policy changes consistent with Health Insurance Portability and Account- ability Act (HIPAA) regulations without the need for new legislation (per- sonal communication, T. Jost, Washington and Lee University School of Law, December 21, 2004). Consistent with CMS's concerns regarding the use of data and protec- tion of providers, CMS required that for this study, the committee use mea- surement data on the performance of each QIO only in the aggregate for the nation as a whole, or deidentify the data to protect the QIOs. The IOM had never requested access to physician or patient data, which would have required protections. The committee finds that the program's lack of trans- parency concerning its key contractors is incompatible with the broader trends within Medicare and the health care system to disclose providers' quality measures through public reporting. OVERSIGHT OF THE QIO PROGRAM3 CMS has the challenge of managing the QIO program in the field, as well as integrating it into the operational responsibilities of the Medicare program. The main oversight functions required of the CMS Quality Im- provement Group, which runs the QIO program, include the following: · Operation of complex data processing and communications systems for the program · Management and evaluation of QIO core contracts and contracts for QIOSCs, special studies, and support contracts · Strategic planning needed to ensure the continued usefulness of the QIO program to the Medicare program Chapter 13 describes the various communications and management mecha- nisms used by the QIO program and its data systems. 3Most of the information in this section was drawn from the committee's site visits, tele- phone interviews, and web-based data collection tool.

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 75 Data Processing and Communications Communications among the QIO Central Office, the Regional Offices, and the QIOs are maintained through e-mails, list serves, restricted and public websites, and electronic reporting tools. To some extent, the QIOs are expected to operate in the virtual world, and they are encouraging pro- viders to move in that direction as well. Training for CMS staff and pro- grams, including QIO staff, is conducted over the Internet; communities of practice stay in touch with monthly conference calls; and meetings are video broadcast across the country to minimize travel. Some of these mechanisms do not work as well as the QIOs would like: CEOs claim that the frequency of conference calls makes it impossible to participate every time; that they do not receive policy memoranda and other necessary guidance promptly; and that reporting systems, such as those for case review and program ac- tivities, are difficult to use. Management communications among CMS Project Officers, QIOs, and CMS Contract Officers are particularly problematic. A QIO can be caught between two different interpretations of an issue, and the interpretation of the Contracts Office has priority. The CMS Project Officer provides routine guidance to the QIO and often serves as a communications link when policy questions arise. While the Project Officer may request an activity outside the scope of the QIO contract and beyond the budget, however, the Con- tracts Office may subsequently refuse to consider a contract modification. The Standard Data Processing System supports the communications tools mentioned above, as well as the flow and processing of data from medical records. It is key to the QIO program, and 63 percent of the QIOs rated the value of the system as excellent or good, although some QIOs mentioned the need to update the system and integrate it with their own equipment. It will be important for QIO and CMS staff to be closely involved with national and regional initiatives concerning data exchange, to understand strategic policy issues related to health information technol- ogy, and to keep their software and hardware current. The main concern QIOs expressed is that the data reported through the Dashboard section of the QIONet internal CMS website and used to monitor the progress of the QIOs' quality improvement efforts often are not timely (see Chapter 13). Although there is some conflicting evidence about the value of rapid data feedback (Beck et al., 2005), the QIOs and providers frequently stressed the usefulness of real-time data for quality improvement and for the care of specific patients. As CMS increases the measures required for public reporting by hospi- tals and physicians, the challenges to the Standard Data Processing System will increase. (CMS conducts much of the processing of the data for nurs- ing home and home health care measures.) The committee concludes that

76 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM the growing volumes of data, the expanded numbers of data sources, the increased need for the timeliness of data processing, and the interest in user-friendly data reports for different stakeholders will exert pressures on the current system. At the same time, the transition to electronic health records may reduce the demand for data abstraction, increase the need for data-auditing services, and improve the timeliness of the data. Long-range planning for these changes is essential to smooth operations and should include an array of people with detailed operational expertise early in the process. The implementation of a national system for quality measurement and reporting will have an additional impact on the demand for data pro- cessing and other activities (IOM, 2006). The committee recognizes, how- ever, that full implementation of both electronic health records and the new national public reporting system will take time. Therefore, paper-based tools that can be used prospectively, as opposed to chart abstraction, will also be important. With DHHS fully supporting providers' move to the use of electronic health records, as well as electronic communications, Medi- care could serve as a guide to the future if its data systems keep pace with the transition. QIO Contract Management and Evaluation CMS's day-to-day management of each QIO contract is a major re- sponsibility of staff in the Regional Offices, who perform detailed on-site assessments at 9 and 18 months into the contract period and in the interim keep in touch with the QIOs to which they are assigned. However, most of a QIO's performance rating for contract renewal is based on data on pro- viders' performance on quality measures and complex formulas, with sepa- rate calculations for each task. The increased complexity of tasks in the 8th SOW led to a significantly more complicated set of formulas and incentive awards than those in the 7th SOW (see Chapter 10). The absence of an overall evaluation plan, guidance, and program priorities has created a situ- ation in which each of CMS's Government Task Leaders designs the evalu- ation formulas for his or her own tasks in the SOW. The formulas do not draw upon consistent time frames, goals, or definitions. The resulting com- plexity makes it difficult to construct a coherent overall assessment of a QIO. It is unclear, therefore, whether these formulas adequately assess "im- provement" and "quality," and the committee finds that the program's sys- tem for evaluating the QIOs requires further development to enhance its effectiveness. As recommended in a 1994 IOM letter report, staff with ex- pertise and experience in program evaluation, as well as outside assistance, are needed to design formative and summative evaluations of the QIOs and to contribute to the design of contract performance reviews (IOM, 1994). The complex method for evaluating the contract performance of QIOs

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 77 under their core contract is in sharp contrast to the limited, relatively simple methods used in the 7th SOW to assess the performance of the QIOSCs, QIOs conducting special studies, and organizations carrying out support contracts. Although these contracts are assessed for timeliness and com- pleteness of deliverables and spending, CMS relies primarily on the judg- ments of the Project Officers. It may be noted that contracts for QIOSCs, special studies, and program support accounted for almost one-third of the total apportionment for the 7th SOW. The value of these expenditures is diminished by the lack of a comprehensive listing of these various contracts and summaries of their contents and results, which are thus not diffused throughout the quality improvement community. Although many of the support contracts are related tangentially to the QIO program, QIOs may be unaware of those contracts or of a special study that is under way and would be of interest to them. CMS is in the process of developing a manage- ment and accountability plan to better administer all of its contracts. Few details were available as of this writing, however, and implementation of the plan remains to be seen. One factor that may have contributed to the previously discussed lim- ited competition from other entities for the 8th SOW was the timing of submission of proposals for the core contract. The request for proposal was released on April 1, 2005, and the deadline for submission of proposals was April 15, 2005, before the SOW was final and the full amount of funds available was known. An official version of the 8th SOW was issued on May 20, 2005, but some elements were still under discussion within DHHS and the Office of Management and Budget that could have required future and unknown modifications to the SOW. In the first contract round, the QIOs were required to submit their bids before the release of the final SOW, and the bids had to be based on a draft SOW and spending estimates. Ma- jor revisions to the 8th SOW were made on November 4, 2005, well past the beginning of the new contract period. Because late revisions can affect QIO staffing and work schedules, QIOs reported that the timing has been frustrating and has hampered their ability to prepare for the start of the new contract (personal communication, D. Schulke and T. Ketch, Ameri- can Health Quality Association, June 30, 2005). Additionally, delayed re- leases may deter organizations with no previous experience in holding a QIO contract from bidding because of the inadequate amount of time to prepare. During the committee's site visits and telephone interviews, several QIOs indicated that the 3-year length of the contract is also a problem. They believe this period of time is insufficient to obtain data describing the initial impact of their quality improvement projects and to allow them to adjust their interventions and later be remeasured for their contract perfor- mance evaluations. Given the experience with the preparation for the 8th

78 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM SOW, as well as the increased expectations of this SOW, it appears that CMS would also benefit from a longer contract period. Strategic Planning Two positive signs concerning policy planning for the QIO program suggest to the committee that CMS is moving in the right direction. First, the Quality Improvement Group has initiated a long-range planning pro- cess to better position the program 12 years into the future. Although some strategic planning discussions have recognized the importance of coordi- nated care and crosscutting issues in improving the overall safety and qual- ity of care, representatives of hospitals, home health care agencies, physi- cian practices, and nursing homes have not had opportunities to meet to discuss appropriate measures and mechanisms for tracking a patient being released from one facility and placed in another. Much of the internal stra- tegic planning process has focused on individual tasks and separate Gov- ernment Task Leaders; the external planning process with stakeholders has followed the same pattern of separating each of the four main health care settings. The committee concludes that implementation of the new Part D prescription drug benefit and therapeutic pharmacy strategies offers impor- tant opportunities for QIOs to focus on the coordination of care across provider settings. Strategic planning separately for different providers and care settings is inconsistent with future measures of quality and perfor- mance that will be encouraged by the national performance measurement and reporting system recommended by this committee (IOM, 2006). Another positive sign concerning strategic planning is the rejuvenation of the CMS Administrator's Quality Council and its staffing by the Quality Team. This management structure could give the QIO program an opportu- nity to integrate its activities with those of other CMS functions, such as public reporting, in the early planning stages. The council has adopted a vision of the QIO program--"the right care for every person every time"-- and has made it the vision of the whole agency. The Quality Council is coordinating the development of a number of quality-related projects and presented a Quality Improvement Roadmap to guide the agency's ongoing quality-related work. The current roadmap is more a listing of projects and strategy options than a guide with priorities. Nonetheless, the QIOs are identified as having a major role in one of five major strategies: "assisting practitioners and providers in making care more effective, particularly in- cluding the use of effective electronic health systems" (CMS, 2005b:13). Given the commitment throughout CMS to improving the safety and qual- ity of care nationally, the QIO program could play an important part in carrying out this strategy.

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 79 The committee concludes that thorough evaluations of the impact of the QIO program as a whole and of the effectiveness of specific quality improvement interventions, based on the articulation of clear and well- defined goals and priorities, would provide useful input to the long-range strategic planning process. Evaluations of QIO efforts in physician offices will be particularly important since the QIOs have had relatively little expe- rience in that provider setting, and there is little evidence of what approaches are most successful in stimulating major system changes. SUMMARY While there is evidence that the quality of care for Medicare beneficia- ries has been improving, the evidence concerning specific quality improve- ment efforts, the overall impact of individual QIOs, and the impact of the QIO program in the aggregate is limited and inconclusive. Nevertheless, the committee considers the QIO program to be a potentially rich resource as the health care system moves toward the increased use of performance mea- surement and pay for performance. The QIO program has shown consider- able adaptability over its 35-year history. It currently has a strong founda- tion of quality improvement experts in each state; a network of collaborative relationships with providers and stakeholders; and a national infrastructure to support data collection, reporting, aggregation, and auditing, as well as research to further the development and the use of quality measures. To make more effective use of these valuable resources, however, a major re- structuring of the program is needed. In Chapters 4 and 5, the committee presents its recommendations and rationales for these changes. First, how- ever, Chapter 3 presents a discussion of the changing health care environ- ment and new functions that will affect QIOs and other organizations con- ducting similar activities. It reviews the committee's first report and how implementation of that report's recommendations concerning the establish- ment of a national system for performance measurement and reporting will likely affect the QIO program. The chapter also examines other entities that might compete to perform the needed functions. REFERENCES Beck C, Richard H, Tu J, Pilote L. 2005. Administrative data feedback for effective cardiac treatment: AFFECT, a cluster randomized trial. Journal of the American Medical Asso- ciation 294(3):309­317. Bradley EH, Carlson MDA, Gallo WT, Scinto J, Campbell MK, Krumholz HM. 2005. From adversary to partner: Have quality improvement organizations made the transition? Health Services Research 40(2):459­476.

80 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Chu LA, Bratzler DW, Lewis RJ, Murray C, Moore L, Shook C, Weingarten SR. 2003. Im- proving the quality of care for patients with pneumonia in very small hospitals. Archives of Internal Medicine 163(3):326­332. CMS (Centers for Medicare and Medicaid Services). 2002. 7th Statement of Work (SOW). [Online]. Available: http://www.cms.hhs.gov/qio [accessed April 9, 2005]. CMS. 2004. The Quality Improvement Organization Program: CMS Briefing for IOM Staff. [Online]. Available: http://www.medqic.org/dcs/ContentServer?cid=1105558772835& pagename=Medqic%2FMQGeneralPage%2FGeneralPageTemplate&c=MQGeneral Page [accessed December 26, 2005]. CMS. 2005a. Best Practice Methods Special Study: Report of First Year Scan of QIO Inpatient Practice. Unpublished. Baltimore, MD: Centers for Medicare and Medicaid Services. CMS. 2005b. Quality Improvement Roadmap. [Online]. Available: http://www. medicaldevices.org/public/issues/documents/CMSMedicareroadmap.pdf [accessed De- cember 26, 2005]. Daniel DM, Norman J, Davis C, Lee H, Hindmarsh MF, McCulloch DK, Wagner EH, Sugarman JR. 2004. A state-level application of the chronic illness breakthrough series: Results from two collaboratives on diabetes in Washington state. Joint Commission Jour- nal on Quality and Safety 30(2):69­79. Dellinger EP, Hausmann SM, Bratzler DW, Johnson RM, Daniel DM, Bunt KM, Baumgardner GA, Sugarman JR. 2005. Hospitals collaborate to decrease surgical site infections. The American Journal of Surgery 190(1):9­15. Gaul GM. 2005, July 26. Once health regulators, now partners. Washington Post. p. A1. Gould BE, Grey MR, Huntington CG, Gruman C, Rosen JH, Storey E, Abrahamson L, Conaty AM, Curry L, Ferreira M, Harrington KL, Paturzo D, Van Hoof TJ. 2002. Improving patient care outcomes by teaching quality improvement to medical students in commu- nity-based practices. Academic Medicine 77(10):1011­1018. IOM (Institute of Medicine). 1994. An Assessment of the HCFA Evaluation Plan for the Medicare Peer Review Organization. Washington, DC: National Academy Press. IOM. 2000. To Err Is Human: Building a Safer Health System. Kohn LT, Corrigan JM, Donaldson MS, eds. Washington, DC: National Academy Press. IOM. 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Wash- ington, DC: National Academy Press. IOM. 2002. Leadership by Example: Coordinating Government Roles in Improving Health Care Quality. Corrigan JM, Eden J, Smith BM, eds. Washington, DC: National Academy Press. IOM. 2004. Keeping Patients Safe: Transforming the Work Environment of Nurses. Page A, eds. Washington, DC: The National Academies Press. IOM. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. Jencks SF, Huff ED, Cuerdon T. 2003. Change in the quality of care delivered to Medicare beneficiaries, 1998­1999 to 2000­2001. Journal of the American Medical Association 289(3):305­312. Jha AK, Li Z, Orav EJ, Epstein AM. 2005. Care in U.S. hospitals--the hospital quality alliance program. New England Journal of Medicine 353(3):265­274. Jost TS. 2005. Racial and Ethnic Disparities in Medicare: What the Department of Health and Human Services and the Centers for Medicare and Medicaid Services Can, and Should Do. National Academy of Social Insurance Study Panel on Sharpening Medicare's Tools to Reduce Racial and Ethnic Disparities. Washington, DC: National Academy of Social Insurance. Kiefe CI, Allison JJ, Williams OD, Person SD, Weaver MT, Weissman NW. 2001. Improving quality improvement using achievable benchmarks for physician feedback: A randomized controlled trial. Journal of the American Medical Association 285(22):2871­2879.

ASSESSMENT OF THE QIO PROGRAM: FINDINGS AND CONCLUSIONS 81 Leatherman S, McCarthy, D. 2005. Quality of Health Care for Medicare Beneficiaries: A Chartbook. New York: The Commonwealth Fund. Marciniak TA, Ellerbeck EF, Radford MJ, Kresowik TF, Gold JA, Krumholz HM, Kiefe CI, Allman RM, Vogel RA, Jencks SF. 1998. Improving the quality of care for Medicare patients with acute myocardial infarction: Results from the Cooperative Cardiovascular Project. Journal of the American Medical Association 279(17):1351­1357. National Health Policy Forum. 2004. Quality Improvement in Maryland: Partnerships and Progress. Washington, DC: National Health Policy Forum. NCQA (National Committee for Quality Assurance). 2005. The State of Health Care Quality Report: Industry Trends and Analysis. Washington, DC: National Committee for Qual- ity Assurance. NORC (National Organization for Research at the University of Chicago). 2004. Final Re- port: Physician Meetings on Take-Up of Electronic Health Records. Unpublished. Wash- ington, DC: National Organization for Research at the University of Chicago. Orlikoff JE, Totten MK. 2005. The pros and cons of board compensation. Does the promise justify the means? Healthcare Executive 20(1):46­48. Qualis Health. 2004. HCQIP Improvement Methodologies Survey Results. Seattle, WA: Qualis Health. Rollow W. 2005. QIO Program: Update and Policy Considerations. PowerPoint Presentation to the Committee on Redesigning Health Insurance, June 13, Washington, DC. Westat. 2005. Survey for Provider Satisfaction with Quality Improvement Organizations. Unpublished. July 1. Rockville, MD: Westat.

Next: 3 Performance Measurement, Quality Improvement, and Other Entities »
Medicare's Quality Improvement Organization Program: Maximizing Potential Get This Book
×
Buy Hardback | $81.00 Buy Ebook | $64.99
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Medicare’s Quality Improvement Organization Program is the second book in the new Pathways to Quality Health Care series. Focusing on performance improvement, it considers the history, role, and effectiveness of the Quality Improvement Organization (QIO) program and its potential to promote quality improvement within a changing health care delivery environment that includes standardized performance measures and new data collection and reporting requirements. This book carefully examines the QIOs that serve every state as well as the national program that guides and supports them. In addition, it highlights the important roles that a national program with private organizations in each state can play in promoting higher quality care. Medicare’s Quality Improvement Organization Program looks closely at the technical assistance role of the QIO program and the need to encourage and support providers to improve their performance. By providing an in-depth assessment of the federal experience with quality improvement and recommendations for program improvement, this book helps point the way for those who strive to create higher quality and better value in health care. Intended for multiple audiences, Medicare’s Quality Improvement Organization Program is essential reading for members of Congress, the federal executive branch, the QIOs, health care providers and clinicians, and stakeholder groups.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!