Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
5 CMS Oversight of the Operations and Management of the QIO Program CHAPTER SUMMARY This chapter focuses on the Centers for Medicare and Medicaid Services (CMS) and its oversight of the operations and manage- ment of the Quality Improvement Organization (QIO) program, including communications among the various program partici- pants, data processing, strategic planning, and program evaluation and funding. CMS's current management is examined from two perspectives: the first considers how CMS could improve the opera- tions of the QIO program in the short term; the second is a longer- range perspective that considers how CMS and the QIO program might fit into the operations of a national performance measure- ment and reporting system once it is fully implemented. The Centers for Medicare and Medicaid Services (CMS)--specifically the Quality Improvement Group, which manages the Quality Improvement Organization (QIO) program--has undertaken considerable discussions re- garding tools and methods the Medicare program can use at the national level to enhance the quality of care received by both Medicare beneficiaries and other patients (Rollow, 2005; personal communication, W. Rollow, CMS, July 7, 2005). Likewise, long-range issues related to better quality measurement and reporting, such as the expansion of public reporting and implementation of pay-for-performance methods, are being discussed at various levels within CMS (CMS, 2005; McClellan, 2005). The Quality Improvement Group is planning, under the guidance of a consultant, well beyond the 8th scope of work (SOW) to consider the role of the QIOs 12 years into the future, during the 12th SOW (W. C. Rollow, unpublished data, 2005; Rollow, 2005). With a horizon of 2017, it can be tempting to assume that major changes, such as the universal adoption of electronic health records (EHRs), will reduce or eliminate certain current problems. 120
CMS OVERSIGHT AND MANAGEMENT 121 The committee encourages both short- and long-range planning. It also cau- tions, however, that managerial and organizational problems of current concern should be tackled now instead of being deferred until potential intervening changes occur. At the same time, near-term actions should be undertaken with the aim of moving the health care delivery system closer to the long-range goal of producing the right care every time for every person. The recommendations presented below reflect three basic themes voiced to the committee by many sources throughout this study: (1) the need to improve the processing and availability of quality-related information at all levels throughout the operations of the QIO program; (2) the need to man- age the program better to increase the likelihood of quality improvements; and (3) the need for evaluations to determine what quality improvement methods do and do not work and under what circumstances, to assess the overall performance of individual QIOs, and to assess the impact of the QIO program overall on the quality of health care. Fundamental to each of these themes is the need for adequate funding, which is addressed in the committee's final recommendation. DATA PROCESSING Recommendation 5: The secretary of DHHS and CMS should re- vise the QIO program's data-handling practices so that data will be available to providers and the QIOs in a timely manner for use in improving services and measuring performance. · CMS should initiate a comprehensive review of its data-sharing systems, processes, and regulations to identify and correct prac- tices and procedures, including abstraction of medical chart data, that restrict the sharing of data by the QIOs for quality improve- ment purposes or that inhibit prompt feedback to the QIOs and providers on provider performance. · The QIO program should support the processes of national re- porting of performance measures, data aggregation, data analy- sis, and feedback. · The secretary of DHHS should allow and encourage the sharing of medical claims data when the sharing of such data is not pre- cluded by the privacy protections of the Health Insurance Port- ability and Accountability Act, as well as the sharing of more detailed complaint-resolution data with complainants. · CMS should work toward the ultimate goal of integrating more care data from all providers and public and private payers to create both records of patient care over time and population- level data.
122 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM · Independently of the core QIO contract, CMS should be respon- sible for ensuring and auditing the accuracy of the data submit- ted by providers that participate in the Medicare program. Pro- viders should be accountable for the validity and accuracy of the quality measurement data they submit. The QIOs should supply providers with technical assistance to improve the validity and accuracy of the data collected. Data Processing for Quality Improvement Interventions The QIO program needs to receive and provide timely data on quality indicators for use by the QIOs and providers during quality improvement interventions. Because the program relies on pre- and postintervention mea- surements both to provide feedback to the participating providers and to enable CMS to evaluate the QIOs' performance during the 8th SOW, time- liness is critical. The current 3-year contract cycle is relatively short, given the time needed to start up new interventions and the remeasurement dead- line for judging contract performance. For example, data presented by CMS on accomplishments under the 7th SOW show that the periods between pre- and postintervention measurements may be as short as 12 months for hospitals. It will be difficult to demonstrate rapid transformational changes during the 8th SOW, particularly because the QIOs need time to adjust to the SOW's new tasks and late release. The feedback loop to all participants in an intervention becomes too slow to provide guidance on the techniques that work or that need to be adjusted to achieve better results within the 3- year period of a SOW (see Chapter 13). Also, some providers value real- time data feedback to improve care for specific beneficiaries. Now that CMS publicly reports data on hospitals, nursing homes, and home health care agencies on at least a quarterly basis, relatively current data are available for a very limited set of measures at a frequency greater than that in the past. A rapid expansion of the measures included in the publicly reported measure sets, as envisioned in the committee's first report, Performance Measurement: Accelerating Improvement (IOM, 2006), would increase the amount of timely data that could be used for technical assis- tance, as well as for QIO performance assessments. Much of the publicly reported data represents all of a provider's patients, regardless of payer, and that approach should eventually be expanded to all provider settings. Until the number of measures collected for public reporting is increased, CMS should give priority to speeding up the process of data feedback and should ensure prompt data processing for future publicly reported measures. Methods for collecting data from paper records in physician offices need to be improved. In physician practices and facilities lacking EHRs, QIOs could help with the testing and use of paper-based templates or flow
CMS OVERSIGHT AND MANAGEMENT 123 sheets that could be used to collect encounter and performance data in real time during provider contact with the patient. Properly designed, such docu- ments could be cost-effective for entering data into electronic databases, could be less time-consuming than retrospective chart abstraction, and could serve as a guide for clinicians at the time of service. Retrospective chart review can also be a valuable tool for research and should not be ignored. At the same time, it is important to consider how the growing use of EHRs and the electronic transmission of the data from those records could enhance the ability of QIOs to offer technical assistance to physicians. The centralized development of tools that physician offices could use indepen- dently would aid in the acquisition and use of health information technol- ogy and would perhaps help the QIO program reach a larger audience. Some physicians who are already using EHRs could also use specially de- signed tools on their own to produce aggregated data and analyses to guide their quality improvements efforts, as well as to facilitate the redesign of their practices to take advantage of the EHRs. This would in turn allow the QIOs to concentrate on providers that have less capacity for adjusting to the new technology and a greater need for guidance, and would benefit from the experiences of other providers. With the expectation that many small physician offices will require hands-on assistance with the adoption of health information technology and that many thousands of practices may request assistance with office redesign, new methods to fill the need must be developed. Confidentiality of QIO Data There are currently three potential key audiences for QIO data on pro- vider performance: · Providers--both those participating in QIO projects who need to measure their own progress from the baseline and all providers who need benchmarking data to see how they compare to their peers. In general, providers tend to show an interest in publicly reported data, and many are motivated by the reporting of these data to improve their performance (Hibbard et al., 2005). · Consumers and beneficiaries--those who may want to compare the performance of providers on more measures than the handful now publicly reported and, if QIOs continue to have a role in case review activities, com- plainants who want to understand the details of their case. · CMS--which needs data for assessment of the performance of the QIOs and providers and for case review, complaints, and appeals. In addi- tion to highlighting specific providers that have quality problems or that provide high-quality care deserving of recognition by a payment incentive,
124 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM the performance data could highlight clinical or geographic areas in need of quality improvement. Beneficiary Confidentiality Even before a national performance measurement and reporting system is implemented, more data will begin to flow through the health care system and will receive increased scrutiny as reporting of measures expands. Greater openness needs to be balanced with the maintenance of protections for patients, particularly with the growing use of data from patient records. It will be essential to include strong safeguards in communications systems to protect the confidentiality of those records. Results of a national con- sumer survey show that two-thirds of the American public and nearly three- quarters of members of racial and ethnic minorities are concerned about the privacy of their personal health information (CHCF, 2005). The current QIONet Exchange, designed for the transmission of clinical quality data in the QIO program, is also used for the hospital public reporting program because it is secure and is compliant with the mandates of the Health In- surance Portability and Accountability Act. To maintain the confidence of beneficiaries, it is essential that any new communications systems or expan- sions of existing systems used for clinical reporting include similar safe- guards. Precautions concerning access to data at geographic or provider levels will also need to be taken when the sample or cell size is small enough to permit identification of individual patients. A national performance mea- surement and reporting system will need to ensure patient confidentiality for all data on measures and providers at the same time that it promotes the transparency of the data. Greater flexibility in the use of case data is also important for tracking the quality of services received by a patient during an episode of care or over a longer period. Longitudinal data that can be tracked across various providers or settings and linked to a patient identifier are essential to pro- moting improved care. Provider Confidentiality The current confidentiality requirements of the U.S. Department of Health and Human Services (DHHS) prohibit QIOs from sharing their data--such as those collected within a quality improvement collaborative, used in case reviews, or derived from a complaint or appeal--with provid- ers other than the source of the data unless the source agrees to their use. These confidentiality requirements also constrain the QIOs from sharing all but a minimum of information with complainants on how their cases were resolved. Complainants may receive notice only of whether their care met
CMS OVERSIGHT AND MANAGEMENT 125 professionally recognized standards; they may not receive information on what if any actions were taken as a result. If the physician withholds con- sent, the QIO cannot release the physician's name or other details to the complainant (Gaul, 2005). Consumers do not have access to data identify- ing sources and providers by name because of the same issues of confidenti- ality; CMS also has limited access to QIO data. (The publicly reported data from hospitals, nursing homes, and home health agencies are considered CMS data, not QIO data, and thus are not subject to the same confidential- ity restrictions.) The current QIO confidentiality restrictions are not necessary or sup- portable in the current era of public reporting and are incompatible with the aim of a national performance measurement system and the goal of CMS to serve multiple audiences with a transparent system. The committee recognizes the need to balance concerns about malpractice with the impor- tance of limiting the discoverability of QIO data. Because the QIO legisla- tion gives the secretary of DHHS the authority to set confidentiality stan- dards by regulation, new legislation for this purpose is not necessary (personal communication, T. Jost, January 7, 2005). Thus the secretary may establish new regulations that increase the transparency of QIO data and the ability to share those data. (See Chapter 7 for more detail on confi- dentiality issues.) Ensuring the Accuracy of Reported Data Many current public reporting efforts rely on the QIO program, its Clinical Data Abstraction Centers, and its national Data Warehouse at the Iowa QIO to edit, validate, aggregate, and store the reported data (personal communication, W. Matos, CMS, July 7, 2005; personal communication, J. Kelly, M. Krushat, and W. Matos, CMS, October 25, 2004; personal communication, M. B. McClellan, June 24, 2005). The Data Warehouse, originally designed for QIO data, has grown to accommodate the public data from hospitals for all payers and will need to grow further as the reporting of performance measures expands. (See Chapter 13 for more de- tail on data issues.) Although public reporting reduces the need for separate abstraction of data from patient records within the QIO program for some settings, CMS has an increased responsibility to audit the data that are reported to ensure their accuracy. Use of the data on quality measures for payment as well as public reporting purposes increases the incentives to manipulate the system and the need for accuracy. Although the data-auditing function will be cru- cial, there is no need for it to be a responsibility of every QIO. In fact, it would be better for the QIOs not to have this added regulatory function, as conflicts of interest could arise. Exercising this function could jeopardize
126 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM the QIOs' collegial relationships with providers in their states and their ability to conduct quality improvement interventions with providers, who participate as voluntary partners. Moreover, in the interest of maintaining good relations with providers, QIOs might be less than zealous in carrying out their regulatory auditing function. In addition, there does not appear to be a strong reason for conducting audits comparing reported data with data in case records at the state level. A national or regional contractor could conduct such reviews. Private organizations and QIOs with strong auditing capacities could compete for such contracts, with the QIOs precluded from conducting audits in states where they offer providers technical assistance for quality improvement. On the other hand, measure validation (consultation with providers on their initial submissions of data on new measures to ensure that the data accurately represent what is intended) is an important function the QIOs could appropriately handle. Because the QIOs have staff available in each state, they could meet with providers, particularly those in small practices or those that serve as safety net providers, to help them interpret their data before the data are publicly posted and to gather comments concerning any possible problems with the data. Decisions about the level at which the data repository should be estab- lished under a national performance measurement and reporting system-- national or subnational--will have to be made as part of the strategy for that system. In any case, the system will have to be able either to accumu- late data at the patient level and aggregate them to the local, state, and national levels or to do the reverse: accumulate the data in a national re- pository, with the ability to break them down to lower levels of aggregation for analyses, such as tracking patients across states and time. QIO PROGRAM MANAGEMENT Recommendation 6: CMS should establish clear goals and strategic priorities for the QIO program. Congress, the secretary of DHHS, and CMS should improve their management of the QIO program as necessary to support those goals, especially by enhancing con- tracting processes for the QIO core contract and QIO Support Cen- ter (QIOSC) contracts; integrating the program's core, support, and special study contracts; and improving coordination and communi- cation within the program. · CMS should provide the QIOs with a coherent and feasible scope of work that sets forth clear priorities for quality improvement and performance measurement.
CMS OVERSIGHT AND MANAGEMENT 127 CMS's priorities and planning efforts should focus on integrating QIO collaboration with various types of provid- ers to improve the coordination of patient care across mul- tiple settings. To prepare for the 9th scope of work, CMS should consider conducting a national survey of the main provider settings (nursing homes, home health agencies, hospitals, outpatient physician practices, end-stage renal disease facilities, prescrip- tion drug plans, and pharmacies) to determine specific unmet needs for technical assistance. Such information might be complemented by information from focus groups conducted with a mix of representatives from the various settings. The QIO core contracts and the QIOSC contracts should in- clude incentives aimed at promoting a broader transfer of knowledge concerning successful quality improvement inter- ventions and more rapid improvement. The QIOs should have the resources they need to conduct at least one locally initiated quality improvement project on the basis of demonstrated need and the design and evaluation cri- teria established by CMS. · Congress and CMS should change the contract structure for core QIO services for the 9th scope of work: Strong incentives and penalties that reward high performance and penalize poor performance should be included. CMS should encourage sufficient competition for the core contracts to permit the selection of a QIO contractor on the basis of contractor-proposed interim and final performance measures and goals. During the contract period, there should be less process management of internal QIO operations by CMS. Congress should permit extension of the core contract from 3 to 5 years to allow for the measurement, refinement, and evaluation of technical assistance efforts and the achievement of transformational goals. There should be greater competition for each new contract. CMS should consider previous experience and performance as a QIO among the selection criteria; demonstrated capacity to support quality improvement on the part of any eligible organization should predominate. Performance periods should be consistent. All QIOs should begin and end the contract cycle on the same date so the plan- ning, implementation, and evaluation of each scope of work can be applied nationally.
128 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM A timetable should be established for goal setting, program planning, and funding processes for the core QIO contracts. The schedule should ensure that new scopes of work are is- sued in a timely fashion, and that contracts and funding lev- els are developed and finalized so as to allow sufficient time for QIOs and competing organizations to prepare in advance for the new contract without major program and staff disruptions. · CMS should award QIOSC contracts several months in advance of a new QIO contract cycle to allow for the preparation of tools and materials for QIO use, definition of the required tasks and deliverables that will serve the QIOs and the Government Task Leaders, and inclusion of explicit methods for assessment of the contractor's performance. Congress and CMS should al- low entities other than QIOs with expertise in quality improve- ment to bid on QIOSC contracts; familiarity with QIO work, the capability to carry out the work, and experience in perform- ing the required functions should be appropriately weighted when the bids are assessed. · The QIO core contract and contracts for special studies, support services, and QIOSCs should all reflect the explicit goals and priorities of the program. · CMS and the Agency for Healthcare Research and Quality should establish ongoing mechanisms for sharing quality im- provement knowledge and research results, especially through QIOSCs. · CMS should take steps to improve coordination and communi- cations within the QIO program and with QIOs. In particular, the roles and responsibilities of and communications among Project Officers, Contract Officers, Government Task Leaders, Scientific Officers, and QIO executives and their staff should be clarified. CMS should build self-assessment, transparency, clearer com- munications, and continuous quality improvement into the daily workings of the team overseeing the QIO program, just as the QIOs expect providers to do. The contracting function should be subordinate to and sup- port the program management and business functions. Ongoing program evaluations (see Recommendation 7) should provide guidance for the continuous improvement of program management, coordination, and communications.
CMS OVERSIGHT AND MANAGEMENT 129 Priorities Whereas the 7th and the 8th SOWs included a multiplicity of different tasks, subtasks, and performance measures and suffered from a lack of pri- orities, the 9th SOW should be clearly focused on quality improvement and related activities to support a national system for performance measure- ment and reporting. At the same time, it will be important to indicate pri- orities within that narrower focus. Future QIO contracts will need to be more coherent and should include clearly stated priorities for QIO activities and precise goals for the SOW overall. Current quality improvement tasks are subdivided and evaluated by provider setting; there needs to be a new priority on improving care across provider settings. As the single largest purchaser of health care in the United States, the Medicare program spent more than $295 billion in benefit pay- ments in 2004 to care for 41.7 million beneficiaries. Among the population over age 65, 84 percent have at least one chronic condition, and 62 percent have two or more such conditions. Transitions in care from one provider setting to another are particularly important for individuals who are chroni- cally ill. The growing need for chronic care among the Medicare population and the implementation of the Medicare Part D prescription drug benefit make a crosscutting perspective particularly important. For example, the continuity of drug therapies as a patient transitions from the hospital to his or her own home or a nursing home could be tracked as new data became available from the Part D benefit. Such tracking of the data would provide an opportunity for the QIOs to integrate the clinical aspects of care, as well as encourage multidisciplinary collaboration (Schulke, 2004). Cross-site coordination should build on the initial efforts made under the 8th SOW and should feature more prominently in future contracts. The need for quality improvement interventions that follow the patterns of pa- tient care delivered by a variety of providers in the community will increase as a national performance measurement and reporting system implements measures that cut across settings of care, monitor patients over time, and include composites from more than one provider. The QIOs will be chal- lenged to broaden their thinking about service delivery and to convene pro- viders from hospitals, nursing homes, home health agencies, and outpatient physician practices to work together on filling the gaps in care that occur when a patient is between settings, a gap through which too many patients now fall. The program should also help the health care system meet the challenge of transforming relationships between patients and clinicians, as well as within health care organizations. Many safety and quality problems, often resulting from poor documentation and communication, occur dur- ing the handoff of patients from one setting to another--a point illustrated
130 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM by beneficiary complaints, each case of which generally requires the review of more than one set of provider records (CMS, 2004). Provider Surveys It is difficult to plan nationally for the technical assistance projects that are needed and will be well received by providers without knowing basic information about providers' internal capacities for quality improvement and measurement work: what they have done in the past and are currently doing internally, what assistance they are purchasing externally, and what gaps remain with which the QIOs could be helpful. Gathering this informa- tion could be viewed as basic market research to identify potential clients and their needs. Provider surveys addressing the above questions might serve multiple purposes beyond a basic needs assessment for QIO assistance. For example, they could become part of an effort to track quality improvement trends and serve as a mechanism for identifying and describing best prac- tices. Although it is unlikely that such a survey could directly contribute to measurement of the impacts of the QIOs' efforts, it could provide a picture of the context within which the QIOs operate. In addition to surveys of each provider setting, it would be useful to conduct some focus groups that included a mix of providers to help identify needs for crosscutting quality improvement interventions. Knowledge Transfer and Local Creativity Given the strong need for evaluation of quality improvement interven- tions (as will be discussed further in Recommendation 7), it should be pos- sible to identify various improvement methods, successful projects, and fail- ures. QIOs should be encouraged to share their unsuccessful efforts and their best practices, and to transfer that knowledge broadly both within the QIO community and to other organizations, such as commercial health plans and provider organizations seeking to improve quality at the local and national levels. One type of incentive for knowledge transfer that might be offered to the QIOs is giving them permission to develop their own locally designed projects. In addition, similar incentives should be provided to encourage the publication of articles on quality improvement. For ex- ample, the evaluation formula in the contract for the 8th SOW provides a bonus point for QIOs that publish reports on their payment review projects (see Chapter 10). Alternatively, all QIOs performing at a high level or meet- ing interim goals could be rewarded for their performance on a local project. Group award fees are built into the contract for the 8th SOW (see Chapter 13) in an attempt to encourage QIOs to share and cooperate to- ward a mutual goal of improvement. The 8th SOW also allows four QIOs
CMS OVERSIGHT AND MANAGEMENT 131 to develop a local nursing home project instead of meeting the requirements delineated for the nursing home task (see Chapter 8). This incentive for top performers could also be made available for projects in other settings. If additional resources were not available to support local projects, one local project could be substituted for one of the required projects. In any case, QIOs must receive approval from CMS for such local projects, whose de- sign must include a thorough evaluation. Consideration should be given to coordinating the local projects of several states in such a way that they could be compared in a single evaluation; this might prove to be a useful testing ground for new approaches to quality improvement. QIO Contract Reforms When the next SOW is developed, all contracts should be put up for bid. Because the 9th SOW will be significantly different from the 8th SOW, one cannot assume that organizations currently holding QIO contracts will be best suited to carrying out the new functions. Other entities that meet structural, staffing, and conflict-of-interest qualifications should be allowed and encouraged to compete. Without the requirement to have a physician- sponsored or physician-access designation (see Recommendation 3), new entities might choose to respond to a request for proposals. Moreover, a SOW with a more limited range of functions focused on quality improve- ment might entice other entities to compete. The degree of competition generated might also reflect the amount of money expected to be paid for the services; other entities might not be interested in bidding on proposals if they did not expect to be adequately remunerated. With open competition for all state core contracts, more-successful contractors might bid on and win contracts to provide QIO services in more than one state, as happens now on a limited basis. Thus, more regionally based contractors might be able to offer services in several states, especially where doing so would be geographically efficient. (See Chapter 13 for further detail on the bidding process.) The QIO core contract should include rewards and penalties for con- tractors on the basis of their performance. With increased competition, CMS should expect bidders to propose rates of improvement they believe they can achieve on given measures above the minimums set by CMS, and should offer financial rewards for those contractors that reach their goals and im- pose penalties for those that do not. Doing so would provide stronger in- centives for QIOs to accelerate improvements. Under a contract providing incentives for high performance (instead of focusing on process management), QIOs would be allowed more flexibility in their intervention methods with less oversight of their daily activities. Although such flexibility was discouraged in the past because of the inabil-
132 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM ity to compare results across states, increased emphasis on formal evalua- tions of individual interventions (see Recommendation 7) would allow bet- ter comparisons, especially for determining the effectiveness of various qual- ity improvement techniques. Therefore, QIOs would be able to focus on achieving the desired result. This approach would also align the QIO pro- gram with the focus on defined quality improvement goals in the national system for performance measurement and reporting. Extension of the QIO contract period from 3 to 5 years would allow more time for providers to implement quality improvements and measure the results, as well as to receive feedback and adjust their quality improve- ment efforts accordingly. With a longer contract period, CMS should per- form more interim monitoring of the progress of the QIOs. CMS should also arrange the contract cycle so that all QIOs receive contracts for the same time period. QIO boards should be held accountable for the opera- tions of their organizations and should be involved in monitoring perfor- mance. Shifting all QIOs to the same time frame for contracting rather than continuing with the current situation of three rounds of contracting stag- gered over 6 months should streamline QIO evaluations and some other management functions, as well as improve CMS's ability to compare per- formance results among QIOs. Effective management planning would be necessary to process 53 contracts concurrently and to prevent other poten- tial work-flow bottlenecks. It would be necessary to plan monitoring func- tions carefully to ensure that they could be completed fairly with limited staff from the Regional Office. (See Chapter 13 for additional discussion of contracts.) Planning for the 9th SOW should move expeditiously to avoid the de- lays and uncertainties of the process for the 8th SOW. As discussed in Chap- ter 2, the release of the 8th SOW, expected in the late summer of 2004, was delayed until the spring of 2005, and significant changes were made as late as November 2005. Much of the delay appeared to be related to internal departmental debates and budgetary discussions. Because these discussions would need to occur only every 3 or 5 years if the QIO contracts were extended as proposed by the committee, it would be helpful for the pro- gram office to establish an ongoing dialogue with the key parties to keep them informed of current program progress, as well as CMS's thoughts about the next SOW. Explicit planning and drafting of each SOW should begin early enough to permit its timely release by CMS so that QIOs will have adequate time to prepare for the SOW, including program planning and staffing. Mindful of the time needed to prepare adequately for a new SOW and the fact that this report will be released after the start of the 8th SOW, the committee formulated most of its recommendations with a view toward the 9th SOW rather than recommending many hasty, major revi- sions to the 8th SOW.
CMS OVERSIGHT AND MANAGEMENT 133 QIO Support Centers and Other Contracts The intent of the QIO Support Centers (QIOSCs) is to serve QIOs as national technical resources, and QIOSCs are often expected to offer train- ing programs for QIO staff and prepare materials all QIOs can use to carry out their tasks. Therefore, the QIOSCs need a head start on each new SOW. In particular, it is important that they have materials ready for the QIOs when the new contract period begins. (See Chapter 7 for detail on QIOSC structure and Chapter 13 for discussion of QIOSC contracts.) This is a par- ticular challenge for QIOSCs that have not conducted special studies or served in this capacity in the past, and it could be a problem for experienced QIOSCs as well if the substance of the QIO tasks should change signifi- cantly. Having the next SOW well defined before the current contract is completed would allow CMS to contract for QIOSCs a few months before the commencement of the SOW. Doing so would become feasible if the QIOSC contract were opened to other entities with the requisite expertise. Currently, only QIOs may act as QIOSCs; therefore, QIOSC contracts can- not be awarded until the bidders know they have received a QIO core contract. The QIOSCs sometimes face competing demands to serve the QIOs that need assistance, as well as the Government Task Leader. The QIOSC contract should define in more detail, to the extent feasible, the specific tasks expected of the QIOSC for each audience and the measures to be used for evaluating its performance. CMS should consider including in the new SOW for the QIOs and QIOSCs the development of national resource teams. These teams would enable QIOs to work collaboratively with multistate providers to promote quality improvement by reducing the barriers encountered by corporate providers operating in more than one state. This task should be based on a thorough assessment of the Corporate Nursing Home Collaboratives, oper- ated through the Nursing Home QIOSC and the Colorado Foundation for Medical Care during the 7th SOW. This task coincides with the above- discussed need for CMS and the QIO program to consider more of the crosscutting issues of health care delivery. The Quality Improvement Group in CMS administers not only the QIO core contracts and QIOSC contracts, but also other contracts with QIOs and other organizations that are intended to support the QIO program and quality improvement in general, as defined more broadly within the Medi- care program. These other contracts would have greater synergistic value if better tracking and management systems were in place to oversee them and ensure their coordination with program priorities. (See Chapter 7 for more detail on contracts.) For example, if the QIOs had a list of all the special studies and the QIOs holding the contracts for those studies, they might use
134 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM this list as both an indicator of expertise on a topic and a guide to the topics under study. CMS should coordinate with other entities, such as the Agency for Healthcare Research and Quality, to facilitate ongoing discussions of research results. QIOSCs are one mechanism through which this knowl- edge transfer could readily occur. Management Relations The development of a new and substantially different SOW presents an opportunity for CMS to assess the different roles, responsibilities, and com- munications systems that will be needed for all QIOs to accomplish the new tasks. To ensure a smooth transition to the new SOW, all participants will need to understand CMS's priorities and policies, as well as those of a na- tional performance measurement and reporting system as it develops. Per- haps some of the lessons QIOs learn in working with providers to improve their communications across various health care settings could be applied to better link the many offices within CMS that are involved with process- ing QIO contracts and working with the contractors. Given the number of personnel involved in running the QIO program, CMS should seek to clarify the roles of all these individuals in light of the defined priorities of the SOW. (See Chapter 13 for further detail on CMS management.) QIO PROGRAM EVALUATIONS Recommendation 7: CMS should develop four types of evaluation to assess the QIO program. CMS should conduct three of these four types of evaluation internally to assess QIO performance against predetermined goals and priorities at the following levels: (1) the program as a whole, (2) individual QIOs with respect to the core contract, and (3) selected quality improvement interventions implemented by QIOs. DHHS should periodically commission the fourth type of evaluation--independent, external evaluations of the QIO program's overall contributions. · The QIOs should be learning organizations, continually improv- ing the assistance they offer to health care providers. CMS should develop explicit benchmarks for use in ongoing measurement of progress on the effectiveness and costs of the program. · CMS should form a technical expert panel to offer ongoing guid- ance on the design of the three types of internal CMS evalua- tions, including options for identifying optimally performing QIOs, as well as methodologies for attributing quality improve- ments to the QIO program's interventions.
CMS OVERSIGHT AND MANAGEMENT 135 · CMS should ensure that evaluations of the effectiveness of qual- ity improvement interventions are conducted. The committee suggests that CMS should use the most rigorous evaluation de- signs practicable, including randomized controlled trials. This approach should also contribute to CMS's overall program evaluation. Evaluations should include concurrent, qualitative descrip- tions and assessments of the nuanced nature of the QIOs' role in quality improvement interventions and the roles of other players. As appropriate, evaluations should be stratified among pro- vider settings and across states and regions. CMS should assess the cost-effectiveness of each type of inter- vention to assist with the allocation of resources. · The secretary of DHHS should allocate adequate funds from the QIO apportionment to carry out, on an ongoing basis, both in- ternal and external evaluations. Evaluation of Progress Toward Goals As discussed in Recommendation 6, CMS must set explicit overall goals and priorities for the QIO program, in part because goals are needed to create a robust evaluation plan for determining achievement. These goals should be considered in conjunction with those already established that also focus on enhancing health, such as those set for the public's health by Healthy People 2010 (HHS, 2000) and the National Healthcare Quality Report (AHRQ, 2004), when applicable. Evaluation of the program's suc- cess in achieving those goals, as well as its overall impact, has been largely ignored, and the focus has remained on evaluation of QIO contract perfor- mance alone. Although contract evaluation is needed, CMS must initiate plans to evaluate the QIO program as a whole and the effectiveness and impacts of individual intervention approaches. Levels of Evaluation Four levels of evaluation are important. The first is evaluation of the program as a whole to help prove its value and efficiency in the arena of health care quality improvement. As noted above, explicit goals against which the program can be evaluated must be set by CMS. This level of evaluation should be focused not only on determining the success of the program in improving quality overall, but also on such factors as CMS's oversight of the program's management and operations and the effective-
136 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM ness and value of the support services provided within the program. With- out such program-level evaluation, it is difficult to justify a specific appor- tionment and budget. Second, it is important for CMS to assess the performance of individual QIOs because under a performance-based contract, the QIOs are compen- sated for achieving specific results. It is necessary to understand which QIOs are excelling across all their quality improvement tasks, not just in a par- ticular provider setting. Thus evaluations must be used to determine the success of individual QIOs in achieving the results delineated in their con- tracts. The results of such assessments can help CMS judge the bids for future QIO contracts. These evaluations can also contribute to setting the bar for success. For example, if some QIOs are able to achieve much higher levels of improvement, it may be that other QIOs could achieve the same levels of improvement if encouraged to do so through knowledge transfer, goal setting, and other methods. To the extent appropriate, QIO boards should be involved in the monitoring and evaluation of their respective organizations. Third, evaluations of the QIOs are linked to the need for CMS to incor- porate requirements for rigorous intervention study designs into the QIO contract. For example, approval of local projects could be contingent on plans to compare the results with those of other QIO projects. With a vari- ety of QIOs carrying out different quality improvement interventions, CMS is uniquely positioned to determine which intervention methods lead to higher levels of quality improvement. The QIO program should be viewed as a learning enterprise that fosters constant improvement, innovation, and transfer of knowledge. Finally, periodic evaluations of the QIO program as a whole should be performed by an external entity to provide an independent point of view and valuable information for CMS on the systems design and operations management aspects of the program that need improvement. Such external evaluations should also address the overall impact of the program. CMS may choose to contract with others for assistance in all or a por- tion of its internal evaluations. When appropriate, CMS should collaborate with universities or others skilled in study design and evaluation to develop the strongest evaluative designs possible. CMS should also appoint and use an ongoing technical expert panel to provide guidance on the design of its internal evaluations and of future SOWs to enhance collection of the neces- sary data. This is not the first time the IOM has recommended an overall evaluation of the QIO program. In its 1990 report Quality Assurance in Medicare, the IOM mentioned the need to document the impact of the or- ganization in each state on the quality of care (IOM, 1990). In 1994, the IOM was asked to review the evaluation plan of the Health Care Financing
CMS OVERSIGHT AND MANAGEMENT 137 Administration (the predecessor of CMS) for the Peer Review Organization program (the predecessor of the QIO program). At that time, the IOM committee recommended both formative evaluations, to learn how to im- prove ongoing projects, and a summative evaluation of the whole program's achievements relative to its goals and objectives (IOM, 1994). The 1994 program evaluation was not conducted, although later almost two dozen quality-of-care measures were tracked for each state over time and were summed to obtain national measures (Jencks et al., 2003). The authors could not attribute the improvements in health care quality to the QIO program. The current IOM committee reiterates the ongoing need for evalu- ations at both the national program level and the state and project levels. While recognizing the challenges involved in conducting sound evalua- tions of quality improvements and their causes (see Chapter 9), the commit- tee is convinced that evaluations of the QIO program must be given high priority. Not only do the technical assistance methods used by QIOs with identified participants need to be examined, but the techniques they employ to improve quality statewide also need to be reviewed. Because the QIOs spent twice as much money on statewide efforts as on identified partici- pants during the 7th SOW, it is important to learn whether these funds were well spent and what was gained. Although studies showing how much more identified participants improved relative to other providers in the state are important in identifying certain impacts of the QIOs, they do not indi- cate whether statewide investments were productive. As noted above, QIOs should be charged as a part of their contract with proposing explicit plans for rigorous evaluations based on CMS guidance and methodological op- tions. These evaluations should be part of the QIOs' interventions and should allow for formal and complete assessment of the effectiveness of those interventions. To date, the QIOs have published a paucity of studies showing the effects of interventions on quality (see Chapter 9). Collecting data from well-designed studies of the quality improvement interventions initiated by each QIO can yield a compendium of information on the effectiveness of specific interventions in multiple states that would contribute greatly to the database on which interventions produce the best results. In addition, CMS and the QIOs should assess how different types of providers selected by various methods to participate in quality improvement projects affect the improvement achieved. In these ways, the program can help build an evi- dence base for determining the most cost-effective ways to improve popula- tion health. The dissemination of this knowledge should contribute to greater improvements in the future.
138 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Types of Study Designs Several types of studies may be considered or combined when evalua- tion plans are designed, and CMS should choose the strongest designs pos- sible. CMS should take advantage of the opportunity presented by the ex- istence of 53 QIOs, namely, a naturally occurring experiment with the potential to become a large observational study. The choice of study de- signs is not easy, and many complex decisions must be made. As stated above, CMS should consider the use of an ongoing technical expert panel to provide guidance on the design of evaluation plans for individual quality improvement interventions, as well as on the design of evaluations of the QIOs themselves. The methodologies that should be considered include the following: · Randomized controlled trials · Time series, crossover analyses · Studies with nonequivalent control groups · Case-control studies · Qualitative analyses Each of these types of design has advantages and disadvantages, which must be carefully considered before an evaluation is undertaken. One major challenge in implementing any study design concerns taking into account the voluntary nature of the program. The challenge is to separate the effects of an intervention from those due to differences in the providers seeking to participate in studies, which lead to selection bias (Campbell and Stanley, 1963; Wholey et al., 2004). Randomized controlled trials are particularly problematic for public programs such as the QIOs; nonetheless, their use should be considered as they may be suitable under specific limited circum- stances. (See Appendix C for descriptions of how each of the above meth- odologies might be applied to aspects of the QIO program when formal evaluation plans are designed.) QIO PROGRAM FUNDING Recommendation 8: Congress and the secretary of DHHS should focus all QIO resources on supporting health care providers' per- formance measurement and quality improvement efforts. The sec- retary should remove from QIO core contracts funds sufficient to support case reviews, appeals, and beneficiary complaints when those functions are devolved to other organizations. The secretary should increase the remaining funds to allow for inflation, the in- corporation of evaluations into all QIO work, the increased num-
CMS OVERSIGHT AND MANAGEMENT 139 bers of providers and beneficiaries being served, and the labor- intensive nature of technical assistance and quality improvement activities. · The multiple evaluations undertaken during the 8th and 9th SOWs should guide future funding decisions, with budget in- creases or decreases being provided according to the evaluation findings. If the evaluations demonstrate that no positive impact is attributable to the QIO program's efforts, CMS will need to rethink its quality improvement approach and the possible ben- efit of transitioning funds to an alternative structure and strat- egy for Medicare. · Once a national performance measurement and reporting sys- tem has been established, its priorities should help guide the funding levels and policy direction of the QIO program, recog- nizing that adequate funding is necessary to reach the goals set for the QIO program. · The secretary of DHHS should ease the conflict-of-interest re- striction with regard to supplementing the QIO quality improve- ment budgets with external funds. Given the limits of federal funding, the QIOs should be allowed to seek funds for quality improvement activities from providers and other organizations as appropriate. The tasks confronting the QIOs during the 8th SOW are of great mag- nitude, and as discussed in Chapter 4, the committee expects that provider demand for technical assistance with quality improvement will increase. The committee recognizes the huge gap between what is known about qual- ity care and the care that is delivered to most patients. It also understands that many health care providers and practitioners need help in making the changes necessary for the consistent delivery of high-quality care. The com- mittee recognizes as well that although the QIOs should no longer be re- sponsible for case reviews, complaints, and appeals, CMS will need to con- tract with other organizations to perform at least some of those functions. Thus, a portion of the current funding should be shifted out of the QIOs' budgets after the 8th SOW. During the 9th SOW, additional funds will be necessary to cover the increased demand for technical assistance from pro- viders and extensive program evaluations. (See Chapter 7 for further dis- cussion of funding.) Before increased funding can be justified past the 9th SOW, however, it is necessary to assess the impacts the QIOs are having on providers at present. It would be unrealistic to expect CMS and the QIO program to perform evaluations with levels of rigor rivaling those of double-blind, ran-
140 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM domized controlled drug trials, but certainly more could be learned about the impact of the program if evaluation were a priority. Decisions on how future funds are to be spent should be based on the results of all evaluations of the overall impact of the QIO program, the technical assistance provided for performance measurement, the specific quality improvement methods and techniques used, and which QIOs are most successful. If those evalua- tions indicate that the QIO program is having a significant positive impact and if there is continuing demand for technical assistance for quality im- provement or new tasks are added, future funding should be increased. Information resulting from the evaluation is essential as a guide to the wise expenditure of additional funds. If the evaluations show that the program has an insufficient impact, CMS must consider other strategies for improv- ing the quality of health care and should shift funding accordingly. At the same time, however, it is important to recognize that adequate funding in proportion to contract requirements is necessary to accomplish the goals established for the program. In 1990, the IOM determined that investment in the QIO program was inadequate to achieve set goals (IOM, 1990). Since then, the program has added more task areas, while funding has become a smaller percentage of the overall Medicare budget (see Chap- ter 7). Additionally, while most of the QIOs' current expenditures for Tasks 1a1d (quality improvement activities in nursing homes, home health set- tings, hospitals, and physician offices) go toward overall statewide activi- ties, work with identified participants often incurs a higher cost per pro- vider (see Chapter 7). For example, under the 7th SOW, QIOs worked with approximately 7.5 percent of physician offices. If demand for help increases in this provider setting, budget increases will be necessary because of the greater cost of such activities per provider in this setting (as compared with overall work with providers). At present, however, it is infeasible to esti- mate the budget requirements for these activities in the 9th and 10th SOWs because of a variety of unknown factors, including the scope of the in- creased demand, the effects of pay-for-performance and public reporting initiatives, and the possible development of more cost-effective tools for use by QIOs in reaching out to a larger number of providers. Additional funds will also be needed to enhance the QIO program's role in supporting the implementation of a national performance measure- ment and reporting system and the capacity of providers to participate in the reporting of quality measures and pay-for-performance programs. Once a national performance measurement and reporting system is functioning, has established goals, and has identified additional measures for reporting, its priorities should guide the policy direction of the QIO program. None- theless, CMS and the QIO program also have responsibilities to the Medi- care program. To the extent possible, efforts should be made to bring those
CMS OVERSIGHT AND MANAGEMENT 141 responsibilities into alignment with the operation of the national perfor- mance measurement and reporting system. Conflict-of-interest rules currently restrict QIOs from soliciting or re- ceiving external funding from providers or grants for the provision of the same services rendered under the QIO contract (see Chapter 7). Many QIOs have already exhibited an ability to contract with non-Medicare sources for quality improvement and case review functions. These contracts are unre- lated to their core contract and state providers. However, some local pro- viders might want to contract with their local QIO for services similar to those provided by the QIO as part of the Medicare program. For example, a hospital might want to pay the QIO for extra or more intensive assistance in its internal quality improvement efforts to improve upon measures ex- plicitly delineated in the QIO contract. In the past, the restriction on such arrangements prevented a conflict for QIOs, which would be accepting money from the same providers they might also review for any number of regulatory activities. If the QIOs' efforts were focused on quality improve- ment and performance measurement (see Recommendations 3 and 4), that inherent conflict would no longer exist. Additionally, increased interaction with the private sector could boost the QIOs' reputations, add to their skill sets, and enhance publicprivate collaboration. Nonetheless, CMS should ensure that the proportion of funds from providers and other private sources is not so large relative to the Medicare core contract that it gives the impres- sion of a conflict of interest or impropriety. Also, QIOs need to prevent perceptions of bias toward paying contractors that might arise if resources for free assistance became scarce. Finally, by removing this restriction, more organizations with multiple lines of business might be enticed to compete for QIO contracts. REFERENCES AHRQ (Agency for Healthcare Research and Quality). 2004. National Healthcare Quality Report. Rockville, Maryland: Agency for Healthcare Research and Quality. Campbell DT, Stanley JC. 1963. Experimental and Quasi-Experimental Designs for Research. Chicago: Rand McNally & Co. CHCF (California Health Care Foundation). 2005. Executive Summary. In National Con- sumer Health Privacy Survey 2005. Bishop L, Holmes BJ, Kelly CM, eds. Oakland, CA: California Health Care Foundation. CMS (Centers for Medicare and Medicaid Services). 2004. The Quality Improvement Organi- zation Program: CMS Briefing for IOM Staff. [Online]. Available: http://www. medqic.org/dcs/ContentServer?cid=1105558772835&pagename=Medqic%2FMQ GeneralPage%2FGeneralPageTemplate&c=MQGeneralPage [accessed December 26, 2005]. CMS. 2005. Quality Improvement Roadmap. [Online]. Available: http://www.medical devices.org/public/issues/documents/CMSMedicareroadmap.pdf [accessed December 26, 2005].
142 MEDICARE'S QUALITY IMPROVEMENT ORGANIZATION PROGRAM Gaul GM. 2005, July 26. Once health regulators, now partners. Washington Post. p. A1. HHS (United States Department of Health and Human Services). 2000. Healthy People 2010: Understanding and Improving Health. Washington, D.C.: U.S. Government Printing Office. Hibbard JH, Stockard J, Tusler M. 2005. Hospital performance reports: Impact on quality, market share, and reputation. Health Affairs 24(4):11501160. IOM (Institute of Medicine). 1990. Quality Assurance in Medicare. Washington, DC: Na- tional Academy Press. IOM. 1994. An Assessment of the HCFA Evaluation Plan for the Medicare Peer Review Orga- nization. Washington, DC: National Academy Press. IOM. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. Jencks SF, Huff ED, Cuerdon T. 2003. Change in the quality of care delivered to Medicare beneficiaries, 19981999 to 20002001. Journal of the American Medical Association 289(3):305312. McClellan MB. 2005, July 21. Statement of Mark McClellan, M.D., Ph.D., Administrator, Centers for Medicare and Medicaid Services, U.S. Department of Health and Human Services. Testimony before the Subcommittee on Health of the House Committee on Ways and Means. Washington, DC: U.S. House of Representatives. Rollow WC. 2005. QIO Program: Update and Policy Considerations. PowerPoint Presenta- tion to the Committee on Redesigning Health Insurance, June 13, Washington, DC. Schulke D. 2004. Opportunities for Quality Improvement Created by the Medicare Drug Benefit. PowerPoint Presentation to the Committee on Redesigning Health Insurance, December 13, Washington, DC. Wholey JS, Hatry HP, Newcomer KE. 2004. Handbook of Practical Program Evaluation. San Francisco: John Wiley & Sons, Inc.