comes of providers in their community. Here, Dr. Hannan noted that the critical issue is the source of data, not the method of risk adjustment. Audited clinical data are needed, and administrative data are inadequate. Dr. Chassin suggested that an important consideration for providers is inclusion in the clinical database of risk factors that they view as important to outcomes. Without these components, providers will not have confidence in the performance reports.
Optimally, an information system used as part of a quality improvement program would include data on both processes of care and outcomes so that providers would know what sort of corrective actions were needed to improve outcomes. Process measures are usually lacking, however, and providers receive performance data in the form of risk-adjusted outcomes. Providers thus lack a roadmap for what to do. Dr. Chassin reported that experience in New York suggests that low-ranking hospitals have used very different strategies to improve performance (e.g., changes in personnel, processes, organization). In some regions, specialty providers have conferred with one another using a “reverse-engineering ” approach to help identify differences in performance that account for outcome differences (e.g., the New England cardiovascular project).
Dr. Berwick pointed out that if quality problems stem from a process error, the steps necessary to correct the problem are relatively straightforward. If, however, poor quality stems from poor underlying skills, it may be difficult to enhance a provider's skills, although professional development training or a mentoring process could be used. Mentoring approaches have been effective in the adoption of new technology, where providers mentored by experienced practitioners helped novices move up the “learning curve.” Simulation models would also be helpful, although relatively few examples of this approach are available.
In concluding the morning session, Dr. Chassin observed that evidence suggests that poor performers could do much better, but it is unlikely that all providers will ever achieve the highest level of performance.
WHAT ARE THE POLICY IMPLICATIONS OF A VOLUME–OUTCOME RELATIONSHIP?
Dr. Dudley and colleagues from the University of California, San Francisco, in their synthesis of the literature relevant to policy issues (Appendix D), put the volume-outcome relationship and its implications into the broader context of quality improvement. Dr. Dudley outlined the two general approaches to quality improvement:
Have providers improve their quality (including correcting overuse, underuse, and misuse).
Have patients switch to higher-quality providers.
The authors listed a number of specific strategies to improve quality of care that could incorporate considerations of volume:
regulatory strategies (e.g., CON, regionalization);
improving the skills of health professionals (e.g., physician education); and
competition-based strategies: consumer-oriented approaches (e.g., report cards) and purchaser initiatives (e.g., selective referral, quality bonuses).
The competition-based strategies depend upon public disclosure of data to consumers and/or groups that purchase health insurance (e.g., employers). Given that the volume-outcome
relationship holds on average, Dr. Dudley pointed out that most individuals could benefit from this information. On a case-by-case basis, however, the information will not have its intended consequences for a significant share of patients. Dr. Hannan provided two examples of the risk of making referrals purely on the basis of volume. He noted that in New York, one-third of high-volume surgeons (i.e., those performing at least 150 procedures per year, or 450 procedures over 3 years) 5 had higher-than-average risk-adjusted mortality rates. Also among the 18 hospitals performing very high volumes of angioplasty (i.e., at least 400 procedures per year),6 8 hospitals had above average risk-adjusted mortality rates. As these examples illustrate, a relatively large share of patients would be cared for by providers with inferior performance if referrals were made solely on the basis of volume.
Dr. Dudley noted that providing consumers with information to support decision-making has been somewhat disappointing. Report cards can be difficult for consumers to understand and, where they have been published, appear to have been used by only a small percentage of consumers. New formats and communication tools are needed to improve their use, and Dr. Dudley suggested that a Consumer Reports model be tested for disseminating information. Consumer Reports represents a trusted, unbiased source of information that provides a wealth of detailed information, as well as easy-to-use summary measures with which to judge products. According to Dr. Dudley, the inclusion of information on functional status, long-term outcomes, and common chronic conditions (e.g., those treated in outpatient settings) would likely increase the relevance of such reports to consumers.
Large purchasers are beginning to hold systems of care accountable for quality improvement. The Pacific Business Group on Health (PBGH), a large purchasing coalition, provides condition-specific volume data for area hospitals on its website (www.healthscope.org) along with guidance on how to interpret the data. PBGH also requires health plans to ensure that patients with selected conditions go to high-volume hospitals. This approach is also being taken by the Leapfrog Group, a newly formed organization comprising many large employers and purchasing coalitions (including PBGH). The Leapfrog Group has developed a set of health plan performance standards that include volume standards for specific conditions, but it will recommend better measures when they become available
Information about volume can be applied without public disclosure, for example, within systems of care for quality improvement programs. Low-volume providers may withdraw voluntarily to avoid scrutiny or be motivated to examine and improve internal structures or processes of care.
Dr. Dudley and colleagues described several potential barriers to implementing a selective referral program based on a volume standard:
a potential for decrements in quality at higher volumes;
patients' preferences for care close to home;
patients' lack of resources to travel to hospitals that are far away;
patients who need immediate treatment or are too unstable to transfer;
loss of access in areas where low-volume services have been closed (e.g., cardiac surgery);
resistance from physicians and hospitals to cooperate in quality monitoring efforts; and
According to these criteria, 30 percent of cardiac surgeons in New York are high-volume providers.
The American College of Cardiology recommends that hospitals perform at least 200 angioplasty procedures per year.
effects on marketplace structure and competition:
increased market power of high-volume hospitals (e.g., prices could rise),
barriers to entry of new competitors (e.g., it is difficult to start at high volume), and
potential for medically inappropriate admissions to boost volumes to meet cutoffs.
The following discussants responded to Dr. Dudley's remarks:
Consumer perspectives—Ellen Stovall, National Coalition of Cancer Survivorship and National Cancer Policy Board; and Art Levin, Center for Medical Consumers
Purchaser perspectives—Bruce Bradley, General Motors; and Stephen Clauser, Health Care Financing Administration
Managed care perspective—George Isham, HealthPartners; and Sam Ho, Pacificare Health Systems
Provider perspectives—Don Nielsen, American Hospital Association
Research perspectives—Irene Fraser, Agency for Healthcare Research and Quality
Ms. Stovall, as a cancer care consumer, raised concerns about institutions ' advertising on the basis of volume without other quality data or explanations of the meaning of volume and without descriptions of how the information should be interpreted.
Mr. Levin described how the New York State Department of Health has taken risk-adjusted mortality data on cardiovascular procedures in New York to produce physician- and hospital-specific reports for consumers. Mr. Levin questioned the relevance of these data to contemporary practice given that as of May 2000, mortality data were limited to the years 1994 to 1997. Volume data provided by the Center for Medical Consumers are more current—1998 hospital- and physician-specific volume data were posted on his organization's website (www.medicalconsumers.org). Mr. Levin noted that efforts to disseminate quality data to the public in New York have met with fierce resistance from physicians (e.g., legal challenges).
Ms. Stovall emphasized the importance of full disclosure with annotation to allow interpretation and informed decision-making. Her organization, the National Coalition of Cancer Survivors, is working with providers in the spirit of offering “carrots” rather than “sticks.” They also are working with large purchasers on patient-friendly displays of information. Ms. Stovall discussed the implications for consumers of not having good evidence regarding medical care. In the absence of evidence on the benefits of high-dose chemotherapy and bone marrow transplantation, providers offered and consumers demanded access to the procedure. Subsequent clinical trial data suggest that the procedure offers no substantial benefit over conventional therapies in most situations. Consumers often make choices based on the trust they have in their doctor and, increasingly, based on advice from consumer groups.
Mr. Bradley described employers' urgency to ensure that employees obtain optimal care and the ethical, legal, and fiduciary responsibility that companies have to ensure that their beneficiaries receive high-quality care. A group of employers has organized the Leapfrog Group to
apply the best available measures in purchasing decisions. At present, volume measures are felt to be ready and may serve to “stir the pot ” to facilitate quality improvement. When provider specific volume data are publicly disclosed, some concerns have arisen regarding liability, especially if the inference is made that low volume reflects poorer outcomes.
Dr. Clauser suggested that his agency, the Health Care Financing Administration (HCFA), could use volume data to target interventions of its peer review organizations (PROs), which operate at the state level, to ensure the quality of care for Medicare beneficiaries. More useful, however, would be information on processes of care that underlie the relationship to affect outcomes. Also, establishing the relationship of volume of services to outcomes of particular relevance to the Medicare population, such as functional status, would be valuable. In Dr. Clauser's view, research is also needed to establish whether the volume–outcome relationship extends to nonsurgical interventions and to aspects of chronic care. HCFA is funding research to better understand the underpinnings of the volume–outcome relationship. A 13-state prospective outcome assessment study examining carotid endarterectomy and a number of PRO quality improvement initiatives in surgery safety and outcomes may guide Medicare decisions.
Dr. Clauser cautioned that volume-based selective referral may run counter to patient preferences and could narrow the choice of providers, a quality highly valued by consumers. In a dynamic market, it may also be difficult to apply volume thresholds. As part of HCFA's transplantation certification program, volume is one criterion, but volume can change quickly when, for example, a hospital's transplant team moves to another facility. Dr. Clauser pointed out that a monitoring program to keep track of changes in the marketplace requires an infrastructure that is expensive to maintain.
Dr. Clauser suggested that data about volume alone may be insufficient to shift care to higher-volume providers. The HCFA experience with centers of excellence for coronary artery bypass surgery suggests that it is very difficult to change physician referral patterns. When centers were able to advertise on the basis of their designation as a center of excellence, there was very little impact on referrals. Dr. Clauser concluded with a request that the research community provide answers to what volume means so that consumers can interpret the information that is increasingly being made available to them.
Managed Care Perspective
Dr. Isham pointed out the potential value of examining the relationship between volume, outcome, and price of procedures performed outside the usual reimbursement system, using a case study approach. There could, for example, be lessons relevant to cataract surgery in the recent surge in outpatient elective laser eye surgery. This procedure has expanded rapidly through high-volume centers' offering the procedure at relatively low cost. Dr. Isham suggested that physicians and hospitals could be required to maintain a logbook of their experience, similar to the logs that pilots are required to keep. Such a log could be made publicly available and could also serve as a tool for self-evaluation or as the basis of certification or accreditation. Dr. Isham raised concerns regarding the emergence of provider cartels in some geographic areas and also about practical applications at the market level (e.g., implications for rural areas).
Dr. Ho recounted experience at Pacificare Health Systems suggesting that disclosure to consumers of information on quality can affect consumer choice, improve performance, and reduce practice variation. Pacificare will consider integrating information about volume into its members' report card program. Twice a year, medical groups are ranked and findings are posted on the Internet (www.pacificare.com) and in the provider directory sent to each member. The
reports show the top-ranking groups (i.e., the top 10 percent) according to their performance in 32 categories related to clinical care, service, and administrative functions. The breadth of the rankings responds to the observation that consumers use different information in choosing providers. Public reporting has been associated with significant net increases in membership within the highly ranked groups and a reduction in variation in provider performance.
In considering the volume–outcome relationship as it applies to quality improvement, Dr. Ho expressed some concerns about unintended consequences of its application:
The focus on high-volume providers may prove a distracting priority. Improvements might be more readily achieved through reducing fragmentation of services, better organization of systems of care, focus on prevention, the institution of standardized performance metrics, and an “upstream, ” population-based focus on health and disease management.
Hospitals with a high volume of a certain procedure may use that as “brand leverage” to misrepresent their general experience.
High-volume hospitals may achieve too much contractual leverage, giving rise to “cartels” and price inflation.
More may not always be better: could there be a quality decrement with very high volume? Might inappropriate utilization be stimulated by volume-based rewards?
Procedures counting toward high volume may not always be indicated or appropriate.
More than a consensus on volume is needed. There could be a significant provider and/or consumer backlash if selective referral is widely implemented, unless all industry stakeholders demonstrate agreement.
Provider Group Perspective
Dr. Nielsen described the overarching goals of the health care system —improving health status, access to care, and the coordination and continuity of care—and suggested that information that furthers these goals needs to be disseminated to all stakeholders in a form that is accurate, reliable, and easily understood. In his view, volume as a measure of quality is a “bluntedged tool.” Given the limited state of research in this area, he pointed out that mortality may be improved with volume, but this is not necessarily true for other important outcomes of care (e.g., functional status, quality of life). In addition, higher volumes do not necessarily mean that there are decreases in associated morbidities such as infections or that certain critical steps in the process of care are utilized. There is the potential for misuse of such information, for example, misrepresentation in advertising. Furthermore, perverse incentives to do more procedures may well be created with the establishment of volume thresholds that could contribute to inappropriate care being rendered. There is also a potential for consumers to misinterpret the information and make decisions about care that are unwarranted (e.g., traveling great distances to tertiary care centers for simple procedures). Dr. Nielsen concluded that volume should not be used alone but should instead be supplemented with ancillary information and disclosed with all of its caveats.
Dr. Fraser from the Agency for Healthcare Research and Quality noted that much of the research conducted to date has examined the clinical processes of care that underlie the volume– outcome relationship, but it has not yet addressed how systems of care might mediate this relationship. A study of the relationship between volume and outcome in acute myocardial infarction, for
example, showed that the underuse of beta-blockers in low-volume facilities helped explain the significant volume–outcome finding. Knowing what systems of care are associated with the appropriate use of beta-blockers in high-volume hospitals is critical to implementing this finding. How the volume–outcome relationship varies by community or health care market, and how the relationship might vary within hospital and by procedure, has pragmatic implications.
AHRQ's Healthcare Cost and Utilization Project (HCUP) is a potential resource for conducting studies of both hospital and physician volume. The HCUP State Inpatient Database includes more than half of all U.S. community hospital discharges (discharges from 22 states). A set of risk-adjusted quality indicators has been developed that could be used as benchmarks for health services researchers. Procedure-specific data sets could be produced for researchers with a specialty interest. Market-specific data, even if qualitative, would be of interest to purchasers to assist in their decision-making. Exceptions to the expected relationship—for example, high-volume, low-quality hospitals, could be explained by a low pricing strategy adopted by a particular hospital. Dr. Fraser suggests that research is also needed on the process and outcome of the purchasing strategy itself—its consequences on the quality of care and its market impact. For implementation, we need to know what kinds of data are most useful to purchasers and how employees can best use this information.
Dr. Fraser also pointed out that consumer use of information about volume of services is limited by the fact that it is usually employers, not employees, who choose a health plan. Most employers offer only one or two plans to their employees, and a minority offers plans that provide a great deal of choice at the point of care.
Dr. Berwick led off the discussion by asking whether the potential for unintended consequences of public disclosure of volume–outcome data was sufficient to delay or limit public disclosure.
Dr. Simone pointed out that consumers are bombarded with medical information, much of it through advertising. He suggested that consumers need explanation but that it cannot be too complicated.
Dr. Chassin suggested that there are different thresholds for different actions. In his view, data on volume should be publicly available with appropriate explanation. All states have some kind of hospital discharge database, and probably two-thirds of states have publicly available hospital data. Purchasers could take this information and use it for decision-making and/or could make the information public. New York, through its CON process, has utilized evidence on volume to achieve regionalized cardiac surgery. New York has 32 hospitals offering cardiac surgery, while California has 125 such programs. The net effect is that close to 80 percent of patients having bypass surgery in New York obtain it in hospitals doing more than 500 procedures a year. In contrast, less than one-half of patients in California undergo surgery in such high-volume settings. The New York regulatory process exists alongside a voluntary program, the cardiac surgery reporting system. With the adoption of these programs, there has been a significant decline in the mortality associated with cardiac surgery in New York, one that exceeds the general decline across the country. There is no evidence that the procedures are overused. In fact, estimates of rates of overuse of angioplasty, angiography, and bypass surgery are very low—4 percent for percutaneous cardiac angioplasy and 2 percent for bypass surgery. Furthermore, estimates of rates of underuse are similar to those observed in California. Lastly, access to these procedures in New York and California is similar—patients need not travel farther for high-volume care. The combined regulatory and state-sponsored voluntary program works, but difficult political battles had to be won before it could be implemented
Ms. DelBanco, from the Leapfrog Group, felt optimistic that its evidence-based outcome referral program would motivate change. Dr. Hillner added that data systems tend to get better with use (e.g., Health Plan Employer Data and Information Set).
Dr. Hannan again stated the limitations of making inferences about the performance of individual providers with administrative data (e.g., statewide hospital discharge data). Although these data can be used to say that, in the aggregate, outcomes for certain procedures are worse in low-volume hospitals, the performance of individual small-volume providers cannot be predicted with confidence because of the instability of small numbers (e.g., a few deaths can greatly skew annual rates). Similar problems arise when evaluating rare conditions or procedures (e.g., surgery for pancreatic cancer). Dr. Hannan recommends against using volume data alone to judge the performance of individual providers. Instead, volume data, if used, should be coupled with risk-adjusted outcome data from administrative databases or used in concert with crude mortality rates. The use of a combination of sources is preferable to using any nonclinical source alone. This will minimize referrals to the numerous high-volume hospitals with poor outcomes. He also pointed out that referrals are generally made to physicians practicing within hospitals, and ideally, one would have quality data at both levels of care.
Dr. Hannan suggests that the most important use of volume data is to identify processes and structures of care that distinguish high-and low-volume providers and that predict outcomes. Findings can then be used to enhance the performance of providers. In an examination of the surgeon volume–outcome relationship among patients undergoing carotid endarterectomy in New York State, volume effects disappeared when type of surgeon was controlled for in the analysis (i.e., vascular surgeons had lower risk-adjusted mortality rates than either general surgeons or neurosurgeons). Furthermore, the use of more appropriate medications and surgical techniques by vascular surgeons explained their mortality advantage. This study illustrates that volume, which may appear to be an important driver of quality, may not be significant once processes and structures of care are taken into account. The ability to tease out the relative contributions of volume, process, and structure of care were possible in this study only because comprehensive data are systematically collected in New York as part of a carotid endarterectomy registry.
For procedures performed infrequently, Dr. Dudley pointed out that it is very difficult to measure quality, so proxies such as volume may have to do. For some conditions with low overall volume (e.g., esophageal cancer), he suggests that it may be impossible to refer selectively based on direct measurement of quality (i.e., physician or hospital-specific outcome data). Here, one could base selective referral on minimum volume standards. For conditions in which high volume has been shown to improve outcome and for which case loads are large enough to support outcomes measurement (e.g., CABG), it is also feasible to base selective referral on outcome, rather than volume.
Dr. Hertzer, a vascular surgeon at the Cleveland Clinic, suggested that prospective outcome assessment appears to be successful in improving physician performance. In an HCFA-sponsored demonstration of carotid endarterectomy, at least some participating areas found improvements in processes of care and outcomes associated with such an assessment.
Dr. Simone discussed the pediatric oncology model as a potential way to reorganize care to achieve optimal outcomes. Most children with cancer are enrolled in clinical trials, irrespective of the site of care. They are cared for in community settings, but according to research protocols that guarantee the use of best practices. All pediatric oncologists contribute outcome data for assessment. Mentoring relationships exist between high- and low-volume hospitals.