New York, for example, facilities must be granted certificates of need before they can offer certain cardiovascular procedures. In part, certificates are granted on the basis of volume. States regionalize some types of care, when evidence shows that such organization improves outcomes. Trauma and neonatal intensive care are, for example, regionalized to ensure that complex cases are cared for in institutions with the facilities and personnel to manage them. Regulators could also require that volume data be reported and, in turn, made available to consumers.

Accrediting Organizations

Organizations such as the Joint Commission on Accreditation of Healthcare Organizations could consider volume in accrediting hospitals and other health care institutions. The National Committee for Quality Assurance could evaluate health plans based on their application of volume information (e.g., to inform enrollees, move care to higher-volume providers).

Professional Organizations

Physician specialty boards could certify providers, taking into consideration their experience with a certain procedure. The American Board of Internal Medicine, for example, considers experience and volume when certifying internists to provide electrophysiology services.

Medical Educators

Knowing how much experience (or volume) is needed to develop expertise could guide curriculum development and the appropriate allocation of training time for certain procedures. In the context of continuing education, research on the volume–outcome relationship could provide information on how to sustain skills or acquire new ones. Methods to train providers without putting patients at risk are needed. In surgery, animals have often been used to help providers acquire new skills, but it is likely that computer-based simulation models commonly used in the airline industry to train pilots will be forthcoming in medicine. Dr. Eisenberg noted that research will be needed to ensure that simulated volume is related to favorable outcomes.

HOW IS VOLUME RELATED TO QUALITY IN HEALTH CARE?

To assess how volume is related to quality in health care, Drs. Ethan Halm, Clara Lee, and Mark Chassin, of the Mount Sinai School of Medicine, conducted a systematic review of the published research literature (Appendix C). As part of their cross-disciplinary review, they examined evidence from 88 studies concerning eight conditions and procedures:3

  • coronary artery bypass graft (CABG) surgery,

  • pediatric cardiac surgery,

  • carotid endarterectomy,

  • abdominal aortic aneurysm repair,

3  

Inclusion criteria for the review included publication since 1980, analysis of population-based sample, volume as an independent variable, and inclusion of health outcome as the depedent variable. Excluded were studies of care that has been regionalized or is highly regulated (i.e., trauma care, newborn intensive care, organ transplantation) and conditions or procedures for which there were very few articles (e.g., knee replacement surgery).



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 4
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary New York, for example, facilities must be granted certificates of need before they can offer certain cardiovascular procedures. In part, certificates are granted on the basis of volume. States regionalize some types of care, when evidence shows that such organization improves outcomes. Trauma and neonatal intensive care are, for example, regionalized to ensure that complex cases are cared for in institutions with the facilities and personnel to manage them. Regulators could also require that volume data be reported and, in turn, made available to consumers. Accrediting Organizations Organizations such as the Joint Commission on Accreditation of Healthcare Organizations could consider volume in accrediting hospitals and other health care institutions. The National Committee for Quality Assurance could evaluate health plans based on their application of volume information (e.g., to inform enrollees, move care to higher-volume providers). Professional Organizations Physician specialty boards could certify providers, taking into consideration their experience with a certain procedure. The American Board of Internal Medicine, for example, considers experience and volume when certifying internists to provide electrophysiology services. Medical Educators Knowing how much experience (or volume) is needed to develop expertise could guide curriculum development and the appropriate allocation of training time for certain procedures. In the context of continuing education, research on the volume–outcome relationship could provide information on how to sustain skills or acquire new ones. Methods to train providers without putting patients at risk are needed. In surgery, animals have often been used to help providers acquire new skills, but it is likely that computer-based simulation models commonly used in the airline industry to train pilots will be forthcoming in medicine. Dr. Eisenberg noted that research will be needed to ensure that simulated volume is related to favorable outcomes. HOW IS VOLUME RELATED TO QUALITY IN HEALTH CARE? To assess how volume is related to quality in health care, Drs. Ethan Halm, Clara Lee, and Mark Chassin, of the Mount Sinai School of Medicine, conducted a systematic review of the published research literature (Appendix C). As part of their cross-disciplinary review, they examined evidence from 88 studies concerning eight conditions and procedures:3 coronary artery bypass graft (CABG) surgery, pediatric cardiac surgery, carotid endarterectomy, abdominal aortic aneurysm repair, 3   Inclusion criteria for the review included publication since 1980, analysis of population-based sample, volume as an independent variable, and inclusion of health outcome as the depedent variable. Excluded were studies of care that has been regionalized or is highly regulated (i.e., trauma care, newborn intensive care, organ transplantation) and conditions or procedures for which there were very few articles (e.g., knee replacement surgery).

OCR for page 4
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary cancer surgery, percutaneous transluminal coronary angioplasty (PTCA), acute myocardial infarction, and acquired immunodeficiency syndrome (AIDS). Their review focused on the nature of the relationship between volume and outcome, the causal pathways that might explain the relationship, and whether the research findings might apply to other clinical areas. Historically, the volume–outcome relationship has been explained by “practice makes perfect” or patient selection (i.e., “busy doctors see healthier patients”). The authors propose a conceptual model that identifies a series of factors that might underlie the relationship between volume and quality of care (Figure 1). The authors conclude that for a wide variety of surgical procedures and medical conditions, higher volume (whether assessed by hospital or by physician) is associated with better health outcomes. The uniformity with which the published research documents or confirms the existence of this association is compelling (although the review authors caution that a publication bias against negative findings cannot be ruled out): Statistically significant associations between higher volume and better outcomes were found in 79 percent of the studies of hospital volume and 77 percent of the studies of physician volume. None of the studies reviewed showed a negative effect of volume. A significant volume effect was observed in all 16 of the studies judged to have the soundest research methods. A higher-volume, better-outcome association was observed in three-quarters of the studies FIGURE 1 Conceptual Framework: How Could Volume Affect Quality? reviewed. There is, however, great variation in outcomes, especially for low-volume providers. Some low-volume providers have excellent outcomes, and conversely, some high-volume providers have very poor outcomes. Furthermore, the performance gap between low- and highvolume hospitals appears to narrow with time as procedures become well established (e.g., the

OCR for page 4
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary volume of some cardiovascular procedures has tripled in less than a decade). Consequently, volume may be most pertinent when a new technology is beginning to diffuse into general practice. Volume per se does not lead to good outcomes in health care. It is a proxy measure for other factors that affect care. In one of the studies, about one-third of the survival benefit among patients with acute myocardial infarction that was attributed to high-volume hospitals could be explained by the use of appropriate processes of care, for example, use of beta-blockers, aspirin, thrombolytics, and revascularization procedures. Unfortunately, most of the studies reviewed did not include information on processes or systems of care that might explain the underlying reasons for the volume–outcome relationship. It is difficult to summarize the findings by procedure or condition because of study differences in the definition of low- and high-volume, in analytic techniques, and in adjustment methodologies. Even so, the volume–outcome relationship appears to be particularly dramatic for certain low-frequency, high-risk, surgical procedures such as surgery for cancer of the pancreas and esophagus. For these procedures, rates of short-term mortality are 2 to 3 times greater in low- versus high-volume hospitals. For other procedures or conditions under review, the volume effect was not as great or as consistent (see Appendix C for procedure- and condition-specific findings). Statistically significant differences in outcomes by volume may not always be of sufficient magnitude to be clinically significant; furthermore, other considerations can mitigate the importance of such differences. For example, if the difference between high- and low-volume hospitals' surgical mortality rates was only 1 or 2 percentage points, and the underlying condition for which the surgery was performed had a poor long-term prognosis, it may not be prudent to implement policy based only on a statistically significant difference in short-term mortality. The authors caution that they may not have identified all relevant studies of the relationship between volume and outcome and that many studies, particularly those involving cardiovascular procedures, were conducted in New York and thus may not generalize to states with less regulation of such procedures. Negative publication bias (researchers' failing to report the absence of a relationship between volume and outcome or journals' failing to publish negative results) could also overestimate the positive nature of the volume–outcome relationship. Drs. Halm and colleagues judged the methodological strength of the research using an explicit quality scoring method4 and concluded that the methodological rigor of the studies was modest, limiting generalizability and leaving many unanswered questions about the nature and causes of the observed association: It is unclear whether volume thresholds exist above which outcomes are better. For example, is it only the busiest providers that have the best outcomes, or do all but the lowest-volume providers have better outcomes? It is likely that the effects of physician and hospital volume combine or interact. One study suggests a multiplicative effect leading to much better outcomes among high-volume physicians practicing in high-volume hospitals. The relative contributions of physician and hospital volume to outcomes, however, have been examined in only a few studies. Studies generally do not illuminate how experience with procedures (or conditions) that are closely related to the procedure under study affects outcomes. Are cognitive and technical 4   The scoring method considered the representativeness of the study sample, sample size, number of adverse events, unit of analysis (e.g., joint effects of physician and hospital volume), inclusion of clinical processes of care, appropriateness of patient selection, categorization of volume, quality of risk adjustment, and outcomes measurement.

OCR for page 4
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary skills, for example, transferable from one cardiovascular procedure to another (i.e., what should “count” toward volume?)? Longitudinal studies are needed to answer some questions. Once high volume is attained, for example, does it have to be sustained, or can lower volumes be adequate to maintain skills? The literature focuses on the effects of the attending physician 's care. The experience of collaborating physicians and other clinical personnel has been far less thoroughly examined (e.g., do members of a surgical team need to achieve a team volume?). With few exceptions, the available literature does not identify clinical processes of care that contribute to observed differences in outcomes. Most studies focus on short-term outcomes such as in-hospital mortality. No studies examine longer-term outcomes such as quality of life or functional status. Five workshop participants were invited to provide reactions to the background paper: Norman Hertzer, a vascular surgeon from the Cleveland Clinic; Edward Hannan, a health services researcher at the State University of New York at Albany; Colin Begg, an epidemiologist from the Memorial Sloan-Kettering Cancer Center; Bruce Hillner, an internist and health services researcher at Virginia Commonwealth University; and Arnold Milstein, medical director at the Pacific Business Group on Health and chief physician at William M. Mercer, Inc. Dr. Hertzer pointed out that the difference between low and high volumes for vascular surgery is sometimes small (e.g., fewer than 3 versus more than 12 procedures for carotid endarterectomy; fewer than 4 versus more than 10 procedures for aortic aneurysm repair). While recognizing that the outcomes of low-volume providers are significantly worse than those of highvolume providers, he noted that providers in the intermediate-volume group often are indistinguishable from either low- or high-volume providers. It is often difficult to isolate the factors responsible for the volume-outcome relationship because, in the case of vascular surgery, different types of surgeons may be involved (e.g., general surgeons, neurosurgeons). Furthermore, the same surgeon may be responsible for a high volume of cases in a large, high-volume hospital and for the cases performed in a small, low-volume hospital. These observations underscore the importance of simultaneously tracking both physician and hospital volume and controlling for type of surgeon. The difficulty of defining surgical experience with volume further complicates these analyses. Surgeons with low volume can include those who are newly trained, but may have state-of-the-art skills, as well as surgeons with years of experience and good outcomes who are reducing their work load before retirement. The quality of available data systems is central to monitoring the quality of health care, and Dr. Hannan described a hierarchy of data for quality research. Optimally, one would be able to track the use of processes of care known through well-designed studies (e.g., clinical trials) to affect care. When accurate data are available to show that a certain procedure is, for example, being under- or overused, it is relatively easy for quality improvement programs to inform providers and hold them accountable. For most clinical care, however, strong links between processes and outcomes are not well established, and even when they are, there are relatively few data systems in place to track adherence to evidence-based practices.

OCR for page 4
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary In the absence of good data on processes of care, data about outcomes that include risk adjustment using detailed clinical data (usually available only in the medical chart or specialized databases) provide the best measurement of quality of care. In New York and New Jersey (and soon in California), statewide clinical databases are available for cardiac surgery, allowing analyses of outcomes by both individual surgeons and hospitals. Risk-adjustment methodologies, however, have not yet been perfected, so the interpretation of findings can be difficult, especially when applied to individual providers. Next in the hierarchy of quality-of-care measures are outcomes for which only administrative data (e.g., hospital discharge databases) are available to perform the risk adjustment. Such measures may suffer from both the specificity of the assessment of severity of illness (in general, risk-adjustment models using administrative data will have less predictive power than models based on detailed clinical data) and from reporting problems that plague administrative databases. While outcomes data can be informative, they are often limited to short-term measures such as death or complications, and other, longer-term outcomes of interest to patients are usually not available (e.g., functional status, quality of life). Dr. Hannan suggested that data on volume can serve as a proxy for quality but that its value as a quality indicator can be enhanced by using it with other imperfect indicators, such as adjusted or unadjusted mortality data from administrative databases. It is often difficult to judge when to implement policies based on research findings. Dr. Begg identified four criteria to consider in assessing the strength of the evidence on the volume– outcome relationship and adopting it as a standard of quality of care: The relationship must be plausible and logical. The observed trend must be consistent. The size of the outcome difference must be substantial and meet stringent statistical criteria. The effect must be confirmed in multiple studies. In his view, these criteria are met for certain procedures included in the literature review (i.e., surgery for cancer of the pancreas and esophagus). Dr. Begg agreed with other workshop participants when they encouraged the examination of processes of care that might explain the volume –outcome relationship, but cautioned that it may not always be possible with available data to identify the relevant processes, especially for procedures that exhibit small volume–outcome trends. In the case of cancer, the administration of adjuvant therapy may be documented in administrative databases, but not the dosage and whether the course of therapy was completed. Dr. Begg conducted a study of the relationship between the volume of surgery for colon cancer and outcomes, showing an outcome difference for stage III colon cancer. Investigators hypothesized that the trend would be explained by better adherence to recommendations on adjuvant chemotherapy use by higher-volume hospitals. When data on adjuvant therapy were retrieved and analyzed, however, appropriate use of adjuvant therapy did not explain the association. Any number of unmeasured or unrecorded processes of care could account for these findings. Surgical judgement, skill, improvisation in the face of unpredictable events, and the duration of surgery may, for example, all contribute to the finding, but they are difficult to study. Dr. Begg also described the difficulties of accurately assessing patient selection factors with available data (e.g., frailty is not necessarily captured in administrative data beyond lists of comorbid conditions).

OCR for page 4
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary Dr. Hillner presented data to show that any policies directing patients to high-volume centers would affect many people, because lower-volume facilities account for a significant fraction of patient care. In Maryland from 1992 to 1996, for example, 36 percent of surgeries for colon cancer were performed by surgeons treating fewer than 5 cases per year (average, 2 per year). Three-quarters of the surgeons performed 10 or fewer surgeries per year. By contrast, in Ireland where care is concentrated in high-volume settings, 80 percent of surgeons who do colectomies perform 10 or more surgeries for colon cancer per year. Dr. Hillner suggested that most individuals, when informed of the relationship between volume and outcome, would not find current practices in the United States acceptable. Dr. Hillner suggested some parallels between programs to promote selective referral for surgical procedures and disease management programs for patients with chronic conditions. For both, patient care resources are concentrated. Establishing the relationship between volume and outcomes for hospital-based procedures has been somewhat easier because of the availability of large databases containing information on hospital outcomes. There is somewhat less evidence of the success of disease management because of the limitations of available data and the need to examine longer-term outcomes of chronic diseases such as diabetes. Dr. Milstein observed that causal models to explain favorable hospital outcomes are in a primitive state. He supported the notion that process measures of quality would be preferable to volume and the consequent need for a significant expansion of routinely collected data elements. Dr. Milstein noted, however, major resistance to such expansion on both national and state levels (especially data to assess the performance of individual physicians). Dr. Milstein questioned the assumption that the most valuable use of volume–outcome research will be for quality improvement. This assumption reflects a faith that the health industry's weak quality improvement capabilities will improve and that little of the association of volume with outcome will ultimately be rooted in volume-influenced proficiency. It may be the case that the frequency of real-world exercise of physical and mental skills will remain important, even after low-volume hospitals are made aware of the “tricks” of the high-volume trade. Since hospital performance differences are likely to persist after provider-to-provider knowledge transfer, Dr. Milstein recommended that social science research be expanded to improve understanding of consumers' use of quality indices in choosing hospitals and physicians. He noted that the existing literature suggests that the use of provider quality comparisons by consumers is not likely to result from today's “information dump of statistical tables.” This will require simplification of information, putting it into multiple formats, and having it conveyed by trusted sources. Lastly, although Dr. Milstein agreed with the need for caution in using volume-based indicators, he questioned whether any workshop participants would commit to personally receiving complex treatments at the nearest low-volume hospital in their health plan's network. He suggested that this is what we currently tolerate for less well educated Americans who live near low-volume hospitals. Discussion moderator Dr. Sisk asked workshop participants to consider the types of measures needed to implement policy and to assess the adequacy of risk-adjustment methodologies. Two main applications of studies of the volume outcome relationship were identified in the discussion. The first is research, identifying whether a relationship exists and the factors that affect the relationship. For these purposes, available risk-adjustment methods are adequate, even when applied to administrative data. According to Dr. Halm, the literature review suggested that risk adjustment, although important, did not greatly affect the magnitude of the association between volume and outcome. The second application is to provide information to individual providers of care as part of a quality improvement program or to inform consumers of the volume or out-