Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 27
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary APPENDIX C How Is Volume Related to Quality in Health Care? A Systematic Review of the Research Literature Ethan A. Halm, M.D., Ph.D.; Clara Lee, M.D., M.P.P.; and Mark R. Chassin, M.D., M.P.P., M.P.H. Department of Health Policy, Mount Sinai School of Medicine EXECUTIVE SUMMARY Measuring and understanding the relationship between volume and outcome in the delivery of health services has engaged the attention of an increasing number of researchers since the 1980s. More recently, health care purchasers have begun to debate whether and how they should make use of these research findings in their relationships with health plans and individuals. We conducted a systematic review of the published research literature to assess the methodological quality of the research and to understand what is known about how volume and outcome are associated, by what causal pathways they might be linked, and how generalizable the research findings are. We developed an 18-point scale to rate the methodological rigor of each study. The scale was based on a conceptual model that identified a series of critical factors that need to be addressed to fully delineate the relationship between volume and quality of care. These factors include: adequacy of risk-adjustment for differences in severity and comorbidity, sample size, unit of analysis, assessment of specific clinical processes of care, organizational processes, skills and experience of the various relevant clinicians, appropriateness of patient selection, and the spectrum of important clinical outcomes. We identified published, English-language studies by electronic searches of MEDLINE, by consulting with experts, and by hand searching the reference lists of identified studies. We excluded studies with patient cohorts treated prior to 1980, samples that were not community or population-based, those in which a health outcome was not the dependent variable, and those in which volume was not an independent variable. This monograph reports the results of our review of 8 conditions and procedures: coronary artery bypass graft (CABG) surgery, pediatric cardiac surgery, carotid endarterectomy, abdominal aortic aneurysm repair, cancer surgery, percutaneous transluminal coronary angioplasty (PTCA), acute myocardial infarction, and acquired immuodeficiency syndrome (AIDS). Overall, 88 studies were included in this review. The median quality score was 8, less than 50% of the maximum possible score of 18. The highest score assigned (to 1 study) was 13, 72% of the maximum possible score. All of the remaining studies scored 11 or less. The principal omissions in the published research, which led to lower quality scores, included: Only 16 considered the independent effects of both physician and hospital volume;
OCR for page 28
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary Some (n = 24) used clinical data to adjust for differences among patients in severity and comorbidity, but only 4 reported statistically robust models; Few (n = 4) adjusted for differences in specific processes of care; and Few studies (n = 2) addressed appropriateness of patient selection. Several conclusions emerged from this review. There can be little doubt that for a wide variety of surgical procedures and medical conditions higher volume (whether assessed by hospital or by physician) is associated with better health outcomes. Although a publication bias against negative findings appearing in print cannot be excluded, the uniformity with which the published research documents or confirms the existence of the association is compelling. Fully 77% of the studies we reviewed found statistically significant associations between higher volume and better outcomes. The remaining 23% did not find statistically significant relationships. No study demonstrated a statistically significant association in the opposite direction. Finally, all 16 of the studies with the highest quality scores found statistically significant associations. Overall, however, the methodological rigor of the studies was modest, limiting generalizability and leaving many questions about the nature and causes of the association unanswered. Studies of the same procedure or condition typically employed widely varying definitions of high and low volume, precluding definitive conclusions about the nature of its relationship to outcome. We do not know, for example, whether a volume threshold exists—or more than one—above which outcomes are better but do not continue to improve with further volume increases. Is there a threshold effect for physician or hospital volume? Both? Neither? Very few studies addressed the question of whether the only relevant volume measure is that of the specific procedure or condition under study. Thus, we do not know the extent to which experience with procedures (or conditions) that are closely related affects outcome. Available research does not shed much light on what specific factors explain outcome differences between high and low volume providers. No longitudinal studies (and very few of any design) address the important question of how much of the variability, especially among low-volume providers, is due to chance. Few investigations have assessed differences in specific clinical processes of care, especially those known to affect outcomes. One intriguing exception showed that about one third of the mortality difference between high- and low-volume hospitals for acute myocardial infarction could be attributed to more frequent use of proven-effective medications at high-volume hospitals. We believe that this direction is the most promising avenue for future research: a detailed understanding of those critical clinical and organizational processes that affect outcomes and that differentiate high- and low-volume hospitals and physicians. Such knowledge could provide a blueprint for improvement. These research findings have several potential implications for public policy in health care. Public policy might be directed at informing consumers and referring physicians about the nature of the relationship between volume and outcome or at improving outcomes, or both. Data on volume of procedures by hospital are publicly accessible in many states through hospital discharge abstract databases. Many of these databases also permit identification of treating physicians. Making these data widely available is one potential way of informing consumers and clinicians. Some of the challenges such efforts would face are the difficulties of explaining what the data mean (and what they do not), the technical challenges of carefully identifying conditions and procedures for which research is conclusive that higher volume is associated with better outcomes (and at what threshold), and the need to update the data regularly.
OCR for page 29
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary Regulation in some states has limited the number of low-volume hospitals for some specific services, but such programs have become increasingly politically unpopular and require significant infrastructure to support. Recent actions by private employers suggest that some purchasers may be prepared to take an active role in steering patients toward high-volume hospitals, using incentives directed either at managed care plans or at employees themselves. No data exist on which to base predictions of the impact of such efforts. Available data suggest that significant numbers of patients do not use data on outcomes or volume where they exist to choose where to receive care. We believe that the most promising use of the research findings linking volume and outcome is to fuel quality improvement. To the extent ongoing research is successful in demonstrating what specific clinical and organizational processes of care are associated with improved outcomes, this knowledge can be put to direct and immediate use to improve hospital and physician care. Public policy can accelerate this improvement by supporting the necessary research, by fostering and evaluating demonstration programs to implement it, and by disseminating successful models. HOW IS VOLUME RELATED TO QUALITY IN HEALTH CARE? A SYSTEMATIC REVIEW OF THE RESEARCH LITERATURE “We must formulate some method of hospital report showing as nearly as possible what are the results of the treatment obtained at different institutions. This report must be made out and published by each hospital in a uniform manner, so that comparison will be possible. . . . A set of statistics had been prepared comparing the mortality at a certain semi-private hospital of 200 beds with that of four of the best general hospitals in America, having a total of 1,200 beds. These statistics were obtained from published reports. They clearly showed that the semi-private hospital not only did more operations, but that the mortality was much lower, especially in some of the more difficult branches of surgery.” Introduction and Historical Context This author goes on to speculate about what factors might have produced such an apparently anomalous result—that a small hospital might have better outcomes than larger, more prestigious institutions. Among the factors he cited in addition to the volume of operations were the skill of the surgeons, the comorbid conditions of the patients, the financial condition of the hospitals, and the way in which surgical departments were organized to triage particular patients to particular surgeons. Data to answer these questions continue to prove elusive, some 86 years after a Boston surgeon, Ernest Avery Codman, first posed them. (Codman 1914) Codman's questions lay dormant for more than 50 years, until the National Halothane Study, undertaken in the 1960s to assess possible hepatotoxic effects of halothane, unexpectedly reported a 24-fold variation in operative mortality among participating U.S. hospitals. (Moses 1968) This finding led to a series of further investigations. (Stanford 1976, Flood 1979, Flood 1984a, Flood 1984b) More recently, beginning with the work of Luft and colleagues (Luft 1979, Bunker 1982, Luft 1987), a large body of research has focused on measuring and explaining the relationship between patient outcomes and the volume of specific health services provided by hospitals and physicians. Many studies have documented that higher volume is associated with better outcomes for a variety of different surgical procedures and medical conditions. Two principal hypotheses have
OCR for page 30
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary been advanced to explain these relationships: 1) that physicians (and hospitals) develop more effective skills if they treat more patients (“practice makes perfect”) or 2) physicians (and hospitals) achieving better outcomes receive more referrals and thus accrue larger volumes (“selective referral”). (Flood 1984a, Flood 1984b, Luft 1987) Although a few studies have found some indirect evidence that is consistent with either or both of these hypotheses, we believe a reconceptualization is warranted of the model that describes the potential mechanisms by which volume and outcome might be related in health care. This reconceptualization is prompted both by several inconsistencies in the older hypotheses and by evidence from recent research. Although the “practice makes perfect” hypothesis has some intuitive appeal, extant research has been unable to demonstrate what skills or practices improve as hospitals or physicians gain experience with particular kinds of patients and why they are uniquely related to volume. For the “selective referral hypothesis” to be an important explanatory factor, the outcomes of different physicians, surgeons, and hospitals would have to be widely known for referring physicians (if not patients) to use in making treatment recommendations. But we know that such data are not readily available and cannot be the basis for very many patient referrals. Even where they do exist, physicians report that they do not use them to make referrals. (Schneider 1996, Hannan 1997) Nor is there evidence that patients either shun hospitals with well-publicized poor outcomes or flock to those with good ones. (Chassin 1996, Schneider 1998) Further, both of these hypotheses focus on explaining how particular hospitals or physicians might have achieved high volume and good outcomes over time. Neither offers much help in explaining the causes of differences in outcomes among hospitals or physicians with different volumes or in suggesting how the poorer performers might improve. Some recent evidence documenting systematic differences among high-and low-volume providers in the use of specific interventions points to a different set of explanatory factors. (Thiemann 1999) We, therefore, begin this systematic review of the research evidence linking volume and outcome in health care by describing a new conceptual model. In subsequent sections of this monograph, we describe how we used this model to develop criteria to judge the quality of the published research we reviewed. We then define our literature search strategies, inclusion and exclusion criteria, and methods for reviewing and rating included studies. We discuss the results of the review separately for each included topic area and then discuss generalizable findings and issues. Finally, we discuss the potential policy implications of the findings. Conceptual Framework Figure 1 displays the factors we believe are likely to be the most potent explanatory variables in understanding how volume of services is related to health outcomes. Patients are selected to receive surgical or medical services in a variety of ways. Some have little time or choice (e.g., those suffering acute myocardial infarction or ruptured aortic aneurysms). The opposite is true for many surgical procedures, especially for the vast majority of elective operations. We know little about the relationship between volume and patient selection. Are patients of high-volume hospitals or physicians likely to be more or less appropriate candidates for surgery than patients of low-volume providers? A single study documented that high-volume surgeons were more likely to perform inappropriate carotid endarterectomies than low-volume surgeons were. (Brook 1990b) Few investigations have addressed this important dimension of quality.
OCR for page 31
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary Figure 1 Conceptual Framework: How Could Volume Affect Quality? More attention has been focused on identifying the key variables that define the severity of patients' presenting illness and the comorbid conditions that also affect the outcomes of treatment. How well these factors are measured and taken into account (i.e., how well the data on outcomes are “risk-adjusted”) are crucial components of the validity of outcome comparisons between high- and low-volume providers. If high-volume surgeons or hospitals treat patients who are systematically sicker (or healthier) than their low-volume counterparts, comparisons of their outcomes are not valid without scientifically rigorous risk adjustment. Once the patient population undergoing treatment is adequately characterized, it is then necessary to consider exactly what treatments the patients receive. Volume of services cannot directly produce better outcomes. If volume is related to outcome, that association must be expressed by differences in the components of care or in the skill with which treatments are provided. For some conditions or procedures, specific component treatments (processes of care) have been demonstrated to improve outcomes. For example, it is essential to know whether patients undergoing treatment for acute myocardial infarction received therapies designed to achieve reperfusion (acute angioplasty or thrombolytic drugs in appropriate settings). Other effective processes for this condition include aspirin, beta blockers, and angiotensin converting enzyme (ACE) inhibitors, each with its own specific indications. Although data on effectiveness and efficacy for most surgical processes of care are lacking, it is equally essential that researchers investigate the extent to which carotid endarterectomies or coronary bypass operations are performed the same way at high- and low-volume centers and by different surgeons. Do high-volume surgeons employ different surgical techniques compared with low-volume surgeons? If they do different operations, we can characterize the differences and assess the extent to which these measurably different processes of care are associated with different outcomes. If they do exactly the same operations but obtain different results, we may attribute such outcome differences to differences in skill (or “unmeasured process” differences). Several aspects of this issue are worth noting. First, if physician experience (volume) is a proxy measure for the attainment of a certain skill level that permits superior performance, it is important to consider how that experience is best measured. Does superior performance in carotid endarterectomy or aortic aneurysm surgery derive from experience with just those procedures, or is the volume of other major vascular surgical procedures important also? This issue may be termed the “volume of what” question.
OCR for page 32
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary Second, it is important also to identify exactly what skills are related to outcomes. In surgical procedures, specific physical skills may be vital—for example, the speed and dexterity with which a carotid artery is dissected, minimizing trauma to adjacent tissues. Performance of tasks like these may improve as volume increases and may require a certain level of volume to achieve or maintain proficiency. The acquisition and maintenance of certain cognitive skills may also be important to producing good outcomes and related to volume. Recognizing uncommon presentations of medical conditions or anatomic variations in surgery and taking appropriate and timely action are examples. To the extent the physical and cognitive skills most crucial to outcomes can be characterized and specified, they may move from the realm of “unmeasured processes” to assume their proper role in the set of crucial and measurable processes that are integral to producing the best outcomes. Third, is the experience of other key participants important? Is the volume of experience in a hospital's emergency department or coronary care unit important to its outcomes for patients with acute myocardial infarction? What about the experience of key consultants? Is the amount of experience of the team managing the cardiopulmonary bypass pump important in contributing to the outcomes of coronary artery bypass surgery? Fourth, what about “organizational skill?” Do high-volume hospitals adopt specific organizational strategies that are particularly effective in enhancing outcomes? Do they employ sophisticated computerized reminder systems or standardized protocols that reduce errors of omission or commission? Do teaching hospitals, which are often high in volume, have an advantage because house staff and fellows provide 24-hour physician coverage? The more complex the treatment process, the more likely it is that physician or surgeon skill will be only one of many important components of the full complement of effective care. Defining the relative contributions of all important contributors is an essential research challenge. Criteria for Rating the Quality of Published Studies Using this conceptual model as a guide, we developed a scoring system to assess the quality of the research studies included in our systematic review. The full list of criteria is attached as Appendix A. Our aim was to create a quantitative method of assessing the research design of the studies we reviewed such that higher scores would reflect increasing likelihood of the study's ability to discern generalizable conclusions about the nature and magnitude of the relationship between volume and outcome. The first four criteria assess various aspects of the patient sample used in the research. We assigned one point if the sample was representative of the general population of all patients who might receive the treatments under study. Thus, studies of managed care plan enrollees or Medicare beneficiaries were not considered representative. We assigned two points if the study included patients of 50 or more physicians and 20 or more hospitals. If only one of these criteria was met, we assigned one point. No points were assigned if neither criterion was met. In some studies authors reported the number of hospitals in their sample but not the number of treating physicians. In these cases we estimated the number of physicians by assuming it would be at least equal to the number of hospitals. The vast majority of these studies included hundreds of hospitals from administrative databases, so we estimated the number of physicians as 50 for scoring this criterion. If the total sample size was 1000 patients or more, we assigned one point. Because statistical power to detect significant relationships in logistic regression models depends more on the total number of adverse events represented in the sample than on total sample size (and because the various conditions and procedures in this lit
OCR for page 33
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary erature have widely varying adverse event rates), we assigned 2 points if the total number of adverse events was greater than 100, one point if it was 21–100, and no points if it was 20 or less. We assigned no points if the study assessed the relationship between outcome and either hospital or physician volume. If both were assessed separately, we assigned one point. If the joint relationships of hospital and physician volume were assessed independently in a multivariate analysis, we assigned 2 points. And if a study examined both of these and the volume of another important component of the care process, we assigned 3 points. If the appropriateness of patient selection was not addressed, we assigned no points. If appropriateness was measured, we assigned one point. If it was measured and taken into account in the analysis of the volume/outcome relationship, we assigned 2 points. If volume was analyzed in only 2 categories, we assigned no points. If more than 2 categories were assessed or if volume was treated as a continuous variable, we assigned one point to credit a more sophisticated assessment of a possible dose-response relationship. In considering the various ways in which outcomes might be risk-adjusted, we assigned no points if no risk-adjustment at all was done. If data from insurance claims, hospital discharge abstract databases, or other sources of administrative data were used, we assigned one point. If data from clinical sources (e.g., medical records or prospectively designed clinical registries) were used for risk-adjustment, we assigned 2 points. If clinical data were used in a logistic regression model that demonstrated good calibration by the Hosmer-Lemeshow test and good discrimination (by a C-statistic of 0.75 or greater), we assigned 3 points. If specific clinical processes of care were not measured, we assigned no points. If a single process was measured and its impact on risk-adjusted outcomes assessed, we assigned one point. If two or more such processes were measured and evaluated, we assigned 2 points. Finally, if death was the only outcome evaluated, we assigned no points. If other adverse outcomes in addition to mortality were assessed, we assigned 2 points. Quality scores were summed across all 10 criteria for each study. The maximum possible total score was 18. Literature Review Methods We performed two electronic subject-based searches of the literature on MEDLINE (1966– 1999). A professional reference librarian assisted us in the development of our search strategy. First, we selected twelve conditions and procedures for which volume and outcomes had been studied (coronary artery bypass, carotid endarterectomy, peripheral vascular surgery, trauma, transplant, hip surgery, knee surgery, liver resection, lung resection, cancer surgery, angioplasty, and myocardial infarction). We then developed a list of search terms based on subject headings from articles known to be highly relevant to our topic and from the official indexing terms of the MEDLINE database. We performed multiple searches with combinations of these terms and evaluated the results of those searches for sensitivity and specificity, with respect to our topic of volume and outcomes. The search algorithm that yielded the greatest number of highly relevant articles combined the 12 conditions with the terms volume, utilization, frequency, statistics, and outcomes. In order to broaden our search to include articles on regionalization of care, we added another search that combined the 12 conditions with the term regionalization. (See Appendix B.) We also performed MEDLINE searches on authors known to have published widely on the study topic, and we searched the Cochrane Collaboration Database for systematic reviews. In addition to performing electronic database searches, we consulted experts in the field for further
OCR for page 34
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary references. Finally, we reviewed the references cited by each article that was ultimately included. We did not hand-search any journals. This review was limited to the English-language research literature. This monograph includes the findings of our review of eight procedures and conditions: coronary artery bypass graft (CABG) surgery, pediatric cardiac surgery, carotid endarterectomy, abdominal aortic aneurysm repair, cancer surgery, coronary angioplasty, acute myocardial infarction, and AIDS. Our electronic search also identified articles about volume and outcome in trauma, newborn intensive care, organ transplantation, hip surgery, knee replacement, neurosurgery, prostatectomy, cataract surgery, endocrine surgery, and general surgical procedures for benign disease, but we did not include them in this review. In the areas of trauma, newborn intensive care, and transplantation, the provision of services has already been regionalized or is otherwise highly regulated by government or other administrative bodies. Other areas were generally limited to very few articles per condition. We developed an initial set of inclusion criteria and tested it on studies of three conditions: cardiac surgery, carotid endarterectomy, and AIDS. After revision, the final inclusion criteria were: Time: patient cohorts treated from 1980 forward. Sample: community or population-based sample—case series or convenience samples were excluded. Multiple publications from the same database excluded; only the most recent or most complete publication was included Health outcome(s) must be assessed as the dependent variable(s). Volume must be an independent variable. We limited the review to studies of patients treated from 1980 to the present, because of the rapidity of changes in hospital care, available treatments, and surgical techniques. In our view, data from patient cohorts prior to 1980 would have questionable relevance to today's policy issues. In a few instances, we included studies if part of their patient sample included patients treated in 1978 or 1979, but most of the sample comprised patients from the 1980s. We excluded studies from single institutions, from voluntary registries, or other convenience samples because of the weak generalizability of such studies. We excluded a few studies in which the only dependent variable was a composite of deaths or long lengths of stay, because, formulated in this way, the dependent variable was not purely a health outcome. We also excluded a few studies in which the only dependent variable was a composite of death or complications, with the latter determined solely by secondary diagnosis codes in administrative databases. These studies were excluded because of the notorious unreliability of using such data to identify complications. In general, we excluded multiple publications from the same set of data, selecting only the most recent or complete, unless different publications reported substantially different analyses (e.g., one reported the relationship of hospital volume to outcome and another analyzed physician volume and outcome). Three reviewers assessed the articles for inclusion or exclusion, with at least two reviewers independently examining each article and applying the criteria. Discrepancies in the application of the criteria were resolved by discussion between the reviewers. We developed an initial set of assessment criteria and tested it on the studies of CABG surgery, carotid endarterectomy, and AIDS which had passed the inclusion criteria. Our final criteria for quality assessment and the
OCR for page 35
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary scoring system were described earlier and are listed in Appendix A. The same pair of reviewers who assessed each article for inclusion or exclusion then independently evaluated each article and assigned quality scores. Discrepancies were resolved by discussion between the 2 reviewers. Overall Results of Literature Review Identification of Studies The MEDLINE searches of subject headings resulted in 740 citations of all types of articles, including reviews, editorials, and letters. The MEDLINE searches of authors known to have published on the topic yielded 160 citations (including overlap with previous searches). Consultation with experts in the field yielded 136 citations (including overlap). The Cochrane Collaboration Database contained no relevant current or pending reviews. We reviewed the titles and abstracts of the 850 citations. We were able to exclude 600 based on the title or the abstract. Four of these were because of a language other than English. 250 citations remained for possible inclusion into the study. Of these, 128 were about one of the eight chosen conditions. We found ten additional citations by examining the references of all included articles. We were unable to obtain the articles for one citation (Choti 1998). Thus, we reviewed a total of 137 articles for possible inclusion into our systematic review. We counted an article that studied more than one condition as more than one study. For example, we counted the article by Hannan published in 1999 as two studies because it examined CABG and AAA. The total number of studies was 162. Exclusion of Studies Of the 162 studies, 74 were excluded, and 88 were included. The most common reason for exclusion was because volume was not an independent variable (31 studies, 19%). The second most common reason for exclusion was a study sample that was not community or populationbased (22 studies, 14%). This was particularly common among the studies of coronary angioplasty and cancer. These were primarily early studies of multiple surgical procedures that analyzed data from the 1970s. A few studies were excluded because the study outcomes were not health outcomes. A summary of this information in Table 1 below. Overall Results of Systematic Review Overall, 88 studies were included in our systematic review. Two authors scored each study on the 10 item methodological quality scale described previously. Possible total quality scores range from 0 to 18. The actual distribution of quality scores ranged from 2 to 13 (See Figure 2). The mean total quality score was 7.8 ± 1.9. The median score was 8 with an interquartile range of 6 to 9.
OCR for page 36
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary TABLE 1 Results of Literature Search Condition No. of Studies Reviewed No. of Studies Included No. of Studies Excluded Coronary Artery Bypass Graft 19 9 10 Pediatric Cardiac Surgery 3 3 0 Carotid Endarterectomy 24 18 6 Abdominal Aortic Aneurysm 16 12 4 Cancer Surgery 45 28 17 Coronary Angioplasty 20 9 11 Acute Myocardial Infarction 13 3 10 AIDS 9 6 3 Multiple Procedures 13* ** 13 TOTALS 162 88 74 * We counted an article that studied more than one procedure as separate studies for each. ** The articles that studied more than one procedure that were included are reflected in the numbers for each procedure. Since we excluded articles that analyzed data collected prior to 1980, the publication dates of those included in our review ranged from 1984 to 2000 (Our last MEDLINE search was March 2000). We found evidence of a dramatic increase in interest in this area as nearly half of all the studies we identified were published since 1998. The methodological characteristics of the 88 articles we reviewed are as follows. Approximately, two-thirds (68%) of the studies were judged to be truly representative of the underlying population of interest (i.e, all patients undergoing the specified procedure). The remaining one-third of authors focused exclusively on the care of a certain category of patients by insurance status such as Medicare or Veterans Affairs patients. Nearly all of the studies were based on administrative data sources, so it was no surprise that 78% of them had sample sizes of 1000 cases or more—many had tens of thousands of cases. Approximately 80% of studies were based on samples that included 20 or more hospitals and 50 or more treating physicians. We used this measure as a rough indication that there was the potential for substantial variation at the two provider levels of interest. However, 9% of authors reported on samples with fewer than 20 hospitals and 50 physicians. Twelve percent of studies had an intermediate number of providers. Figure 2 Distribution of Quality Scores of Volume/Outcomes Studies (N = 88)
OCR for page 37
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary While we had no easy way of measuring the statistical power of each study, we did abstract the absolute number of primary outcomes (e.g., inpatient deaths) as a proxy indicator of the potential power of the study to detect modest associations or perform any significant multivariate modeling. It is worth noting that event rates are a function of both the overall sample size and the absolute rate of the adverse outcome of interest. Three-fourths of studies reported more than 100 adverse events upon which the volume-outcomes analyses were based. The rest of the studies had 20 to 100 events, except for one which had fewer than 20 (Wade 1996). The primary outcome was mortality in 78% of the studies. This was most commonly inpatient death, though many reported 30 day death rates, and a few reported on 1, 3 or 5 year survival. The nineteen studies (22%) that examined more than just mortality were primarily studies of carotid endarterectomy that considered death or stroke as primary outcomes and investigations of coronary angioplasty that focused on death or emergency coronary artery bypass grafting together as major adverse events. The vast majority of investigators examined provider volume as a categorical variable. Twelve percent dichotomized volume into low and high without any more detailed look for doseresponse relationships. The remaining 88% reported at least three categories of volume, usually textiles or quartiles, or numerous categories broken into even-numbered ranges. A few authors analyzed volume as a continuous variable. Among the 88 studies we reviewed, there were 78 attempts to detect hospital level volume-outcome effects and 35 comparisons of physician level influences. The predominance of analyses of hospital volume effects is largely due to the fact that most of the studies were based on state or national hospital discharge databases that often lacked unique physician identifiers. Most studies (72%) were primarily focused on examining either the effects of hospital or physician volume on outcomes. Another 10% of studies reported both hospital and physician levels effects, but separately. Sixteen studies (18%) performed some type of multivariate or stratified analysis designed to try to tease apart the independent or synergistic contributions of physician and hospital volume to outcomes. We paid particularly close attention to the level of sophistication of the risk adjustment techniques used by investigators (if any), because failure to account for imbalances in patient case mix among providers can seriously confound the relationship between volume and outcome. Fifteen percent of studies performed no risk adjustment at all. The majority of studies (58%) relied exclusively on risk adjustment models based on administrative data, most commonly sociodemographics and ICD-9 discharge diagnoses. The limitations of these risk adjustment techniques are beyond the scope of our discussion but have been reviewed elsewhere (Iezzoni 1997). Only 24 studies (27%) used any clinical data in their risk adjustment models. Among this group, only six studies reported risk adjustment models that were robustly discriminating (model C statistic ≥ 0.75) and calibrated (by Hosmer-Lemeshow or other testing). There were two other higher level methodological issues that few studies seemed to address. Only 2 of 88 studies (2%) measured the appropriateness of patient selection—both were studies of carotid endarterectomy where this important issue has been recognized for a decade (Brook 1990a, Karp 1998). Similarly, only 4 of 88 authors (5%) made any attempt to adjust for the use of any key processes of care. Two studies of breast cancer adjusted for use of adjuvant therapy or
OCR for page 92
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 58. Field D, Hodges S, Mason E, Burton P. Survival and place of treatment after premature delivery. Archives of Disease in Childhood. 1991; 66(4 Spec No):408-10; discussion 410-1. 59. Fink A, Yano EM, Brook RH. The condition of the literature on differences in hospital mortality . Med Care. 1989; 27(4):315-36. 60. Finlayson SR, Birkmeyer JD, Tosteson AN, Nease RF, Jr. Patient preferences for location of care: implications for regionalization . Medical Care. 1999; 37(2):204-9. 61. Fisher ES, Malenka DJ, Solomon NA, Bubolz TA, Whaley FS, Wennberg JE. Risk of carotid endarterectomy in the elderly. American Journal of Public Health. 1989; 79(12):1617-20. 62. Flood AB, Ewy W, Scott WR, Forrest WH, Jr., Brown BW, Jr. The relationship between intensity and duration of medical services and outcomes for hospitalized patients. Med Care. 1979; 17(11):1088-102. 63. Flood AB, Scott WR, Ewy W. Does practice make perfect? Part I: The relation between hospital volume and outcomes for selected diagnostic categories. Medical Care. 1984a; 22(2):98-114. 64. Flood AB, Scott WR, Ewy W. Does practice make perfect? Part II: The relation between volume and outcomes and other hospital characteristics. Med Care. 1984b; 22(2):115-25. 65. Frances CD, Go AS, Dauterman KW, et al. Outcome following acute myocardial infarction: are differences among physician specialties the result of quality of care or case mix? Arch Intern Med. 1999; 159(13):1429-36. 66. Freeland MS, Hunt SS, Luft HS. Selective contracting for hospital care based on volume, quality, and price: prospects, problems, and unanswered questions. J Health Polit Policy Law. 1987; 12(3):409-26. 67. Garnick DW, Luft HS, McPhee SJ, Mark DH. Surgeon volume vs hospital volume: which matters more? JAMA. 1989; 262(4):547-8. 68. Gilchrist IC, Gardner LH, Muhlestein JB, et al. Effect of institutional volume and academic status on outcomes of coronary interventions: the IMPACT-II experience . Am Heart J. 1999; 138(5 Pt 1):976-82. 69. Gillis CR, Hole DJ. Survival outcome of care by specialist surgeons in breast cancer: a study of 3786 patients in the west of Scotland. BMJ. 1996; 312(7024):145-8. 70. Glasgow RE, Mulvihill SJ. Hospital volume influences outcome in patients undergoing pancreatic resection for cancer. Western Journal of Medicine. 1996; 165(5):294-300. 71. Glasgow RE, Showstack JA, Katz PP, Corvera CU, Warren RS, Mulvihill SJ. The relationship between hospital volume and outcomes of hepatic resection for hepatocellular carcinoma. Arch Surg. 1999; 134(1):30-5. 72. Gordon TA, Burleyson GP, Tielsch JM, Cameron JL. The effects of regionalization on cost and outcome for one general high-risk surgical procedure. Annals of Surgery. 1995; 221(1):43-9. 73. Gordon TA, Bowman HM, Tielsch JM, Bass EB, Burleyson GP, Cameron JL. Statewide regionalization of pancreaticoduodenectomy and its effect on in-hospital mortality. Annals of Surgery. 1998; 228(1):71-8. 74. Gordon TA, Bowman HM, Bass EB, et al. Complex gastrointestinal surgery: impact of provider experience on clinical and economic outcomes. J Am Coll Surg. 1999; 189(1):46-56. 75. Goss ME. Comments on “The condition of the literature on differences in hospital mortality ”. Med Care. 1990; 28(7):563-6. 76. Grassman ED, Johnson SA, Krone RJ. Predictors of success and major complications for primary percutaneous transluminal coronary angioplasty in acute myocardial infarction . An analysis of the 1990 to 1994 Society for Cardiac Angiography and Interventions registries. Journal of the American College of Cardiology. 1997; 30(1):201-8. 77. Gray HH. Cardiac interventional procedures in the UK 1992 to 1996. Council of the British Cardiovascular Intervention Society. Heart. 1999; 82 Suppl 2:II10-7. 78. Grover FL, Johnson RR, Shroyer AL, Marshall G, Hammermeister KE. The Veterans Affairs Continuous Improvement in Cardiac Surgery Study . Ann Thorac Surg. 1994; 58(6):1845-51.
OCR for page 93
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 79. Grumbach K, Anderson GM, Luft HS, Roos LL, Brook R. Regionalization of cardiac surgery in the United States and Canada. Geographic access, choice, and outcomes. JAMA. 1995; 274(16):1282-8. 80. Hamilton SM, Letourneau S, Pekeles E, Voaklander D, Johnston DW. The impact of regionalization on a surgery program in the Canadian health care system. Arch Surg. 1997; 132(6):605-9; discussion 609-11. 81. Hamilton BH, Hamilton VH. Estimating surgical volume-outcome relationships applying survival models: accounting for frailty and hospital fixed effects. Health Econ. 1997; 6(4):383-95. 82. Hamilton BH, Ho V. Does practice make perfect? Examining the relationship between hospital surgical volume and outcomes for hip fracture patients in Quebec . Med Care. 1998; 36(6):892-903. 83. Hannan EL, O'Donnell JF, Kilburn H, Jr., Bernard HR, Yazici A. Investigation of the relationship between volume and mortality for surgical procedures performed in New York State hospitals. JAMA. 1989; 262(4):503-10. 84. Hannan EL, Kilburn H, Jr., Bernard H, O'Donnell JF, Lukacik G, Shields EP. Coronary artery bypass surgery: the relationship between inhospital mortality rate and surgical volume after controlling for clinical risk factors. Medical Care. 1991; 29(11):1094-107. 85. Hannan EL, Kilburn H, Jr., O'Donnell JF, et al. A longitudinal analysis of the relationship between in-hospital mortality in New York State and the volume of abdominal aortic aneurysm surgeries performed. Health Serv Res. 1992a; 27(4):517-42. 86. Hannan E. Provider Volume-Patient Outcome Relationships in the Provision of Medical Care: An Update. 1992b (unpublished). 87. Hannan EL, Kilburn H, Jr., Racz M, Shields E, Chassin MR. Improving the outcomes of coronary artery bypass surgery in New York State. JAMA. 1994; 271(10):761-6. 88. Hannan EL, Siu AL, Kumar D, Kilburn H, Jr., Chassin MR. The decline in coronary artery bypass graft surgery mortality in New York State. The role of surgeon volume. JAMA. 1995; 273(3):209-13. 89. Hannan EL, Racz M, Ryan TJ, et al. Coronary angioplasty volume-outcome relationships for hospitals and cardiologists. JAMA. 1997a; 277(11):892-8. 90. Hannan EL, Racz MJ, Jollis JG, Peterson ED. Using Medicare claims data to assess provider quality for CABG surgery: does it work well enough? Health Serv Res. 1997b; 31(6):. 91. Hannan EL, Stone CC, Biddle TL, DeBuono BA. Public release of cardiac surgery outcomes data in New York: what do New York state cardiologists think of it? Am Heart J. 1997c; 134(6):1120-8. 92. Hannan EL, Racz M, Kavey RE, Quaegebeur JM, Williams R. Pediatric cardiac surgery: the effect of hospital and surgeon volume on in-hospital mortality. Pediatrics. 1998a; 101(6):963-9. 93. Hannan EL, Popp AJ, Tranmer B, Fuestel P, Waldman J, Shah D. Relationship between provider volume and mortality for carotid endarterectomies in New York State. Stroke. 1998b; 29:2292-7. 94. Hannan EL. The relation between volume and outcome in health care. N Engl J Med. 1999a; 340(21):1677-9. 95. Hannan EL. Percutaneous coronary interventions: heed the American College of Cardiology volume recommendations, but strive to improve quality . J Am Coll Cardiol. 1999b; 34(5):1481-3. 96. Hannan EL, Imperato PJ, Nenner RP, Starr H. Laparoscopic and open cholecystectomy in New York State: mortality, complications, and choice of procedure. Surgery. 1999c; 125(2):223-31. 97. Hannan E. The influence of hospital and surgeon volume on mortality for common cancer procedures. 2000b (in press). 98. Hansing CE. The risk and cost of coronary angiography. II. The risk of coronary angiography in Washington State. JAMA. 1979; 242(8):735-8. 99. Harmon JW, Tang DG, Gordon TA, et al. Hospital volume can serve as a surrogate for surgeon volume for achieving excellent outcomes in colorectal resection. Ann Surg. 1999; 230(3):404-11; discussion 411-3.
OCR for page 94
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 100. Hartz AJ, Kuhn EM, Krakauer H. The relationship of the value of outcome comparisons to the number of patients per provider. Int J Qual Health Care. 1997; 9(4):247-54. 101. Hermanek P, Hohenberger W. The importance of volume in colorectal cancer surgery. Eur J Surg Oncol. 1996; 22(3):213-5. 102. Hertzer NR, Avellone JC, Farrell CJ, et al. The risk of vascular surgery in a metropolitan community. With observations on surgeon experience and hospital size. J Vasc Surg. 1984; 1(1):13-21. 103. Hewitt M, Simone J. (Institute of Medicine and Commission on Life Sciences, National Research Council). Ensuring Quality Cancer Care. 1999. 104. Higashihara E, Baba S, Nakagawa K, et al. Learning curve and conversion to open surgery in cases of laparoscopic adrenalectomy and nephrectomy. J Urol. 1998; 159(3):650-3. 105. Hillner BE, Smith TJ. Hospital volume and patient outcomes in major cancer surgery: a catalyst for quality assessment and concentration of cancer services. JAMA. 1998; 280(20):1783-4. 106. Hirsch N, Hailey D, Martin C. The evolution of heart, lung and liver transplantation services in Australia. Health Policy. 1995; 34(1):63-71. 107. Hogg RS, Raboud J, Bigham M, Montaner JS, O'Shaughnessy M, Schechter MT. Relation between hospital HIV/AIDS caseload and mortality among persons with HIV/AIDS in Canada. Clinical and Investigative Medicine. Medecine Clinique et Experimental . 1998; 21(1):27-32. 108. Horbar JD, Badger GJ, Lewit EM, Rogowski J, Shiono PH. Hospital and patient characteristics associated with variation in 28-day mortality rates for very low birth weight infants. Vermont Oxford Network. Pediatrics. 1997; 99(2):149-56. 109. Horowitz MM, Przepiorka D, Champlin RE, et al. Should HLA-identical sibling bone marrow transplants for leukemia be restricted to large centers? Blood. 1992; 79(10):2771-4. 110. Hosenpud JD, Breen TJ, Edwards EB, Daily OP, Hunsicker LG. The effect of transplant center volume on cardiac transplant outcome . A report of the United Network for Organ Sharing Scientific Registry . JAMA. 1994; 271(23):1844-9. 111. Houghton A. Variation in outcome of surgical procedures. Br J Surg. 1994; 81(5):653-60. 112. Hsia DC, Moscoe LM, Krushat WM. Epidemiology of carotid endarterectomy among Medicare beneficiaries: 1985-1996 update. Stroke. 1998; 29(2):346-50. 113. Hughes RG, Hunt SS, Luft HS. Effects of surgeon volume and hospital volume on quality of care in hospitals. Medical Care. 1987; 25(6):489-503. 114. Hughes RG, Garnick DW, Luft HS, McPhee SJ, Hunt SS. Hospital volume and patient outcomes. The case of hip fracture patients . Medical Care. 1988; 26(11):1057-67. 115. Hulka F, Mullins RJ, Mann NC, et al. Influence of a statewide trauma system on pediatric hospitalization and outcome. J Trauma. 1997; 42(3):514-9. 116. Hunt J, Hill D, Besser M, West R, Roncal S. Outcome of patients with neurotrauma: the effect of a regionalized trauma system. Aust N Z J Surg. 1995; 65(2):83-6. 117. Iezzoni LI. Risk adjustment for measuring healthcare outcomes. Health Administration Press. Chicago, IL. 1997. 118. Imperato PJ, Nenner RP, Starr HA, Will TO, Rosenberg CR, Dearie MB. The effects of regionalization on clinical outcomes for a high risk surgical procedure: a study of the Whipple procedure in New York State. American Journal of Medical Quality. 1996; 11(4):193-7. 119. International Neonatal Network. The CRIB (clinical risk index for babies) score: a tool for assessing initial neonatal risk and comparing performance of neonatal intensive care units, [published erratum appears in Lancet 1993 Sep 4;342(8871):626]. Lancet. 1993; 342(8865):193-8. 120. Jenkins KJ, Newburger JW, Lock JE, Davis RB, Coffman GA, Iezzoni LI. In-hospital mortality for surgical repair of congenital heart defects: preliminary observations of variation by hospital caseload. Pediatrics. 1995; 95(3):323-30. 121. Jollis JG, Peterson ED, DeLong ER, et al. The relation between the volume of coronary angioplasty procedures at hospitals treating Medicare beneficiaries and short-term mortality . New England Journal of Medicine. 1994; 331(24):1625-9.
OCR for page 95
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 122. Jollis JG, DeLong ER, Peterson ED, et al. Outcome of acute myocardial infarction according to the specialty of the admitting physician. N Engl J Med. 1996; 335(25):1880-7. 123. Jollis JG, Peterson ED, Nelson CL, et al. Relationship between physician and hospital coronary angioplasty volume and outcome in elderly patients. Circulation. 1997; 95(11):2485-91. 124. Jones J, Rowan K. Is there a relationship between the volume of work carried out in intensive care and its outcome? International Journal of Technology Assessment in Health Care. 1995; 11(4):762-9. 125. Jubelirer RA, Agarwal NN, Beyer FCd, et al. Pediatric trauma triage: review of 1,307 cases. J Trauma. 1990; 30(12):1544-7. 126. Kagan RJ, Baker RJ. The impact of the volume of neurotrauma experience on mortality after head injury. American Surgeon. 1994; 60(6):394-400. 127. Kantonen I, Lepantalo M, Salenius JP, Matzke S, Luther M, Ylonen K. Influence of surgical experience on the results of carotid surgery . The Finnvasc Study Group. European Journal of Vascular and Endovascular Surgery. 1998; 15(2):155-60. 128. Karp HR, Flanders WD, Shipp CC, Taylor B, Martin D. Carotid endarterectomy among Medicare beneficiaries: a statewide evaluation of appropriateness and outcome. Stroke. 1998; 29(1):46-52. 129. Kastrati A, Neumann FJ, Schomig A. Operator volume and outcome of patients undergoing coronary stent placement. J Am Coll Cardiol. 1998; 32(4):970-6. 130. Katz DJ, Stanley JC, Zelenock GB. Operative mortality rates for intact and ruptured abdominal aortic aneurysms in Michigan: an eleven-year statewide experience. J Vasc Surg. 1994; 19(5):804-15; discussion 816-7. 131. Kazmers A, Jacobs L, Perkins A, Lindenauer SM, Bates E. Abdominal aortic aneurysm repair in Veterans Affairs medical centers . Journal of Vascular Surgery. 1996; 23(2):191-200. 132. Keeler EB, Rubenstein LV, Kahn KL, et al. Hospital characteristics and quality of care. JAMA. 1992; 268(13):1709-14. 133. Kelly JV, Hellinger FJ. Physician and hospital factors associated with mortality of surgical patients. Med Care. 1986; 24(9):785-800. 134. Kelly JV, Hellinger FJ. Heart disease and hospital deaths: an empirical study. Health Services Research. 1987; 22(3):369-95. 135. Kempczinski RF, Brott TG, Labutta RJ. The influence of surgical specialty and caseload on the results of carotid endarterectomy. J Vasc Surg. 1986; 3(6):911-6. 136. Khuri SF, Daley J, Henderson W, et al. Relation of surgical volume to outcome in eight common operations: results from the VA National Surgical Quality Improvement Program . Ann Surg. 1999; 230(3):414-29; discussion 429-32. 137. Kimmel SE, Berlin JA, Laskey WK. The relationship between coronary angioplasty procedure volume and major complications. JAMA. 1995; 274(14):1137-42. 138. Kimmel SE, Kolansky DM. Operator volume as a “risk factor”. J Am Coll Cardiol. 1997; 30(4):878-80. 139. Kingsnorth AN. Surgery for periampullary and pancreatic carcinoma: a Liverpool experience . Ann R Coll Surg Engl. 1997; 79(4):259-63. 140. Kirshner DL, O'Brien MS, Ricotta JJ. Risk factors in a community experience with carotid endarterectomy . J Vasc Surg. 1989; 10(2):178-86. 141. Kitahata MM, Koepsell TD, Deyo RA, Maxwell CL, Dodge WT, Wagner EH. Physicians' experience with the acquired immunodeficiency syndrome as a factor in patients' survival. N Engl J Med. 1996; 334(11):701-6. 142. Klein LW, Schaer GL, Calvin JE, et al. Does low individual operator coronary interventional procedural volume correlate with worse institutional procedural outcome?. J Am Coll Cardiol. 1997; 30(4):870-7. 143. Konvolinka CW, Copes WS, Sacco WJ. Institution and per-surgeon volume versus survival outcome in Pennsylvania 's trauma centers. Am J Surg. 1995; 170(4):333-40. 144. Krakauer H, Shekar SS, Kaye MP. The relationship of clinical outcomes to status as a Medicare-approved heart transplant center. Transplantation. 1995; 59(6):840-6.
OCR for page 96
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 145. Kramer S, Meadows AT, Pastore G, Jarrett P, Bruce D. Influence of place of treatment on diagnosis, treatment, and survival in three pediatric solid tumors. Journal of Clinical Oncology. 1984; 2(8):917-23. 146. Kreder HJ, Deyo RA, Koepsell T, Swiontkowski MF, Kreuter W. Relationship between the volume of total hip replacements performed by providers and the rates of postoperative complications in the state of Washington. Journal of Bone and Joint Surgery. American Volume. 1997; 79(4):485-94. 147. Kreder HJ, Williams JI, Jaglal S, Hu R, Axcell T, Stephen D. Are complication rates for elective primary total hip arthroplasty in Ontario related to surgeon and hospital volumes? A preliminary investigation. Can J Surg. 1998; 41(6):431-7. 148. Kuhn EM, Hartz AJ, Krakauer H, Bailey RC, Rimm AA. The relationship of hospital ownership and teaching status to 30-and 180-day adjusted mortality rates. Medical Care. 1994; 32(11):1098-108. 149. Laffel GL, Barnett AI, Finkelstein S, Kaye MP. The relation between experience and outcome in heart transplantation . N Engl J Med. 1992; 327(17):1220-5. 150. Lavernia CJ, Guzman JF. Relationship of surgical volume to short-term mortality, morbidity, and hospital charges in arthroplasty. Journal of Arthroplasty. 1995; 10(2):133-40. 151. Leape LL, Park RE, Solomon DH, Chassin MR, Kosecoff J, Brook RH. Relation between surgeons' practice volumes and geographic variation in the rate of carotid endarterectomy. N Engl J Med. 1989; 321(10):653-7. 152. LeFevre M, Sanner L, Anderson S, Tsutakawa R. The relationship between neonatal mortality and hospital level. Journal of Family Practice. 1992; 35(3):259-64. 153. Lewis SJ. Trends in cardiac care: contractions, concentration, collaboration . Hosp Technol Ser. 1993; 12(15):1-29. 154. Lieberman MD, Kilburn H, Lindsey M, Brennan MF. Relation of perioperative deaths to hospital volume among patients undergoing pancreatic resection for malignancy. Annals of Surgery. 1995; 222(5):638-45. 155. Luft HS, Bunker JP, Enthoven AC. Should operations be regionalized? The empirical relation between surgical volume and mortality. N Engl J Med. 1979; 301(25):1364-9. 156. Luft HS. The relation between surgical volume and mortality: an exploration of causal factors and alternative models. Med Care. 1980; 18(9):940-59. 157. Luft HS, Hunt SS, Maerki SC. The volume-outcome relationship: practice-makes-perfect or selective-referral patterns? Health Services Research. 1987; 22(2):157-82. 158. Luft H, Garnick D, Mark D, McPhee S. Hospital Volume, Physician Volume, and Patient Outcomes; 1990. 159. Luft HS, Parker JD. Volume and mortality in coronary artery bypass grafting [published erratum appears in BMJ 1995 Dec 2;311(7018):1508]. BMJ. 1995; 311(7015):1304-5. 160. Luft HS. Pushing good research to go farther. Health Serv Res. 1998 ;33(5 Pt 1):1185-90. 161. Lupon J, Valle V, Marrugat J, et al. Six-month outcome in unstable angina patients without previous myocardial infarction according to the use of tertiary cardiologic resources. RESCATE Investigators. Recursos Empleados en el Sindrome Coronario Agudo y Tiempos de Espera. J Am Coll Cardiol. 1999; 34(7):1947-53. 162. Maerki SC, Luft HS, Hunt SS. Selecting categories of patients for regionalization. Implications of the relationship between volume and outcome. Medical Care. 1986; 24(2):148-58. 163. Malenka DJ, McGrath PD, Wennberg DE, et al. The relationship between operator volume and outcomes after percutaneous coronary interventions in high volume hospitals in 1994-1996: the northern New England experience. Northern New England Cardiovascular Disease Study Group. J Am Coll Cardiol. 1999; 34(5):1471-80. 164. Manheim LM, Sohn MW, Feinglass J, Ujiki M, Parker MA, Pearce WH. Hospital vascular surgery volume and procedure mortality rates in California, 1982-1994. J Vasc Surg. 1998; 28(1):45-56; discussion 56-8.
OCR for page 97
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 165. Marrugat J, Sanz G, Masia R, et al. Six-month outcome in patients with myocardial infarction initially admitted to tertiary and nontertiary hospitals. RESCATE Investigators. Recursos Empleados en el Sindrome Coronario Agudo y Tiempos de Espera . J Am Coll Cardiol. 1997; 30(5):1187-92. 166. Marshall MN, Shekelle PG, Leatherman S, Brook RH. The public release of performance data: what do we expect to gain? A review of the evidence [In Process Citation]. JAMA. 2000; 283(14):1866-74. 167. Mayfield JA, Rosenblatt RA, Baldwin LM, Chu J, Logerfo JP. The relation of obstetrical volume and nursery level to perinatal mortality. American Journal of Public Health. 1990; 80(7):819-23. 168. Maynard C, Every NR, Chapko MK, Ritchie JL. Institutional volumes and coronary angioplasty outcomes before and after the introduction of stenting. Eff Clin Pract. 1999; 2(3):108-13. 169. McArdle CS, Hole D. Impact of variability among surgeons on postoperative morbidity and mortality and ultimate survival. BMJ. 1991; 302(6791):1501-5. 170. McGrath PD, Wennberg DE, Malenka DJ, et al. Operator volume and outcomes in 12,998 percutaneous coronary interventions. Northern New England Cardiovascular Disease Study Group. J Am Coll Cardiol. 1998; 31(3):570-6. 171. McMillan RW, Uppot R, Zibari GB, Aultman DF, Dies DF, McDonald JC. Can low volume liver transplant centers be successful? The Regional Transplant Center of Willis-Knighton & Louisiana State University Medical Center. The first 120 liver transplants. J La State Med Soc. 1999; 151(7):367-72. 172. McWilliam LJ, Knox F, Wilkinson N, Oogarah P. Performance of skin biopsies by general practitioners. BMJ. 1991; 303(6811):1177-9. 173. Meadows AT, Kramer S, Hopson R, Lustbader E, Jarrett P, Evans AE. Survival in childhood acute lymphocytic leukemia: effect of protocol and place of treatment. Cancer Investigation. 1983; 1(1):49-55. 174. Mehta RH, Stalhandske EJ, McCargar PA, Ruane TJ, Eagle KA. Elderly patients at highest risk with acute myocardial infarction are more frequently transferred from community hospitals to tertiary centers: reality or myth? Am Heart J. 1999; 138(4 Pt 1):688-95. 175. Moses LE, Mosteller F. Institutional differences in Postoperative death rates. Commentary on some of the findings of the National Halothane Study. JAMA. 1968; 203(7):492-4. 176. Mullins RJ, Veum-Stone J, Helfand M, et al. Outcome of hospitalized injured patients after institution of a trauma system in an urban area. JAMA. 1994; 271(24):1919-24. 177. Munoz E, Boiardo R, Mulloy K, et al. Economies of scale, physician volume for orthopedic surgical patients, and the DRG prospective payment system. Orthopedics. 1990; 13(1):39-44. 178. Munoz E, Mulloy K, Goldstein J, Tenenbaum N, Wise L. Costs, quality, and the volume of surgical oncology procedures. Arch Surg. 1990; 125(3):360-3. 179. Neoptolemos JP, Russell RC, Bramhall S, Theis B. Low mortality following resection for pancreatic and periampullary tumours in 1026 patients: UK survey of specialist pancreatic units. UK Pancreatic Cancer Group. British Journal of Surgery. 1997; 84(10):1370-6. 180. Nguyen HN, Averette HE, Hoskins W, Penalver M, Sevin BU, Steren A. National survey of ovarian carcinoma. Part V. The impact of physician 's specialty on patients' survival. Cancer. 1993; 72(12):3663-70. 181. Norton EC, Garfinkel SA, McQuay LJ, et al. The effect of hospital volume on the in-hospital complication rate in knee replacement patients.. 1998. 182. O'Keefe GE, Jurkovich GJ, Copass M, Maier RV. Ten-year trend in survival and resource utilization at a level I trauma center. Ann Surg. 1999; 229(3):409-15. 183. Paneth N, Kiely JL, Wallenstein S, Marcus M, Pakter J, Susser M. Newborn intensive care and neonatal mortality in low-birth-weight infants: a population study. New England Journal of Medicine. 1982; 307(3):149-55. 184. Park RE, Brook RH, Kosecoff J, et al. Explaining variations in hospital death rates. Randomness, severity of illness, quality of care. JAMA. 1990; 264(4):484-90.
OCR for page 98
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 185. Parry JM, Collins S, Mathers J, Scott NA, Woodman CB. Influence of volume of work on the outcome of treatment for patients with colorectal cancer. Br J Surg. 1999; 86(4):475-81. 186. Parsonnet V, Bernstein AD. Outcome versus volume in coronary-bypass operations. Ann Thorac Surg. 1996; 61(6):1879-80. 187. Patti M, Corvera CU, Glasgow RE, Way LW. A hospital's annual rate of esophagectomy influences the operative mortality rate. Journal of Gastrointestinal Surgery. 1998; 2:186-92. 188. Pearce WH, Parker MA, Feinglass J, Ujiki M, Manheim LM. The importance of surgeon volume and training in outcomes for vascular surgical procedures. J Vasc Surg. 1999; 29(5):768-76; discussion 777-8. 189. Pellegrini CA, Heck CF, Raper S, Way LW. An analysis of the reduced morbidity and mortality rates after pancreaticoduodenectomy . Arch Surg. 1989; 124(7):778-81. 190. Perler BA, Dardik A, Burleyson GP, Gordon TA, Williams GM. Influence of age and hospital volume on the results of carotid endarterectomy: a statewide analysis of 9918 cases. Journal of Vascular Surgery. 1998; 27(1):25-31; discussion 31-3. 191. Pession A, Locatelli F, Rondelli R, Prete A, Paolucci G. Current use and outcome of blood and marrow transplantation in childhood according to the Italian Registry. Bone Marrow Transplant. 1998; 22 Suppl 5:S25-7. 192. Pett SB, Jr. Coronary artery bypass grafting volume and mortality. Ann Thorac Surg. 1996; 62(3):941-3. 193. Phibbs CS, Bronstein JM, Buxton E, Phibbs RH. The effects of patient volume and level of care at the hospital of birth on neonatal mortality. JAMA. 1996; 276(13):1054-9. 194. Phillips KA, Luft HS, Ritchie JL. The association of hospital volumes of percutaneous transluminal coronary angioplasty with adverse outcomes, length of stay, and charges in California. Med Care. 1995; 33(5):502-14. 195. Phillips KA, Luft HS, Ritchie JL. Coronary angioplasty procedure volume and major complications. JAMA. 1996; 275(8):595; discussion 596. 196. Phillips KA, Luft HS. The policy implications of using hospital and physician volumes as “indicators” of quality of care in a changing health care environment. Int J Qual Health Care. 1997; 9(5):341-8. 197. Pilcher DB, Davis JH, Ashikaga T, et al. Treatment of abdominal aortic aneurysm in an entire state over 7 1/2 years. American Journal of Surgery. 1980; 139(4):487-94. 198. Pollack MM, Alexander SR, Clarke N, Ruttimann UE, Tesselaar HM, Bachulis AC. Improved outcomes from tertiary center pediatric intensive care: a statewide comparison of tertiary and nontertiary care facilities . Critical Care Medicine. 1991; 19(2):150-9. 199. Porter GA, Soskolne CL, Yakimets WW, Newman SC. Surgeon-related factors and outcome in rectal cancer. Ann Surg. 1998; 227(2):157-67. 200. Potosky AL, Warren JL. Radical prostatectomy: does higher volume lead to better quality? J Natl Cancer Inst. 1999; 91(22):1906-7. 201. Pronovost PJ, Jenckes MW, Dorman T, et al. Organizational characteristics of intensive care units related to outcomes of abdominal aortic surgery. JAMA. 1999; 281(14):1310-7. 202. Reis GJ. Coronary angioplasty procedure volume and major complications. JAMA. 1996; 275(8):595-6. 203. Richardson JD, Main KA. Carotid endarterectomy in the elderly population: a statewide experience . J Vasc Surg. 1989; 9(1):65-73. 204. Riley G, Lubitz J. Outcomes of surgery among the Medicare aged: surgical volume and mortality. Health Care Financing Review. 1985; 7(1):37-47. 205. Ritchie JL, Phillips KA, Luft HS. Coronary angioplasty. Statewide experience in California. Circulation. 1993; 88(6):2735-43. 206. Ritchie JL, Maynard C, Chapko MK, Every NR, Martin DC. Association between percutaneous transluminal coronary angioplasty volumes and outcomes in the Healthcare Cost and Utilization Project 1993-1994. Am J Cardiol. 1999; 83(4):493-7.
OCR for page 99
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 207. Rogers FB, Osler TM, Shackford SR, Cohen M, Camp L. Financial outcome of treating trauma in a rural environment. J Trauma. 1997; 43(1):65-72; discussion 72-3. 208. Rohrer JE, Vaughan MS, Wenzel RP. Regionalization of tertiary care: impact of safe cardiovascular volumes in Iowa. Health Serv Manage Res. 1997; 10(1):1-6. 209. Romano PS, Mark DH. Patient and hospital characteristics related to in-hospital mortality after lung cancer resection. Chest. 1992; 101(5):1332-7. 210. Roohan PJ, Bickell NA, Baptiste MS, Therriault GD, Ferrara EP, Siu AL. Hospital volume differences and five-year survival from breast cancer . American Journal of Public Health. 1998; 88(3):454-7. 211. Roos LL, Jr., Cageorge SM, Roos NP, Danzinger R. Centralization, certification, and monitoring. Readmissions and complications after surgery. Medical Care. 1986; 24(11):1044-66. 212. Roos LL, Roos NP, Sharp SM. Monitoring adverse outcomes of surgery using administrative data. Health Care Financing Review. 1987; Spec No:5-16. 213. Roos LL, Sharp SM. Innovation, centralization, and growth. Coronary artery bypass graft surgery in Manitoba. Med Care. 1989; 27(5):441-52. 214. Rosenthal GE, Harper DL, Quinn LM, Cooper GS. Severity-adjusted mortality and length of stay in teaching and nonteaching hospitals. Results of a regional study. JAMA. 1997; 278(6):485-90. 215. Ruby ST, Robinson D, Lynch JT, Mark H. Outcome analysis of carotid endarterectomy in Connecticut: the impact of volume and specialty. Ann Vasc Surg. 1996; 10(1):22-6. 216. Rutledge R, Fakhry SM, Meyer A, Sheldon GF, Baker CC. An analysis of the association of trauma centers with per capita hospitalizations and death rates from injury. Ann Surg. 1993; 218(4):512-21; discussion 521-4. 217. Rutledge R, Oller DW, Meyer AA, Johnson GJ, Jr. A statewide, population-based time-series analysis of the outcome of ruptured abdominal aortic aneurysm. Ann Surg. 1996; 223(5):492-502; discussion 503-5. 218. Ryan TJ. The critical question of procedure volume minimums for coronary angioplasty . JAMA. 1995; 274(14):1169-70. 219. Sagar PM, Hartley MN, MacFie J, Taylor BA, Copeland GP. Comparison of individual surgeon's performance. Risk-adjusted analysis with POSSUM scoring system. Dis Colon Rectum. 1996; 39(6):654-8. 220. Saigal S, Rosenbaum P, Stoskopf B, Sinclair JC. Outcome in infants 501 to 1000 gm birth weight delivered to residents of the McMaster Health Region. Journal of Pediatrics. 1984; 105(6):969-76. 221. Sainsbury R, Haward B, Rider L, Johnston C, Round C. Influence of clinician workload and patterns of treatment on survival from breast cancer. Lancet. 1995; 345(8960):1265-70. 222. Sampalis JS, Lavoie A, Boukas S, et al. Trauma center designation: initial impact on trauma-related mortality . J Trauma. 1995; 39(2):232-7; discussion 237-9. 223. Sampalis JS, Denis R, Frechette P, Brown R, Fleiszer D, Mulder D. Direct transport to tertiary trauma centers versus transfer from lower level facilities: impact on mortality and morbidity among patients with major trauma. Journal of Trauma. 1997; 43(2):288-95; discussion 295-6. 224. Sandrick K. Learning from experience. In AIDS treatment, knowledge means quality . Hospitals. 1993; 67(7):32-5. 225. Schein OD, Steinberg EP, Javitt JC, et al. Variation in cataract surgery practice and clinical outcomes. Ophthalmology. 1994; 101(6):1142-52. 226. Schlenker RE, Hittle DF, Hrincevich CA, Kaehny MM. Volume/outcome relationships in small rural hospitals. J Rural Health. 1996; 12(5):395-409. 227. Schneider EC, Epstein AM. Influence of cardiac-surgery performance reports on referral practices and access to care. A survey of cardiovascular specialists. N Engl J Med. 1996; 335(4):251-6. 228. Schneider EC, Epstein AM. Use of public performance reports: a survey of patients undergoing cardiac surgery. JAMA. 1998; 279(20):1638-42. 229. Segal HE, Rummel L, Wu B. The utility of PRO data on surgical volume: the example of carotid endarterectomy. QRB Qual Rev Bull. 1993; 19(5):152-7.
OCR for page 100
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 230. Selby JV, Fireman BH, Lundstrom RJ, et al. Variation among hospitals in coronary-angiography practices and outcomes after myocardial infarction in a large health maintenance organization . N Engl J Med. 1996; 335(25):1888-96. 231. Shapiro MF, Greenfield S. Experience and outcomes in AIDS. JAMA. 1992; 268(19):2698-9. 232. Sheikh K. The relation between volume and outcome in health care. N Engl J Med. 1999; 341(14):1085-6. 233. Shook TL, Sun GW, Burstein S, Eisenhauer AC, Matthews RV. Comparison of percutaneous transluminal coronary angioplasty outcome and hospital costs for low-volume and high-volume operators. Am J Cardiol. 1996; 77(5):331-6. 234. Shortell SM, LoGerfo JP. Hospital medical staff organization and quality of care: results for myocardial infarction and appendectomy. Med Care. 1981; 19(10):1041-55. 235. Shortell SM, Zimmerman JE, Rousseau DM, et al. The performance of intensive care units: does good management make a difference? Med Care. 1994; 32(5):508-25. 236. Showstack JA, Rosenfeld KE, Garnick DW, Luft HS, Schaffarzick RW, Fowles J. Association of volume with outcome of coronary artery bypass graft surgery. Scheduled vs nonscheduled operations. JAMA. 1987; 257(6):785-9. 237. Shroyer AL, Marshall G, Warner BA, et al. No continuous relationship between Veterans Affairs hospital coronary artery bypass grafting surgical volume and operative mortality. Annals of Thoracic Surgery. 1996; 61(1):17-20. 238. Simunovic M, To T, Theriault M, Langer B. Relation between hospital surgical volume and outcome for pancreatic resection for neoplasm in a publicly funded health care system. CMAJ. 1999; 160(5):643-8. 239. Sloan FA, Shayne MW, Doyle MD. Is there a rationale for regionalizing organ transplantation services? Journal of Health Politics, Policy and Law. 1989; 14(1):115-67. 240. Sloan EP, Callahan EP, Duda J, Sheaff CM, Robin AP, Barrett JA. The effect of urban trauma system hospital bypass on prehospital transport times and Level 1 trauma patient survival. Ann Emerg Med. 1989; 18(11):1146-50. 241. Smith DB, Larsson JL. The impact of learning on cost: the case of heart transplantation . Hosp Health Serv Adm. 1989; 34(1):85-97. 242. Sollano JA, Gelijns AC, Moskowitz AJ, et al. Volume-outcome relationships in cardiovascular operations: New York State, 1990-1995. J Thorac Cardiovasc Surg. 1999; 117(3):419-28; discussion 428-30. 243. Solomon RA, Mayer SA, Tarmey JJ. Relationship between the volume of craniotomies for cerebral aneurysm performed at New York state hospitals and in-hospital mortality. Stroke. 1996; 27(1):13-7. 244. Sosa JA, Bowman HM, Gordon TA, et al. Importance of hospital volume in the overall management of pancreatic cancer. Annals of Surgery. 1998a; 228(3):429-38. 245. Sosa JA, Bowman HM, Tielsch JM, Powe NR, Gordon TA, Udelsman R. The importance of surgeon experience for clinical and economic outcomes from thyroidectomy. Ann Surg. 1998b; 228(3):320-30. 246. Sosa JA, Powe NR, Levine MA, Udelsman R, Zeiger MA. Profile of a clinical practice: Thresholds for surgery and surgical outcomes for patients with primary hyperparathyroidism: a national survey of endocrine surgeons. J Clin Endocrinol Metab. 1998c; 83(8):2658-65. 247. Sowden AJ, Deeks JJ, Sheldon TA. Volume and outcome in coronary artery bypass graft surgery: true association or artefact?. BMJ. 1995; 311(6998):151-5. 248. Stanford Center for Health Care Research. Comparison of hospitals with regard to outcomes of surgery. Health Serv Res. 1976; 11(2):112-27. 249. Steele RJ. The influence of surgeon case volume on outcome in site-specific cancer surgery. Eur J Surg Oncol. 1996; 22(3):211-3. 250. Stiller CA. Centralisation of treatment and survival rates for cancer. Archives of Disease in Childhood. 1988; 63(1):23-30.
OCR for page 101
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 251. Stiller CA, Draper GJ. Treatment centre size, entry to trials, and survival in acute lymphoblastic leukaemia. Archives of Disease in Childhood. 1989; 64(5):657-61. 252. Stiller CA. Centralised treatment, entry to trials and survival . Br J Cancer. 1994; 70(2):352-62. 253. Stone VE, Seage GRd, Hertz T, Epstein AM. The relation between hospital experience and mortality for patients with AIDS. JAMA. 1992; 268(19):2655-61. 254. Taylor HD, Dennis DA, Crane HS. Relationship between mortality rates and hospital patient volume for Medicare patients undergoing major orthopaedic surgery of the hip, knee, spine, and femur. Journal of Arthroplasty. 1997; 12(3):235-42. 255. Taylor CL, Yuan Z, Selman WR, Ratcheson RA, Rimm AA. Mortality rates, hospital length of stay, and the cost of treating subarachnoid hemorrhage in older patients: institutional and geographical differences. Journal of Neurosurgery. 1997; 86(4):583-8. 256. Tepas JJ, 3rd, Patel JC, DiScala C, Wears RL, Veldenz HC. Relationship of trauma patient volume to outcome experience: can a relationship be defined? Journal of Trauma. 1998; 44(5):827-30; discussion 830-1. 257. Thiemann DR, Coresh J, Oetgen WJ, Powe NR. The association between hospital volume and survival after acute myocardial infarction in elderly patients. N Engl J Med. 1999; 340(21):1640-8. 258. Thorpe AC, Cleary R, Coles J, Vernon S, Reynolds J, Neal DE. Deaths and complications following prostatectomy in 1400 men in the northern region of England. Northern Regional Prostate Audit Group. British Journal of Urology. 1994; 74(5):559-65. 259. Tu JV, Naylor CD. Coronary artery bypass mortality rates in Ontario. A Canadian approach to quality assurance in cardiac surgery. Steering Committee of the Provincial Adult Cardiac Care Network of Ontario. Circulation. 1996; 94(10):2429-33. 260. Tu JV, Hannan EL, Anderson GM, et al. The fall and rise of carotid endarterectomy in the United States and Canada. N Engl J Med. 1998; 339(20):1441-7. 261. Turner BJ, Ball JK. Variations in inpatient mortality for AIDS in a national sample of hospitals. J Acquir Immune Defic Syndr. 1992; 5(10):978-87. 262. Turner BJ, McKee L, Fanning T, Markson LE. AIDS specialist versus generalist ambulatory care for advanced HIV infection and impact on hospital use. Med Care. 1994; 32(9):902-16. 263. Veith FJ, Goldsmith J, Leather RP, Hannan EL. The need for quality assurance in vascular surgery. J Vasc Surg. 1991; 13(4):523-6. 264. Verloove-Vanhorick SP, Verwey RA, Ebeling MC, Brand R, Ruys JH. Mortality in very preterm and very low birth weight infants according to place of birth and level of care: results of a national collaborative survey of preterm and very low birth weight infants in The Netherlands . Pediatrics. 1988; 81(3):404-11. 265. Wachter RM, Luce JM, Turner J, Volberding P, Hopewell PC. Intensive care of patients with the acquired immunodeficiency syndrome. Outcome and changing patterns of utilization. Am Rev Respir Dis. 1986; 134(5):891-6. 266. Wade TP, Halaby IA, Stapleton DR, Virgo KS, Johnson FE. Population-based analysis of treatment of pancreatic cancer and Whipple resection: Department of Defense hospitals, 1989-1994. Surgery. 1996; 120(4):680-5; discussion 686-7. 267. Wen SW, Simunovic M, Williams JI, Johnston KW, Naylor CD. Hospital volume, calendar age, and short term outcomes in patients undergoing repair of abdominal aortic aneurysms: the Ontario experience, 1988-92. Journal of Epidemiology and Community Health. 1996; 50(2):207-13. 268. Wennberg DE, Lucas FL, Birkmeyer JD, Bredenberg CE, Fisher ES. Variation in carotid endarterectomy mortality in the Medicare population: trial hospitals, volume, and patient characteristics. JAMA. 1998; 279(16):1278-81. 269. Wexler MJ. More procedures, better quality of care? Is there a case for regionalization of pancreatic resection for neoplasm? CMAJ. 1999; 160(5):671-3. 270. Wexner SD, Cohen SM, Ulrich A, Reissman P. Laparoscopic colorectal surgery--are we being honest with our patients? Dis Colon Rectum. 1995; 38(7):723-7.
OCR for page 102
Interpreting the Volume–Outcome Relationship in the Context of Health Care Quality: Workshop Summary 271. Whittle J, Lin CJ, Lave JR, et al. Relationship of provider characteristics to outcomes, process, and costs of care for community-acquired pneumonia. Medical Care. 1998; 36(7):977-87. 272. Williams RL. Measuring the effectiveness of perinatal medical care. Medical Care. 1979; 17(2):95-110. 273. Williamson HA, Jr., Hart LG, Pirani MJ, Rosenblatt RA. Rural hospital inpatient surgical volume: cutting-edge service or operating on the margin? J Rural Health. 1994; 10(1):16-25. 274. Wilson JM, Menkhaus P, Gustin BW. Volume and outcome of coronary artery bypass graft surgery. JAMA. 1987; 257(18):2434-5. 275. Witt PD, Wahlen JC, Marsh JL, Grames LM, Pilgram TK. The effect of surgeon experience on velopharyngeal functional outcome following palatoplasty: is there a learning curve? Plast Reconstr Surg. 1998; 102(5):1375-84. 276. Woods JR, Saywell RM, Jr., Nyhuis AW, Jay SJ, Lohrman RG, Halbrook HG. The learning curve and the cost of heart transplantation. Health Serv Res. 1992; 27(2):219-38. 277. Wright SM, Daley J, Peterson ED, Thibault GE. Outcomes of acute myocardial infarction in the Department of Veterans Affairs: does regionalization of health care work? Med Care. 1997; 35(2):128-41. 278. Yao SL, Lu-Yao G. Population-based study of relationships between hospital volume of prostatectomies, patient outcomes, and length of hospital stay. J Natl Cancer Inst. 1999; 91(22):1950-6. 279. Young GJ, Charns MP, Desai K, et al. Patterns of coordination and clinical outcomes: a study of surgical services. Health Serv Res. 1998; 33(5 Pt 1):1211-36. 280. Zelen J, Bilfinger TV, Anagnostopoulos CE. Coronary artery bypass grafting. The relationship of surgical volume, hospital location, and outcome. New York State Journal of Medicine. 1991; 91(7):290-2. 281. Zimmerman JE, Shortell SM, Knaus WA, et al. Value and cost of teaching hospitals: a prospective, multicenter, inception cohort study. Critical Care Medicine. 1993; 21(10):1432-42.
Representative terms from entire chapter: