ing raw and risk-adjusted mortality rates, as well as the rankings, widely. HCFA’s aims in publishing comparative mortality performance data were to assist peer review organizations in targeting their Medicare quality oversight activities, to inform health care consumers so they could make better choices about their own care, and to help health care professionals improve the quality of the care they delivered (Krakauer et al., 1992).

A series of studies, including one analysis performed by the HCFA statistical team itself, evaluated the HCFA mortality reports against various gold-standard clinical measures (Green et al., 1990, 1991; Krakauer et al., 1992; Rosen and Green, 1987). Positive predictive value (Weinstein and Fineberg, 1980) for the HCFA mortality reports ranged from 25 to 64 percent. In other words, for every 12 hospitals labeled as “high mortality outliers,” at least 4 and as many as 9 were falsely identified. Among those hospitals judged to demonstrate excellent outcomes, as many as 3 in 5 were miscategorized (Green et al., 1991).

In 1993, Bruce Vladek, HCFA’s administrator, halted release of the Medicare mortality reports. He judged that the reports were having little impact on care delivery performance while continuing to generate controversy and consume significant resources (Vladek, 1991).

Over a decade has passed since HCFA (now the Centers for Medicare and Medicaid Services) produced the annual mortality reports described in this case study. The intervening years have allowed dispassionate reflection on the reasoning, methods, barriers, effects, and limitations associated with a national attempt to use clinical data to drive change in health care delivery. The Medicare mortality reports, updated by parallel consideration of similar, more recent efforts, can serve as a useful case study for understanding issues surrounding data standards for patient safety.

Selection of Measures

Outcome measures, although of keen interest to regulators, purchasers, and individuals, are particularly difficult to use for accountability purposes since they do not necessarily measure competence (Trunkey and Botney, 2001). The outcomes individuals experience are influenced by multiple factors, many of which are outside the control of the health care provider. Given the pejorative nature and potential professional and business consequences of outcome-based accountability systems, health care providers demand accurate, reliable rankings. Patients and patient representatives also require

The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement