National Academies Press: OpenBook

Performance Measurement: Accelerating Improvement (2006)

Chapter: Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier

« Previous: Appendix G Starter Set of Measures
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Appendix H
Commissioned Paper

Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions

Kyle L. Grazier

INTRODUCTION

This paper was commissioned by the Institute of Medicine (IOM) to provide an overview of “value-based” or efficiency measurement in health care. It will define selected terms; provide a brief history of the development of these measurement sets; assemble information on the efficiency measurement sets in current use; identify challenges to applying these in practice and research; and identify gaps in efficiency measurement.

DEFINITION OF EFFICIENCY

Central to this work is the manner in which “efficiency” and “value based” are defined. Among others, the economics, statistics, management science, and health services research literatures have contributed variations on these definitions that differ in their specificity to health care and their generalizability beyond the economic costs of health care services. Specifically, definitions differ as to whether the mix of inputs includes quality, and the mix of outputs includes health, health status, or mortality.

Economic efficiency is commonly expressed as the relationships between a given quantity and quality of output using a bundle of inputs that minimizes the cost of production. Several different combinations of capital, labor, and raw materials (where each of these can have multiple dimensions, e.g., physician labor, nurse labor, etc.), could feasibly be used as inputs to produce a particular quantity and quality of output. Generally, only one of these combinations will have the lowest cost associated with that input bundle.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Palmer and Torgerson’s (1999) definition of efficiency includes both health care inputs and health outcomes. The goals for measurement determine which aspect of efficiency is emphasized. They suggest that “allocative efficiency” should dictate policy decisions focused on resource distribution (Palmer and Torgerson, 1999). This aspect of efficiency requires that a specific outcome be defined in advance, after which a choice is made among alterative interventions or resources based on their relative costs. The resulting costs may not reflect the most efficient combination of inputs and outputs but it does allow for an allocation strategy. An example: If one is interested in promoting one of two surgical interventions, and the identified criteria for selection is a fixed minimum postsurgical mortality rate, then one can compare the relative costs of each to achieve a fixed mortality threshold.

To assess “productive efficiency,” one maximizes “health outcome for a given cost,” or minimizes “cost for a given outcome.” For example, one chooses different combinations of inputs to achieve the best health outcome for a given cost. “Technical efficiency” is achieved if the physical mix of labor and capital inputs achieves the maximum output. For instance, if surgical procedure A and surgical procedure B produce the identical defined outcomes of hospital discharge in 3 days, but procedure A uses less labor but identical amounts of capital, then procedure B is considered technically inefficient.

The measurement of the individual inputs and outputs in the efficiency function also vary by setting, goals, and the availability of data. The definition of costs or economic resources has been relatively consistent in services research: direct and indirect monetary resources that contribute to the institution’s costs of providing a service. However, as the goals of measurement change to incorporate an understanding of system resources, then the physician’s resource use is included, as are out-of-pocket direct costs, and even indirect costs of lost workplace productivity and reductions in general economic production. Such expansive cost constructs can inhibit practical solutions due to conceptual and data complexities. For the most part, this paper focuses on the service-related resource costs consumed in the delivery of medical care within the health care system.

Over a decade ago, the IOM defined quality as “the degree to which health services for individuals and populations increase the likelihood of desired health outcomes and are consistent with current professional knowledge” (IOM, 1990). But as many authors have noted recently, the definition of quality, as in quality care or quality improvement, has not reached national consensus (Berwick, 2002; McGlynn, 1995; McGlynn et al., 2003; McKee, 2001; Palmer and Torgerson, 1999; Wennberg et al., 2002). Complicating these efforts are the paucity of “gold standards” for health outcomes, definitive levels of health that are measurable, valid, and reliable.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Patient, population, and clinical characteristics introduce variations in outcomes. In addition, the choice of services and the process for delivering them have limited clinical evidence of their efficacy and effectiveness. Finally, deficits in management costing have limited the ability to measure accurately the resources consumed in the care delivery process and the quantitative outcomes.

In the discourse on performance measurement in health care, “efficiency” is used in many contexts and for many purposes. Policymakers at national, local, plan, and purchaser levels are deliberating how to maximize health-related outcomes of their enrollees, beneficiaries, or employees receiving services, while minimizing costs for a standard outcome. Maximizing efficiency or reducing expenditures may compete for attention with a target morbidity rate.

These challenges influence which measures of value and efficiency to evaluate or support; which methods to endorse for practitioners, services, and resources; and how to implement and integrate efforts to improve intermediate and longer-term population-, firm-, or patient-specific outcomes. Despite these many challenges, considerable effort has advanced thinking and action in the research and practice arenas.

While there is not yet consensus on the definition of “efficiency” or “value based,” this paper will incorporate both the Institute of Medicine (IOM) landmark report’s definition of efficiency (eliminating waste) and the theoretical economics definition of efficiency (IOM, 2001; Palmer and Torgerson, 1999). For these purposes, efficiency will be broadly defined as the mix of health care resource inputs that produce optimal quantity and quality of health and health care outputs. In short, the bias is toward measuring the production efficiency of relative health care resources among individual, institutional, and groups of providers.

It is important to note here that there are several current initiatives and programs to assess, improve, promote, and reward improvements in and delivery of quality health care (AHRQ, 2004; Bridges to Excellence, 2004; Kerr et al., 2004; Leapfrog Group, 2005). Other consortia of employers, purchasers, and health plans are planning programs to measure and reward institutional performance in effectiveness and efficiency (Leapfrog Group, 2005; PBGH, 2005; Worthington, 2004). Although this paper addresses the broader definition of efficiency to include “value,” and therefore, quality inputs and outcomes, no attempt will be made to discuss all measures of quality, performance, or effectiveness currently in use.

MOTIVATIONS FOR VALUE-BASED MEASURES AND MEASUREMENT

Policy makers, researchers, providers, and others are motivated to seek value-based or efficiency measures for various reasons. In the past two de-

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

cades, the quality of the available data and the rigor of the analysis have advanced our ability to measure the economic outputs that are derived from resource inputs. As a result, numerous health care institutions and researchers are willing to invest in value-based measurements, with a clear focus on quality-adjusted outcomes. Many purchasing groups, health plans, insurers, and consumer groups are at least as concerned, if not more so, with the cost-efficiency of services. Algorithms for assessing relative efficiency of providers vary in their transparency to the user, but are widespread among health plans and physician group practices. Outputs from these types of analyses trigger decisions on appointing and reappointing physicians within a practice or network; form the basis for monetary incentive packages for providers and groups; and generally are aimed at the containment and management of contract and practice costs of physicians delivering inpatient and outpatient, general and specialty care in solo, single-, or multispecialty practices.

The following purposes for efficiency measurement have been documented in the literature (Berwick, 2002, 2003; Fiscella et al., 2000; Franks et al., 1993; Galvin and McGlynn, 2003; Iezzoni et al., 1992b, 1994a; IOM, 1990; Kerr et al., 2004; Leatherman et al., 2003; McGlynn, 2003a,b; McGlynn and Brook, 2001; McGlynn and Halfon, 1998; McGlynn et al., 2003; Nauert, 1996; NCQA, 2004; Shahian and Normand, 2003; Schield et al., 2000; Siu et al., 1992).

While extensive, the list is not exhaustive:

  • Improve quality of care

  • Encourage payer involvement

  • Integrate responsibility for employment, payment, health status

  • Reduce waste

  • Re/appoint/certify medical staff for network participation

  • Increase financial risk associated with practice decisions

  • Alter practice patterns

  • Assist in cost containment

  • Encourage/steer selection of efficient health plans

  • Allocate service resources differently

  • Deploy alternative labor and capital

  • Track/evaluate relationships to health management, health status, survival

MEASUREMENT CONSIDERATIONS

Validity

There are generic guidelines for selecting measurement criteria, not all of which can be met in the current efforts to measure efficiency. Regardless

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

of the goals for measuring efficiency, the measure used for efficiency or value must be valid. Unfortunately, gold standards for health care efficiency don’t exist, complicating efforts to establish the validity and reliability of a measure.

Surrogates for validity in measuring practice efficiency include the notion of “accuracy” of the programs and “consistency” or “stability” across practices and providers (Thomas et al., 2004b). Technical accuracy is highlighted by holding constant an outcome, and comparing inputs, namely costs, across physicians of the same specialty. By varying the methods used in measuring the inputs, and comparing the consistency of the outputs, production efficiency is captured. By establishing the “stability” of the output measure over time, over different types of physician specialties and patient panel sizes, one can learn more about potential variation in the inputs and outputs, and the financial and health consequences.

Unit of Analysis

Currently, the majority of practice efficiency measurement tools rely on the physician as the unit of analysis, rather than the physician group, individual patient, or community member. The purpose of this physician-focused measurement is to establish the economic resources consumed by the physician in the delivery of care, relative to physician peers. The visit, service, or case descriptors attempt to bundle patient and clinical care characteristics into discrete, homogeneous categories. These categories are then used to help define the services a patient might expect to receive when presenting with the characteristics defined by a particular resource category (Franks and Fiscella, 2002; Franks et al., 2003). However, there is still considerable variation in which variables contribute to “case” categories and resource use, and the algorithms for assigning the costs of those resources to providers.

Attribution of Resource Use

Attributing patient-specific resource use to an individual physician is particularly complicated when services are delivered as part of a team of providers, over an extended period of time, for complex or persistent conditions.

Under a gatekeeper model, primary care physicians are held responsible for all services delivered, whether provided by the physician, referred to another approved physician, or provided by other clinical staff within the practice. Although the underlying risk-sharing arrangement within a primary care practice may not be known, many efficiency tools assume that all consumed resources can be attributed to the primary care physician. When evaluating the resources used by nonprimary care physicians, or “special-

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

ists,” attributing responsibility for services across the providers is usually based on a formula. These formulas differ in their attribution decision rules, and vary the amounts of resources assigned to a responsible provider proportionally or nonproportionally to the primary care, nonprimary care, or total resources consumed across the episode.

Data

The data sources for these efforts have traditionally included encounter and claims data supplied through an employer, insurer, or plan’s administrative data systems. In some cases, the administrative data have been validated against medical records, but these efforts have been inconclusive in determining which source is better than another for these purposes (Hannan et al., 2003). Claims or encounter data at this time are generally more accessible and less expensive to analyze than medical charts or patient surveys, although efforts to identify quality and value metrics continue to explore these sources as well as electronic medical records and online order entry systems (Birkmeyer et al., 1999, 2002, 2003; Fisher et al., 1990a,b, 1992; Malenka et al., 1994; Thomas et al., 2004a).

Different types and amounts of data can be extracted from the same claims data set (Baron et al., 1994; Fisher et al., 1992). Many profiling tools capture and use in their algorithms different numbers of diagnoses, procedures, and different time periods for services. Current episoding algorithms vary in the numbers of episode categories to which diagnoses and procedures are assigned. They also differ in the length of the “clean periods,” those time periods during which no services for the condition are received, thus triggering the end of one episode and the beginning of another.

It is common in profiling methods to aggregate all costs of care that appear with an episode and attribute this total to a provider. But there is also variation in the complexity or severity of the case or in patient characteristics that are not captured in episode categories defined by time of service (Iezzoni et al., 1992a,b, 1994b). Several risk adjustment methods that have been perfected for other purposes as well as for physician efficiency profiling are applied to episodes to explain better the resources identified as inputs in the model.

Risk Adjustment

Risk adjustment is used to adjust claims profiles to account for differences in the health status (and thus expected resource use) of patients served. Without proper adjustment, practice patterns of physicians whose patient panels include greater than average proportions of elderly patients or

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

patients with severe or chronic disease could appear, incorrectly, to reflect inappropriately high levels of resource use (such as office visits, ancillary services, prescription medications, specialist referrals, and hospital days). Different risk-adjustment methodologies—all purporting to “do” the same thing—can produce quite different results. Research on hospital profiling demonstrates that comparative judgments about provider performance can be influenced significantly by the specific risk measurement methodology utilized (Iezzoni, 1997).

There are several models of risk adjustment that have been tested over time and on various data sets. The vast literature reflects the range of purposes, data sources, algorithms, analytic models, and outputs associated with risk adjustment methods (Thomas et al., 2004a). Researchers and policy makers see a growing role for risk adjustment payment models, financing policies, and performance measurement. Patient interviews, surveys, claims records, medical records, or some combination of these have been suggested as sources for data on health or medical risk (Ash et al., 2001; Grazier and Thomas, 2002; Hornbrook and Goodman, 1996; Newhouse et al., 1997; Pope et al., 2004; Street, 2003; Worthington, 2004; Zhao et al., 2001). The costs associated with collection are weighed against the quality and volume of the information from each source.

There are many physician profiling and efficiency tools based solely on administrative data, although even in these cases there are significant differences in the data fields used in the algorithms that define risk categories; models may include age, sex, one or more primary, principal, or secondary procedures and diagnoses, and pharmacy National Drug Codes (NDC) (Ash et al., 2001; de Brantes, 2002; de Brantes et al., 2003; Goldman et al., 2004; Grazier and Thomas, 2002; Pope et al., 2004; Thomas et al., 2002; Worthington, 2004; Zhao et al., 2001). Many efficiency-profiling packages also require specific record layouts and field definitions.

Resource Costs

Service or resource costs used in efficiency measurement are seldom collected from institutional management accounting processes; instead they rely on the monetary data appearing in the claim record; these include paid charges, allowable charges, or relative value adjusted charges. In some cases, to remove the effects of price variations in the reported charges, charges are standardized to a regional or local mean value for similar procedures or practices. In cases in which detecting price variations and their impact on practice is central to the profiling effort, actual recorded paid charges are used without standardizing for market differences.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Thresholds

Physician profiling tools assess the extent to which “costs” of the resources used for an individual type of patient or panel of patients exceeds a predetermined percentile, a group-specific median or mean, a national specialty group consensus level, other national benchmarks, or a relative value based on annual budgets or financial targets. Patient or episode cost outliers can influence many of the algorithms for assessing efficiency. Case outliers are often examined separately from the pool to determine what factors affect their occurrence. The width of the threshold bands determines in part how stable efficiency rankings are over time and across specialties.

Outputs

The output from efficiency measurement for individual physicians is most commonly the ratio of the observed costs to the expected costs (Thomas et al., 2004a,b). The closer a physician comes to using (spending) resources at levels expected for the clinical risk of the patient or panel of patients, the more efficient he or she is assumed to be. While use of the observed/expected cost ratio is prevalent, users should be cautious when applying the ratio to physicians with small patient panels, since misclassification is in many cases related to panel group size. Use of a measure of the difference between the standardized expected costs and the standardized observed costs for a patient or panel could dampen this small sample bias.

MEASURES OF “VALUE-BASED” METRICS (EFFICIENCY MEASURES)

In 2003, the National Quality Forum (NQF) endorsed national voluntary consensus standards for hospital care performance measures. The initial 39 measures were “intended to promote both public accountability and quality improvement.” The Institute for Healthcare Improvement, through several programs and as described in several white papers as part of their innovation series, has initiated efforts among hospitals to improve the outcomes and experiences of patients and providers on medical/surgical units. Although not specifically designed to measure efficiency, they promote the potential increased value to patients and providers through use of the measures (Institute for Healthcare Improvement, 2005). The IOM, NQF, the Agency for Healthcare Research and Quality (ARHQ), and the National Committee for Quality Assurance (NCQA) singularly and as part of consortia have produced topics, criteria, and measures for clinical conditions and priority areas for health care quality improvement activities (AHRQ,

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

2004; IOM, 2005; NCQA, 2004; NQF, 2005). These works continue to contribute measures of quality into the value-based efficiency measurement equation. The report on measuring provider efficiency, a collaborative effort of the Leapfrog Group and the Bridges to Excellence, notes “reporting performance on efficiency should be linked to reporting performance on quality to better understand, measure and communicate the value that is delivered by physicians and hospitals” (Bridges to Excellence, 2004; NCQA, 2004).

Other organizations and sponsors have begun or are considering using data collected for earlier purposes, such as quality measures for accreditation or internal monitoring, for value measurement. The NCQA monitors health plan performance by collecting and analyzing the Health Plan Employer Data and Information Set. As noted earlier, it has convened technical panels to design efficiency measures for implementation among member health plans. The AHRQ is providing guidance based on its own research as to how best to use the quality indicators that they make publicly available for performance and potentially efficiency measurement (Remus and Irene, 2004). The Joint Commission on Accreditation of Healthcare Organizations (JCAHO) is considering reporting some of its measures collected during accreditation processes. AHRQ reports “JCAHO will be replacing hospital performance reports with quality reports in 2004.”

Table H-1 presents some of the measures of value and efficiency that have been proposed or are in use either by or under the sponsorship of several of the above-named organizations. Few of the existing measures endorsed by national organizations are specifically for measuring efficiency; however, some programs are included if they noted in their documentation their preparations for expansion of quality measurement to “efficiency” or “value.”

GAPS IN THE LITERATURE AND EMPIRICAL WORK

Health care value can be viewed as a set of individual and conflated components (e.g., quality, cost, population health, clinical measurement, payment methods, practice patterns, and delivery system). The dynamic nature of the research in each of these areas leads to frequent, important contributions. Recent advances stimulate efforts to identify and fill the gaps remaining in our knowledge of value-based metrics, and in related policies and practices. Several remaining challenges are being addressed in demonstrations, experiments, and practice; some have not yet been rigorously examined; and many remain ripe for rigorous study.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Standardization in the Measures Used to Assess Efficiency

Standardization has been a necessary step in the advancement of numerous technologies and in improvements in production. The need for standardized measures of quality, effectiveness, and efficiency has been documented extensively.

Most commercial products on the market and many of those in development by NCQA and others measure efficiency by comparing actual observed expenses with expected expenses incurred in the delivery of services. In some cases, the effect of prices is removed. The price-adjusted (or standardized) measure assumes that “paid amount” noted on claims reflects volume, type of services, and price. Unless the intent was to assess the impact of price variation on provider efficiency, the ratio of observed to expected costs would be standardized to remove this variation. To accomplish this, standard or average regional prices for similar services are applied to the services data. Recent studies have recommended that both price-adjusted and unadjusted observed versus expected costs be measured and compared with one another (Bridges to Excellence, 2004; Leapfrog Group, 2005; Thomas et al., 2004a). NCQA efforts to create an efficiency indicator for health plans include examining both standardized and unadjusted efficiency measures, to understand better the extent of variation in outcomes due to regional or price differences.

Physicians are obvious stakeholders in the standardization of these measures, and many complain that efficiency performance is being measured and interpreted differently within and across health plans, insurers, health systems, and consumer groups. Policymakers must consider the cost to the plans or practices of imposing one particular episoding and/or risk adjustment commercial product, rather than specifying standardized input and output measures. Transparency in methods and algorithms aids evaluation of the logic and components that could and should contribute to a standard. To advance understanding and promote progress in standard setting, product details need to be revealed; examples of information needed for this purpose include: the underlying logic and processes of the algorithms used for preparing data for the application, and for episoding and risk adjustment; standard errors and statistical significance of output measures; outlier threshold levels; frequency and types of omitted cases; total member panel size and number of valid episodes per physician per time period; and the attribution method used within specialty and across specialties.

Inclusion of Quality Dimensions in the Measures

Significant progress has been made in identifying process and outcomes components of quality care, particularly for certain conditions treated in

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

certain settings. Experts in clinical care and measurement recommend that recently piloted processes be expanded and that current larger-scale empirical work be tested on other samples and in other venues.

For instance, clinical quality measures for diabetes care and heart/stroke care included in Bridges to Excellence/NCQA Provider Recognition Programs are available for use in assessing efficiency performance (Tom Lee, personal communication, 2004). The End of Life metrics developed by the Dartmouth Atlas team (Wennberg et al., 2002) have been proposed as a proxy for hospital system efficiency (Eugene Nelson, personal communication, 2004). Active research programs and demonstrations by the NQF, the NCQA, Bridges to Excellence, the Leapfrog Group, research groups, and others are rapidly advancing the measurement of quality using medical records and administration data. These efforts need to be shared and combined on an ongoing basis into the measurement of health care value.

Validated Clinical (Medical Service, Pharmacy) and Financial Data

A number of studies have examined the validity of self-reported data, medical records, and administrative data and found that, with some caveats, claims data are adequate for many purposes related to value measurement. Although recent, these studies may not be generalizable to future information systems in which the electronic medical record, integrated services/encounter data, and advanced cost accounting systems are the norm. Concurrent with efforts to measure efficiency and performance are demonstrations and experiments in facility-based standardized records and information systems that can form the basis for reliable measurement of services, quality, and providers across sites and health systems (Physician Practice Connections for the Bridges to Excellence rewards program, Physician Office Link, the product of a collaboration between NCQA and Bridges to Excellence).

Although these efforts will undoubtedly lead to important answers and recommendations, ongoing empirical work should include sampling and analysis of:

  1. medical records for office visits and inpatient stays to validate data that appear on and are extracted from claims-based files and similar administrative records;

  2. cost data collected from multiple sources, including facility-specific, payer-specific records of billed charges, allowed charges, paid charges, and retroactive adjustments to assess the validity of resource measures;

  3. physician or group panel member characteristics including age, sex, race/ethnicity, and zipcode (to measure average socioeconomic status) relative to the service area or plan population. This can serve several purposes.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

It would allow for relative assessments of a provider’s practice case mix, which differs from the case or severity mix of treated patients, and plays an important role in determining the efficiency of a provider with a “sicker” panel of patients versus a sicker panel of “potential” patients. Researchers have made a strong case recently for the relevance of these characteristics to patterns of use, treatment, costs, and outcomes.

To ensure that data provide information on the persons who use services and those who do not, several population-based characteristics and patient- and service-specific data elements are needed, sometimes from several sources.

There are facility- and service-specific standardized forms and conventions for data collection. These include the UB-92 and HCFA 1500 forms for inpatient and professional services, and procedure, diagnosis, and pharmacy coding schemes (CPT, HCPCC, NDC, ICD) for clinical services delivered. What appears on claims records and what is extracted from them as part of measurement algorithms can differ across claims administrators, payers and product designers. Provider characteristics, including specialty and details on physicians’ panels, referrals patterns, and physician payment algorithms are normally not readily available from administrative data sources. Cost data are also collected and presented variably in claims records and billing forms, requiring scrutiny of the definitions of data elements and the cost adjustment processes used by systems administrators. Validation studies are required prior to using these different data sources.

Multiyear, Multisystem, Linked, Population-Based Data that Captures:

  • Acute and chronic care episodes

  • Pharmacy data

  • Population characteristics (age, sex, race/ethnicity)

  • Provider characteristics

  • Service delivery and payment model (FFS, HMO, PPO, POS, etc.)

Due to temporal variation in services delivery, claims reporting, episode construction, and services utilization, measuring value requires longitudinal data for several units of analysis. It also requires the capacity to link the units through a unique personal (e.g., member, patient, or provider) identifier. Although there is no published research on the optimal time frame for collecting physician experience data to ensure validity of the performance measures, actuarial models of medical care utilization indicate the importance of at least two years of claims experience to estimate with moderate confidence future utilization behavior.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

While one year of claims data may allow for detection of acute care episodes, it may omit lengthy or complex episodes, particularly if the profiling algorithm truncates those cases that show no end date within the contract year or capture only episodes with clean periods at both ends.

One year of data is also likely to omit those patients who consistently incur high costs from year to year, whether because of severe and persistent illnesses, or due to high-frequency moderately resource-intensive service needs. Analysis of three years of claims and exposure data from the Society of Actuaries medical claims study (Grazier and G’Sell, 2004) indicates that for claimants with annual claims expense of more than $25,000, over 13 percent have annual claim costs in the subsequent two years of over $25,000; for those with annual claims in one year exceeding $50,000, almost 25 percent have total annual claims exceeding $25,000 in the subsequent year; and for those with annual claims cost exceeding $100,000, over 30 percent have claims exceeding $25,000 in subsequent years. While these data are for patients and not per physician, the effect of such cases on a panel from one year to the next could be misinterpreted if multiple years of data were not captured in the algorithms.

More than one year of data would be needed to establish a fuller picture of use, and to accommodate “clean periods” for episodes that span the limits of inforce coverage contracts or reflect care for chronic conditions. In the White Paper released by Bridges to Excellence (2004), authors recommended “at least two years of data, based on incurred claims” to “develop a statistically reliable determination of provider efficiency.”

Recent research on measuring efficiency and quality has used administrative claims and member data either from commercial carriers or employers, or beneficiary claims data from fee-for-service Medicare. Because of the different payment models reflected in these data sets, care should be taken to ensure internal and external consistency. Within commercial population data, health mainenance organziation (HMO), exclusive provider organization, preferred provider organization (PPO), and traditional indemnity covered care may be captured differently. For instance, HMO encounter data may not incorporate professional fees with the inpatient/hospital records, while traditional-coverage-generated data may have both. Commercial claims data cannot be directly combined with Medicare data, without adjusting for beneficiary, coverage, and charge differences across the payers.

Most claims systems used by commercial carriers or those developed in-house separate pharmacy data systems. If quality is to be incorporated into efficiency measurement, then pharmacy data should be incorporated into the measurement and assessment of the appropriateness of resources (Goldman et al., 2004). If it were combined, then pharmacy data can be edited and aggregated and then linked by unique member identifiers across commercial data sets. If comparable pharmacy data are not available, such as in

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Medicare claims, then pharmacy data should be removed from both sets prior to combination for analysis. Most large employers are requiring their third-party administrators or their health coverage carriers to collect and link pharmacy with medical claims information for analysis.

Identifying Validity of Measures Across Different Physician Specialties

  • Primary care

  • Inpatient specialty

  • Ambulatory care, doctor office/group specialty

Very few studies are available to inform the use of efficiency metrics for different physician specialties. Primary care providers have been the subjects of physician cost profiling algorithms for almost a decade. Several vendor products are available for specialist profiling, using similar methods as those used for primary care. One study examined the reliability of different profiling algorithms for different types of specialists, and cautioned policymakers in using the outputs from such algorithms. More recent recommendations include evaluating only those physicians who are responsible for a fixed proportion of cases, and for whom peer specialists are available within the system or region for comparison. While national benchmarks are often used to compare specific results, transparency in those benchmarks is necessary to determine their appropriateness for these purposes.

Results of research to date have been confounded by complexities in capturing the underlying referral and payment allocation mechanics of plans or practices. It has also been limited in many cases to cost efficiency measurement, and not necessarily value.

Attribution of Resource Use to Provider, Site, Patient, Geographic Unit

  • Fractional vs. fixed attribution methods

  • Inpatient vs. outpatient

  • Individual provider vs. team vs. health system

There are few studies that systematically examine the impacts of using different thresholds of responsibility for resources consumed. In one study, total resources were assigned proportionately to participating specialists if the specialist was responsible for 30, 50, or 80 percent of the total resources used. The selected threshold for inclusion obviously limits the numbers of physicians that can be measured. It also influences the apparent efficiency of the provider.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

In the 2004 White Paper (Andrianos and Stam, 2004; Bridges to Excellence, 2004; Leapfrog Group, 2005), “Episodes were attributed to providers who had the highest professional claims within an episode, exceeding a minimum threshold for 25 percent of eligible clinician fees (eligible clinician fees included all professional fees, excluding hospital based physicians).” This was based on analysis performed by the authors to alter the attribution rules in increments of 10 percent, from 0 percent to 100 percent for the “minimum portion of total professional dollars in the episode required to be delivered by the attributed clinician.” They concluded, “very little data is lost as the threshold changes from 0 percent to 30 percent, whether we consider episode volume, number of attributed clinicians, or total dollars. In this large sample of commercial claims, more than 88 percent of all episodes featured only one clinician eligible to be the attributed clinician.”

More research is required on attribution threshold and distribution methods.

Consensus on the Basis for Selection of Service or Provider for Measurement

  • Most prevalent conditions treated

  • Highest cost episodes

  • Highest volume of episodes

  • Highest total costs

Research on quality and efficiency has progressed at different rates for different dimensions of value, and for different types of facilities, practitioners, and diseases. Users of value-based metrics may have different goals for their use.

Measurement of processes and outcomes associated with quality care for patients with certain conditions, such as cardiovascular disease, diabetes, and Chronic Obstructive Pulmonary Disease (COPD), is highly advanced. Adding cost and efficiency dimensions to that research may expedite our understanding of the potential for these types of approaches. Current purchaser-based initiatives on pay for performance may lend themselves to additional study of the incremental value and cost of using alternative algorithms to assess provider relative efficiency.

In addition to the many goals driving selection of the unit of measurement, there are also likely to be changes in the quality of the data available for analysis. How we measure and to whom we attribute resources may change as cost data improve in quality and availability within and across integrated systems.

Given that no consensus across stakeholders exists and that no one dominant basis for selecting physicians has been established, value-based

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

measurement research should continue to study the marginal value of applying these metrics to current alternatives: primary care or other specialty physicians; inpatient and ambulatory care or office-based services; on the basis of disease or condition prevalence, panel health risk, total costs, or attributable costs.

Risk-Adjustment Methods

  • Episode-based measures

  • Encounter-based measures

  • Provider-based measures

Episode grouping algorithms are integral to several existing commercial vendor and public sector products, as well as products and processes under development. The empirical support and logic behind this approach to understanding the package of resources used to treat a patient with certain conditions has positioned episode systems as superior to other alternatives. However, further analysis is necessary to compare episoding algorithms, including the use and length of “clean periods” for different conditions; the parsing of clinician conditions into episodes; and the effect of delivery system and payment method on resources assigned per episode.

By risk-adjusting episodes, total resource use can be considered in light of clinical condition and severity. Risk adjustment has received considerable attention in the literature; however, no dominant clinical risk-adjustment system has surfaced for episodes or non-Medicare cases. Research is still needed on the optimal method for determining clinical risk as it relates to the quality and efficiency of services and for adjusting for it using valid and reliable methods. Trade-offs among methods that utilize different or increasing numbers of variables from multiple data sources need to be made explicit.

Consensus in the Principles Behind and the Goals of Value-Based Measurement

The large number of stakeholders with interests in value-based metrics forces policymakers to recognize and prioritize the goals of such measurement. A consensus can streamline decisions on choice of methods and measures. For instance, agreement that high-quality, efficient allocation of resources to the public demands that value-based methods include measure of population health status. As another example, consensus as to the importance of the principle of fairness in the application of these metrics across plans, providers, and over time implicitly imposes a commitment to evaluate the consistency of the processes and the validity of the measures.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

More broadly, the process of sharing values and selecting priorities through multiple stakeholder discussions can more rapidly integrate lessons learned and promote progress toward multiple goals.

CONCLUSION

The goals of value-based health care measurement are to improve practice, ensure high-quality care, and reduce underuse, overuse, and misuse of health care resources. Methods are available that permit identification of many primary care and other specialists who treat the most prevalent illnesses, the highest cost caseload, and the highest volume of services delivered. Administrative data are sufficiently ubiquitous to provide a ready palette for careful analysis when internal service records are not adequate or available. Multiple years of linked data improve identification of full episodes of care, evaluation of chronic care delivery models, and reliability of patient or member risk levels. Risk adjustment methods continue to be refined and evaluated. Efficiency metrics are still under development and testing, requiring some redundancy in use. Both standardized and nonstandardized observed costs should be further modeled.

It is critical when using any method that identifies or ranks the most or least efficient physicians or hospitals that those using these systems understand the underlying practice philosophy and service system within which the provider operates. If services were delivered by primary care teams, through care managers, or under indemnity or other insurance models in which patients have more choice in supply sensitive services, then profiling algorithms either must reflect these variations or highlight inputs and metrics for additional attention.

Further research and demonstrations on these and other features of a value-based metric system are under way.

REFERENCES

AHRQ (Agency for Healthcare Research and Quality). 2004. What Consumers Say About the Quality of Their Health Plans and Medical Care, National CAHPS® Benchmarking Database 2003, Chartbook Volume 1: Composites and Ratings. Rockville, MD: AHRQ.

Andrianos J, Stam D. 2004. Study of Professional Episode Attribution Thresholds Summary: Vast majority of episodes involve only one managing clinician. [Online]. Available: http://www.regence.com/research/docs/professionalEpisodeAttributionStudy.pdf [accessed November 9, 2004].

Ash A, Yang Z, Randall PE, Kramer MS. 2001. Finding future high-cost cases: Comparing prior cost versus diagnosis-based methods. Health Services Research 26(6):194–206.


Baron JA, Lu-Yao G, Barrett J, McLerran D, Fisher ES. 1994. Internal validation of Medicare claims data. Epidemiology 5(5):541–544.

Berwick DM. 2002. A user’s manual for the IOM’s “Quality Chasm” report. Health Affairs 21(3):80–90.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Berwick DM. 2003. Improvement, trust, and the healthcare workforce. Quality and Safety in Health Care 12(Suppl.1):i2–i6.

Birkmeyer JD, Lucas FL, Wennberg DE. 1999. Potential benefits of regionalizing major surgery in Medicare patients. Effective Clinical Practice 2(6):277–283.

Birkmeyer JD, Siewers AE, Finlayson EV, Stukel TA, Lucas FL, Batista I, et al. 2002. Hospital volume and surgical mortality in the United States. New England Journal of Medicine 346(15):1128–1137.

Birkmeyer JD, Stukel TA, Siewers AE, Goodney PP, Wennberg DE, Lucas FL. 2003. Surgeon volume and operative mortality in the United States. New England Journal of Medicine 349(22):2117–2127.

Bridges to Excellence (BTE). 2004. Measuring Provider Efficiency. A Collaborative multi-stakeholder effort. Version 1.0.


de Brantes FS. 2002. Bridges To Excellence: A program to start closing the quality chasm in healthcare. Journal of Healthcare Quality 24(2):2–11.

de Brantes FS, Galvin RS, Lee T. 2003. Bridges to Excellence: A business case for quality. Journal of Clinical Outcomes Management 10(8):431–438.


Fiscella K, Franks P, Gold MR, Clancy CM. 2000. Inequality in quality: Addressing socioeconomic, racial, and ethnic disparities in health care. Journal of the American Medical Association 283(19):2579–2584.

Fisher ES, Baron JA, Malenka DJ, Barrett J, Bubolz TA. 1990a. Overcoming potential pitfalls in the use of Medicare data for epidemiologic research. American Journal of Public Health 80(12):1487–1490.

Fisher, ES, Malenka DJ, Wennberg JE, Roos NP. 1990b. Technology assessment using insurance claims: Example of prostatectomy. International Journal of Technology Assessment in Health Care 6(2):194–202.

Fisher ES, Whaley FS, Krushat WM, Malenka DJ, Fleming C, Baron JA, et al. 1992. The accuracy of Medicare’s hospital claims data: Progress has been made, but problems remain. American Journal of Public Health 82(2):243–248.

Franks P, Fiscella K. 2002. Effect of patient socioeconomic status on physician profiles for prevention, disease management, and diagnostic testing costs. Medical Care 40(8):717–724.

Franks P, Nutting PA, Clancy CM. 1993. Health care reform, primary care, and the need for research. Journal of the American Medical Association 270(12):1449–1453.

Franks P, Fiscella K, Beckett L, Zwanziger J, Mooney C, Gorthy S. 2003. Effects of patient and physician practice socioeconomic status on the health care of privately insured managed care patients. Medical Care 41(7):842–852.


Galvin RS, McGlynn EA. 2003. Using performance measurement to drive improvement: A road map for change. Medical Care 41(Suppl. 1):I48–I60.

Goldman D, Joyce GF, Escarce JJ, Pace JE, Solomon MD, Laouri M, Landsman PB, Teutsch SM. 2004. Pharmacy benefits and the use of drugs by the chronically ill. Journal of the American Medical Association 291(19):2344–2350.

Grazier KL, G’Sell WA. 2004. Group Medical Insurance Claims Database Collections and Analysis. Schaumburg, IL: Society of Actuaries.

Grazier KL, Thomas JW. 2002. A Comparative Evaluation of Risk-Adjustment Methodologies for Profiling Physician Practice Efficiency. A report to the Robert Wood Johnson Foundation.


Hannan EL, Doran DR, Rosenthal GE, Vaughn MS. 2003. Provider profiling and quality improvement efforts in coronary artery bypass graft surgery: The effect on short-term mortality among Medicare beneficiaries. Medical Care 4(10):1164–1172.

Hornbrook M, Goodman M. 1996. Chronic disease, functional health status and demographics: A multi-dimensional approach to risk adjustment. Health Services Research 31(3):283–307.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Iezzoni LI. 1997. The risks of risk-adjustment. Journal of the American Medical Association 278:1600–1607.

Iezzoni LI, Foley SM, Daley J, Hughes J, Fisher ES, Heeren T. 1992a. Comorbidities, complications, and coding bias. Does the number of diagnosis codes matter in predicting in-hospital mortality. Journal of the American Medical Association 267(16):2197–2203.

Iezzoni LI, Foley SM, Heeren T, Daley J, Duncan CC, Fisher ES, et al. 1992b. A method for screening the quality of hospital care using administrative data: Preliminary validation results. QRB Quality Review Bulletin 18(11):361–371.

Iezzoni LI, Daley J, Heeren T, Foley SM, Fisher ES, Duncan C, et al. 1994a. Identifying complications of care using administrative data. Medical Care 32(7):700–715.

Iezzoni LI, Daley J, Heeren T, Foley SM, Hughes JS, Fisher ES et al. 1994b. Using administrative data to screen hospitals for high complication rates. Inquiry 31(1):40–55.

Institute for Healthcare Improvement. 2005. [Online]. Available: http://www.ihi.org/IHI/ [accessed November 15, 2004].

Institute of Medicine (IOM). 1990. Medicare: A Strategy for Quality Assurance, Vol. II. Washington, DC: National Academy Press.

IOM. 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.

IOM. 2005. Pathways to Better Health Services: Measuring Quality. Washington, DC: The National Academies Press.


Kerr EA, McGlynn EA, Adams J, Keesey J, Asch SM. 2004. Profiling the quality of care in twelve communities: results from the CQI study. Health Affairs 23(3):247–256.


Leapfrog Group. 2005. The Leapfrog Group. [Online]. Available: http://www.leapfroggroup.org/home [accessed November 2, 2005].

Leatherman ST, Hibbard JH, McGlynn EA. 2003. A research agenda to advance quality measurement and improvement. Medical Care 41(Suppl. 1):I80–186.


Malenka DJ, McLerran D, Roos N, Fisher ES, Wennberg JE. 1994. Using administrative data to describe casemix: A comparison with the medical record. Journal of Clinical Epidemiology 47(9):1027–1032.

McGlynn EA. 1995. Quality assessment of reproductive health services. Western Journal of Medicine 163(Suppl. 3):19–27.

McGlynn EA. 2003a. Introduction and overview of the conceptual framework for a national quality measurement and reporting system. Medical Care 41(Suppl. 1):I-1–I-7.

McGlynn EA. 2003b. Selecting common measures of quality and system performance. Medical Care 41(Suppl. 1):I-39–I-47.

McGlynn EA, Brook RH. 2001. Keeping quality on the policy agenda. Health Affairs 20(3): 82–90.

McGlynn EA, Halfon N. 1998. Overview of issues in improving quality of care for children. Health Services Research 33(4 Pt. 2):977–1000.

McGlynn EA, Cassel CK, Leatherman ST, DeCristofaro A, Smits HL. 2003. Establishing national goals for quality improvement. Medical Care 41(Suppl. 1): I-16–I-29.

McKee M. 2001. Measuring the efficiency of health systems. The world health report sets the agenda, but there’s still a long way to go. British Medical Journal 323(7308):295–296.


Nauert R. 1996. The quest for value in health care. Journal of Health Care Finance 22(3):52–61.

NCQA (National Committee for Quality Assurance). 2004. State of Health Care Quality. Washington, DC: National Committee for Quality Assurance.

Newhouse JP, Beeuwkes Buntin M, Chapman JD. 1997. Risk adjustment and Medicare: Taking a closer look. Health Affairs 16(5):26–43.

NQF (National Quality Forum). 2005. National Priorities for Healthcare Quality Measurement and Reporting. [Online]. Available: http://www.qualityforum.org/webprioritiespublic.pdf [accessed January 19, 2005].

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Palmer S, Torgerson DJ. 1999. Economic notes: Definitions of efficiency. British Medical Journal 318(7191):1136.

PBGH (Pacific Business Group on Health). 2005. Value Based Purchasing. [Online]. Available: http://www.pbgh.org/programs/value_based_purchasing.asp [accessed October 24, 2005].

Pope GC, Kautter J, Randall PE, Ash AS, Ayanian JZ, Iezzoni LI, Ingber MJ, Levy JM, Robst J. 2004. Risk adjustment of Medicare capitation payments using the CMS-HCC model. Health Care Financing Review 25(4):119–141.


Remus D, Irene F. 2004. Guidance for Using the AHRQ Quality Indicators for Hospital-level Public Reporting or Payment . [Online]. Available: http://www.qualityindicators.ahrq.gov [accessed October 26, 2005].


Schield JM, Bolnick HJ, Murphy JJ. October 2000. Evaluating Managed Care Effectiveness: A Societal Perspective. Paper presented to the Society of Actuaries, Schaumburg, IL. [Online]. Available: http://www.soa.org/ccm/content/?categoryID=1079102 [accessed January 10, 2005].

Shahian DM, Normand SL. 2003. The volume-outcome relationship: From Luft to Leapfrog. Annals of Thoracic Surgery 75(3):1048–1058.

Siu AL, McGlynn EA, Morgenstern H, Beers MH, Carlisle DM, Keeler EB, et al. 1992. Choosing quality of care measures based on the expected impact of improved care on health. Health Services Research 27(5):619–650.

Street A. 2003. How much confidence should we place in efficiency estimates? Health Economics 12(11):895–907.


Thomas CP, Wallack SS, Lee S, Ritter GA. 2002. Impact of health plan design and management on retirees’ prescription drug use and spending, 2001. Health Affairs Suppl Web Exclusives:W408–W419.

Thomas JW, Grazier KL, Ward K. 2004a. Comparing accuracy of risk-adjustment methodologies used in economic profiling of physicians. Inquiry 41(2):218–231.

Thomas JW, Grazier KL, Ward K. 2004b. Economic profiling of primary care physicians: Consistency among risk-adjusted measures. Health Services Research 39(4 Pt. 1):985–1003.


Wennberg JE, Fisher ES, Skinner JS. 2002. Geography and the debate over Medicare reform. Health Affairs Supp Web Exclusives:W96–W114.

Worthington AC. 2004. Frontier efficiency measurement in health care: A review of empirical techniques and selected applications. Medical Care 61(2):135–170.


Zhao Y, Randall PE, Ash AS, Calabrese D, Ayanian J, Slaughter JP, Weyuker L, Bowen B. 2001. Measuring population health risks using inpatient diagnoses and outpatient pharmacy data. Health Services Research 26(6 Pt. 2):180–193.

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

TABLE H-1 “Value-Based” and Efficiency Metrics

Measures

Definition: Input:Output

Stated Purpose/Function

Health Care Setting

Disease-specific (e.g., CVD) cost-episodes per person

Person or patient annual episode specific costs for CVD services: health-related process or outcome measures

Measure guideline concordance; aggregate resources consumed; attribute resource use to provider; compare across physician groups; pay for performance

Acute care hospital

Agreement between pairs of efficiency rankings using the weighted kappa statistic

Relative practice efficiency rankings

Physician efficiency profiling; nine clinical specialties selected, based on numbers of episodes managed, numbers of physicians in the specialty, and whether the specialty was medical or surgical

Mixed group model/IPA HMO

NCQA plan efficiency measurement

Relative resource consumption across plans

Measure and report relative resource consumption, risk adjusted for underlying population risk

Health plans

Process and outcome measures related to transitional care (across settings) (Mary Naylor)

 

Multiple settings: home care, hospital, ED, nursing home

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Required Enhancements/Methods

Data Sources

Output Measure

Risk adjustment using ETGs

Hospital-reported data; payer claims paid charges for procedure codes (CPT-9-CMxxxx, …) for episode length of time

Patient health status; provider payment; patient disposition; patient, provider satisfaction

Two different episode definition systems—ETGs and MEGs; three cost outlier tests: Winzorized at 10% and 90%; Winzorized at 90%; and Winzorized at 80%

Four years of claims and membership data

Two measures of costs were used in the analyses: Actual costs, as recorded by the HMO, and standard costs, determined by assigning the same costs to all procedures of the same type

 

Plan costs (total costs vs. disease specific costs)

 

 

30-day rehospitalization; Emergent care for wound infections (Source: OASIS, OBQM)

Emergent care for improper medication administration, medication side effects (Source: OASIS, OBQM)

Emergent care for hypo/hyperglycemia (Source: OASIS, OBQM)

Discharge to the community needing wound care or medication assistance (Source: OASIS, OBQM)

Acute care hospitalization (Source: OASIS/OBQI)

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Measures

Definition: Input:Output

Stated Purpose/Function

Health Care Setting

“Risk-adjustment accuracy” across primary care physicians

Group R2 analyses

Compare six different risk-adjustment methods in terms of capacity to explain variations in annual claims cost among HMO members

Physicians in HMO/IPA

Identification of high-outlier PCPs (family practitioners, general internists, and general practitioners, pediatricians)

 

Bridges to Excellence/NCQA Provider Recognition Programs: clinical quality measures for diabetes care (Diabetes Care Link); clinical quality measures for heart/stroke care Provider Recognition Programs Cardiac Care Link; adoption of electronic medical records and other office systems:

 

Measure quality processes and outcomes; reporting; recognition and possible financial rewards

Hospitals

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Required Enhancements/Methods

Data Sources

Output Measure

 

Unexpected nursing home admission (Source: OBQM)

Discharge to the community (Source: OASIS/OBQI)

Emergent care (Source: OASIS/OBQI)

Outlier removal

HMO, one state; member and adjudicated claims files (inpatient, outpatient/professional, and pharmacy) for calendar years 1997 and 1998 for the 156,280 continuously enrolled members

Reasonably consistent estimates of member level expected costs, across a broad range of panel sizes

 

Identification of high-outlier PCPs ranged from 54% to 58% for adult care physicians (family practitioners, general internists, and general practitioners), and from 67% to 77% for pediatricians, when rankings were based on the standardized cost difference measure which accounts for physician panel size

 

Hospital sampling, self-report

Rates of adherence

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Measures

Definition: Input:Output

Stated Purpose/Function

Health Care Setting

Physician Office Link: Clinical Information Systems/Evidence-Based Medicine (See Bridges to Excellence)

 

Leapfrog Group: Computer physician order entry (CPOE) systems

Presence of systems; use

Electronic prescribing systems that intercept errors

Hospitals

Evidence-based hospital (EHR) Safety Standard

Combination of outcome, process and volume

Adoption of clinical processes for high-risk procedures; volume of procedures per year; direct outcome measures (i.e, risk-adjusted mortality) for coronary artery bypass graft and percutaneous coronary interventions, using robust and approved measurement systems for the EHR Safety Standards

 

ICU physician staffing

Operate adult and/or pediatric ICUs that are managed or comanaged by intensivists who:

1. Are present during daytime hours and provide clinical care exclusively in the ICU and,

2. At other times can—at least 95% of the time, (i) return ICU pages within five minutes and (ii) arrange for a FCCS-certified nonphysician effector to reach ICU patients within five minutes

Patients are cared for exclusively by critical-care specialists or teams that are closer on hand for both fine-tuning routine care and dealing with emergencies

 

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Required Enhancements/Methods

Data Sources

Output Measure

Upfront capital

Voluntary hospital self-report; data survey

Explicit: extent to which standards are met, relative to other hospitals; implicit: costs of adverse drug events: mortality, morbidity; other costs

An EHR standard does not apply to hospitals that do not perform the procedure or treat the condition. Patients under 18 are excluded

Hospitals to report their volume and process or performance information for these procedures and conditions by responding to the Leapfrog Hospital Patient Safety Survey on the Leapfrog Website

 

 

Hospitals with adult or pediatric ICUs to respond to the Leapfrog Group

Presence/absence of intensivists in ICU; organization of closed/open ICU

 

Voluntary online survey

 

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Measures

Definition: Input:Output

Stated Purpose/Function

Health Care Setting

Leapfrog Group: Expert Panel-Endorsed Process Measures

Cases meeting endorsed process measure: eligible cases (meeting criteria for inclusion)

Establish, monitor, and report measures of process-oriented quality

Hospitals

IHI Whole System Measures: efficiency

Costs per capita; hospital specific standardized reimbursement

 

The Agency for Healthcare Research and Quality Quality Indicators (QIs) are measures of health care quality

Prevention QIs identify hospital admissions that evidence suggests could have been avoided, at least in part, through high-quality outpatient care

National tracking or quality improvement

 

 

Inpatient QIs reflect quality of care inside hospitals including inpatient mortality for medical conditions and surgical procedures

 

Patient Safety Indicators also reflect quality of care inside hospitals, but focus on potentially avoidable complications and iatrogenic events

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Required Enhancements/Methods

Data Sources

Output Measure

Exclude transferred patients; expired patients

Hospital: random sample of at least 60 cases with the condition; principal or secondary discharge diagnosis

Rate of adherence to endorsed process measures of quality

Measure health care quality using administrative data; update for ICD codes

 

Currently being considered for uses other than tracking quality improvement; namely provider payment and public reporting

Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 222
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 223
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 224
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 225
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 226
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 227
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 228
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 229
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 230
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 231
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 232
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 233
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 234
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 235
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 236
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 237
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 238
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 239
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 240
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 241
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 242
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 243
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 244
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 245
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 246
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 247
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 248
Suggested Citation:"Appendix H Commissioned Paper: Efficiency/Value-Based Measures for Services, Defined Populations, Acute Episodes, and Chronic Conditions--Kyle L. Grazier." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 249
Next: Appendix I Commissioned Paper: Transitional Care Performance Measurement--Eric A. Coleman »
Performance Measurement: Accelerating Improvement Get This Book
×
Buy Hardback | $75.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Performance Measurement is the first in a new series of an ongoing effort by the Institute of Medicine (IOM) to improve health care quality. Performance Measurement offers a comprehensive review of available measures and introduces a new framework to examine these measures against the six aims of the health care system: health care should be safe, effective, patient-centered, timely, efficient, and equitable. This new book also addresses the gaps in performance measurement and introduces the need for measures that are longitudinal, comprehensive, population-based, and patient-centered. This book is directed toward all concerned with improving the quality and performance of the nation’s health care system in its multiple dimensions and in both the public and private sectors.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!