Core Measurement Needs for Better Care, Better Health, and Lower Costs: Counting What Counts: Workshop Summary (2013)

5

Specifying the Shape of a Core Metrics Set

An important component of this workshop was the 2-hour working group session, during which participants engaged in discussion to identify potential sets of core metrics for tracking progress toward better care, better health, and lower costs at national, state, community, organizational, and individual levels. Participants were assigned to one of three breakout groups, with each breakout group considering one dimension of the three-part aim. In addition to identifying the area’s metrics that were most important to reliable assessment and monitoring of progress at these different levels, the participants were charged with identifying implementation challenges associated with this dimension and proposing approaches to address those challenges.

To assist each group in its deliberations, the workshop organizing committee, with the assistance of Institute of Medicine (IOM) staff, put together a packet of information on each specific aspect of measurement and included a table of potential metric categories with example metrics (see Table 5-1). The three groups took different approaches to assessing the suitability of those metric categories and to identifying making recommendations about additional categories and sub-categories. This chapter summarizes the discussions that took place during the breakout group discussions¹ and in the subsequent discussions that involved all of the workshop participants.

__________________

¹ The summaries of the working group discussions are intended to demonstrate the diversity of perspectives and divergent opinions and should not be construed to reflect any group consensus.

TABLE 5-1 Matrix of Potential Metric Categories Across the Three-Part Aim

HEALTH CARE BREAKOUT GROUP

Key points from the breakout group discussion chaired by David Stevens, associate chief medical officer and director of the Quality Center at the National Association of Community Health Centers and research professor at the George Washington University School of Public Health and Health Services, are summarized here. Before starting this discussion, Mary Barton, vice president for performance measurement at the National Committee for Quality Assurance, gave a brief synopsis of the state of the field.

Defining a Core Metric and Understanding Current Limitations

The group began by considering the fundamental characteristics of a core metric set. An important characteristic of a set of core metrics is that each measure should be mutually exclusive but that collectively they should be exhaustive, that is, when put together they provide a unified picture of progress on the three-part aim. The denominator of a core metric should be adaptable for different populations and population sizes so that the metric

makes it possible to drill down for more detailed analysis. The group also noted that it was important that metrics balance cost and quality of care in a way that does not overemphasize cost at the expense of the entire three-part aim. Finally, it was deemed important that any metrics of health outcomes be adjusted for risk.

The group also identified several current limitations of core measures. Many metrics today appear to be snapshots that reflect episodes of care at a specific moment in time rather than on a specific service or condition, and the breakout group expressed some concern that such metrics may not be that useful in a learning environment. A core metric should also be sensitive to change, a point that Stevens stressed, and should connect to a system that can learn over time, track results, and improve as a result of changes in the metric.

Reviewing Potential Metric Categories

Several members of the breakout group indicated their comfort with the initial set of measurement categories contained in the background material, which were largely drawn from the Quality Chasm definition of quality (IOM, 2001). To improve this initial set by reducing the number of domains, some participants proposed that a timeliness metric belonged both in patient-centered care and coordinated care and that communication should be included as part of patient-centered care. Attendees had differing views on whether an equitable care metric should span all domains or if doing so would cause it to be lost. Stevens noted the latter concern could be addressed by emphasizing that all populations should be examined across those domains.

One proposed overarching concept was overall modifiable risk. Currently this metric is used only for cardiovascular disease, but the concept could be extended to many conditions and diseases. This metric would be actionable at multiple levels, from the patient to the system level. A conceptual area that was raised in the discussion involved engaging providers. Although there has been significant attention paid to developing metrics that assess how engaged patients are in their own care, little work has been done to measure the changes and values that clinical professionals need to operate in a system designed around the three-part aim. These changes include working in teams, collaborating across organizations and regions, and being able to work with patients in self-management. Another area of need is for metrics that can assess the local capacity to assemble providers, public health structures, and community organizations to reach the three-part aim.

Examples of core metrics for the three-part aim include the composite Consumer Assessment of Healthcare Providers and Systems score for

patient experience, risk-adjusted mortality, functional status, a composite safety measure that is now under development, readmissions, and ambulatory care–sensitive readmissions. The calculation and use of many of these measures are constrained by the limited amount of consistent data collected across the wide range of health care delivery settings.

POPULATION HEALTH BREAKOUT GROUP

Participants in this group, chaired by Patrick Remington, associate dean for public health at the University of Wisconsin School of Medicine and Public Health, focused on three topics: defining populations, selecting metrics, and measuring community health. Before discussing these topics, Steven Teutsch, chief science officer of the Los Angeles County Health Department, gave a brief overview of the current state of population health measurement.

Defining Populations

Populations at risk serve as the denominator of any metric dealing with population health, making it essential to define specific populations. Geopolitical boundaries are an obvious defining feature, one with which public health is comfortable. However, health care usually thinks about populations of patients in a care system or populations of people with a particular condition, neither of which aligns with geopolitical boundaries. The definition of population, Remington said, depends on who is asking the question, whether it be policy makers, government entities, health systems, or patient groups.

Selecting Metrics

In talking about the categories for metrics of population health, participants set forth two general categories: current health, and factors that help predict health in the future. Health outcomes are essentially measures of death and disease or mortality and morbidity. The group noted that many metrics could be suitable for assessing these concepts, including health-adjusted life years, premature death rates, health-related quality of life, and disease incidence.

The group discussed briefly the notion of using rates of indicator diseases, such as heart attack rates, as sentinel indicators of health system performance. It also talked at length about whether determinants of health could be used to measure risk in populations, in terms of both factors that are intrinsic to individuals and factors that are extrinsic to individuals. Health risk appraisal was also mentioned as a potentially useful metric.

Quality of life, whether measured in terms of morbidity or judged by the individuals themselves, is an important metric, and both mental and physical health play a role. Group members noted that there may be specific measures that might be proxies of quality of life, such as the percent of 60-year-old adults who have had a knee replacement. The group also discussed the idea that the three-part aim requires measures for population health that are outside the traditional purview of the health care system and that it would be the responsibility of those communities to address those metrics.

The group also explored the concept of community health. A healthy community is one with a robust combination of supporting elements, such as safety, quality of the educational system, the availability of jobs, and others. The breakout group noted that performance metrics for community health should be measured in context, that is, not just by the health or by the length or quality of life of individuals, but relative to the conditions in which people live.

Implementation

The group’s participants conducted a substantial discussion about implementing metrics of population health. Remington captured five themes from this discussion. The first was that words matter: Measures such as health-related quality of life may be understood by policy makers, but the general public may not be familiar with this phrasing. Terms such as “healthy community” and “community well-being” are ambiguous and can have different meanings for different constituencies. Though the group did not solve the question of which words to use, it did recommend that experts in communication be brought into future discussions.

The group had a heated discussion about whether surveys or broad-based, all-inclusive individual questions are the best approach to collecting data on community health metrics. Both methods have costs associated with them that can be substantial, whether those costs are episodic in the case of surveys or ongoing for individual questionnaires given at the time an individual receives health care services. The group did agree that information needs to be collected on states of wellness and well-being and not be limited to people who are interacting with the health care system at any given moment. This is particularly important given that there are many people who are not in a structured health care system but who may constitute a substantial portion of the overall population.

While indices can serve as important metrics, it is important that they be transparent and not just a black box measurement. They can be helpful for summarizing information, but they must be amenable to drilling down. There was also a debate on when metrics should focus on everyone ver

sus the leading innovators in promoting health. By looking at innovators, metrics can provide a glimpse of the future of where the health care system could go as opposed to where it has been.

Finally, all metrics should be actionable in the short term. Long-term metrics may be interesting to researchers, but they hold little sway over policy makers. Remington noted that this is not a theoretical exercise given that there are places that already measure the health of communities or populations and where public health and health care systems are working together.

COST BREAKOUT GROUP

The working group participants discussed several aspects of cost as viewed from different perspectives. Kate Goodrich, senior medical advisor in the Office of Clinical Standards and Quality at the Centers for Medicare & Medicaid Services, chaired the group, and Dennis Scanlon, professor of health policy and administration at Pennsylvania State University, summarized the current state of cost measurement before the group began its work.

Purpose of Measurement

In a short, straightforward discussion, the group first addressed the issue of identifying the purposes that cost metrics should serve. Core metrics need to be related to the aim of lowering cost in general—and to lowering per capita costs specifically—and they should help explain variations in trends. Cost metrics are also needed to understand the drivers of cost and waste; to inform choices for plans, patients, and administrators; to inform value-based payments; and to fuel transparency. After a more heated discussion, participants in the group leaned toward a definition of cost that refers to health care spending within the delivery system, that is, what is spent on payments to providers and out-of-pocket expenses for patients, including the cost of insurance. This definition of cost does not include the bigger issues of public health, such as education and crime reduction.

Core Metrics

The group spent much of its time discussing high-level core metrics and outlined three of primary importance. The first two measures were straightforward: per capita costs and a measure that assesses utilization. These would be risk-adjusted measures, and they could be analyzed by delivery systems, geographies, and populations. The third metric, which several at-

tendees felt was novel and actionable, was an affordability measure, since a primary goal of the three-part aim is to make health care affordable.

While the working group participants agreed that there are many good subsets of metrics within these three core metric categories, there was not sufficient time to discuss these measures. However, the group recommended that settling on broad subsets of measures should be the next order of business.

The group also discussed how actionable these metrics would be—for example, what a provider can do once it knows its per capita costs. Several participants noted the need for core metrics and sub-metrics to be understandable at the local level and at greater levels of aggregation. The group debated whether there should be an underlying set of appropriate resource measures, but it did not reach agreement on this point and agreed that more discussion was needed to resolve this issue.

Implementation

The group identified a number of implementation issues that will need to be resolved, including antitrust issues, distrust among different organizations collecting data, the privacy provisions of the Health Insurance Portability and Accountability Act, and data standardization. The group noted that benefit design is a major issue when measuring per capita cost because of variations in attribution, legal provisions, and paying party. Fragmentation in the system across all levels is a major stumbling block to implementing meaningful cost metrics. During the general discussion, several participants remarked that data collection for cost metrics needs to be standardized at the national level in such a way that states and communities can then use this metric. It was also noted that issues over who controls cost data need to be resolved.

DISCUSSION

Following the reports from the three breakout groups, IOM staff prepared a summary of the specific metrics and categories that were discussed in the breakout sessions (see Tables 5-2, 5-3, and 5-4). These summaries were then the subject of a discussion among all the workshop participants. That discussion is summarized here.

Population Health

Multiple attendees voiced support for creating two major metric categories for current and future health. One participant suggested that example metrics in the future health category could be intrinsic risk, as measured

TABLE 5-2 Summary of Population Health Breakout Group Discussion

NOTE: HALY = health-adjusted life year; QALY = quality-adjusted life year.

at the individual level in terms of health behaviors and perhaps genetic predisposition, and extrinsic or community risk, which would be measured using social and ecological models and include some measure of place-based risk. Another participant remarked that quality-of-life metrics in the current health category should include some measure of self-reported health. There was some discussion about how to phrase metrics in the future health category so that the general public will understand the concepts involved. It was noted that the term “health determinants” failed to resonate with consumers when it was tested in Wisconsin, and instead researchers there are using the term “health factors” and explaining it as things that will predict how healthy an individual will be in the future. The term “health influences” was also suggested.

There were several comments about how to conceptualize population health as it moves from the entire population to subpopulations. One participant remarked that population health at the level of a 5,000-person accountable care organization serving Medicare patients is not going to look the same as a subpopulation of Medicaid patients or a subpopulation in the hundreds of thousands and wondered how those differences will be

TABLE 5-3 Summary of Health Care Breakout Group Discussion

NOTE: HCAHPS = Hospital Consumer Assessment of Healthcare Providers and Systems.

reflected in the implemented metrics. Participants also asked how measures of population health across subpopulations should reflect the concepts of equity, health disparities, insurance status, and access to care.

Health Care

When the breakout group reported its discussion to all workshop attendees, Marcus Thygeson, vice president for medical services at Blue Shield of California, remarked that it would be useful to have metrics that measure how patients do over time. Such metrics would assess the path to recovery and the speed of recovery, and they would differ from a timeliness

TABLE 5-4 Summary of Cost Breakout Group Discussion

metric or one that measured functional status. During this same discussion, Stevens noted that health care measures could be analyzed individually or be combined to support real-time improvement for individual patients, populations, providers, or health systems.

During the discussion, two participants from the health care breakout group raised suggestions for improving the summary list of measures by including several of the example measures from the background materials. Another participant suggested that the concept of appropriateness of care should be added under the effectiveness category and that a measure of composite medical harm should be included in the safety category. It was also recommended that ambulatory care admissions and readmissions be included in the efficiency category. One place to start in creating such a measure would be existing lists of conditions that should be treated on an outpatient basis.

Another participant, who had been part of the cost breakout group, asked if efficiency should be a subset of cost and if equitable care should be a metric itself rather than merely an example. A participant suggested that functional status should be broadened to that of overall health status, reflecting the changes in a patient’s overall health after receiving health care services. It was noted that patient-reported outcomes could serve as a general measure of functional status.

There were also comments about implementation, particularly concerning the settings in which the data for these metrics would be collected. It will be important going forward, one participant noted, to identify the least disruptive setting or most accessible and least expensive setting in which to collect data if these metrics are to be truly useful on a large scale. A participant commented that a number of professional organizations are developing and implementing process metrics that are condition-specific and that these metrics are being well received by the medical community.

Cost

The discussion highlighted the fact that total cost depends on two factors—the price of health care services and the utilization of those services. Unpacking overall costs in this way allows for a better understanding of which of these two factors is driving cost. However, one participant noted that this may lead to conceptual issues, as utilization can sometimes be considered in the health care quality domain of the three-part aim as opposed to the cost domain. For example, one participant noted that good health care services can directly reduce utilization, but their impact on cost may be unclear because of the multiple business layers between utilization and cost. Another participant remarked that utilization can be appropriate or inappropriate and that perhaps a metric for waste should be included

in the cost domain, while another participant suggested further delineating waste into under-utilization and over-utilization.

Concerning the affordability metric, participants noted that the examples should also include measures of health care spending by states and communities as a percentage of state and local community economic output in addition to the comparison of health care spending with national gross domestic product. Other participants noted that similar metrics could be included that illustrate the level of spending devoted to multiple programs and sources of payment, such as Medicare, Medicaid, employer-sponsored health programs, and health spending by the uninsured, with this spending normalized by the size of the economy, governmental budgets, organizational budgets, or individual budgets. One speaker suggested that premiums be included in the percentage of household spending on health and not listed separately, and another participant suggested there should be a metric involving value; while the members of this workshop all understand that value is included throughout these metrics, the participant noted, it would be useful to make this explicit for the general public.

REFERENCE

IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.

This page intentionally left blank.