Specifying the Shape of a Core Metrics Set
KEY BREAKOUT GROUP THEMES
Health Care
- Each core metric needs to be independent of the others, and collectively they should be comprehensive.
- Health care metrics need to be able to be properly adjusted for different populations.
- The six domains of quality from the Quality Chasm report provide an important starting point, with one new potential concept being overall modifiable risk.
Population Health
- Population health metrics can be divided into two categories: current health, such as length of life and quality of life, and future health, including factors that are both intrinsic and extrinsic to individuals.
- The three-part aim requires measures for population health that are outside the traditional purview of the health care system. New responsibilities in communities will be needed to address those metrics.
- Metric development should consider the eventual use of a measure and its potential users and should aim for the measure to be actionable for the intended user.
Cost
- Cost metrics are needed to understand the drivers of cost and waste; to inform choices for plans, patients, and clinicians; to inform value-based payments; and to fuel transparency.
- Three high-level concepts are of primary importance: risk-adjusted per capita costs, utilization, and affordability.
- Operationalizing these metrics requires addressing issues involving antitrust concerns, distrust among different organizations collecting data, the privacy provisions of the Health Insurance Portability and Accountability Act, and data standardization.
An important component of this workshop was the 2-hour working group session, during which participants engaged in discussion to identify potential sets of core metrics for tracking progress toward better care, better health, and lower costs at national, state, community, organizational, and individual levels. Participants were assigned to one of three breakout groups, with each breakout group considering one dimension of the three-part aim. In addition to identifying the area’s metrics that were most important to reliable assessment and monitoring of progress at these different levels, the participants were charged with identifying implementation challenges associated with this dimension and proposing approaches to address those challenges.
To assist each group in its deliberations, the workshop organizing committee, with the assistance of Institute of Medicine (IOM) staff, put together a packet of information on each specific aspect of measurement and included a table of potential metric categories with example metrics (see Table 5-1). The three groups took different approaches to assessing the suitability of those metric categories and to identifying making recommendations about additional categories and sub-categories. This chapter summarizes the discussions that took place during the breakout group discussions1 and in the subsequent discussions that involved all of the workshop participants.
__________________
1 The summaries of the working group discussions are intended to demonstrate the diversity of perspectives and divergent opinions and should not be construed to reflect any group consensus.
TABLE 5-1 Matrix of Potential Metric Categories Across the Three-Part Aim
Metric Domain | Metric Categories | Example Metrics | National Priority from National Quality Strategy |
Population Health | Length of life | Mortality, life expectancy, premature birth, preventable deaths | Promote wide use of best practices to enable healthy living and well-being. (National Priority 1) |
Quality of life | Physical health, functional status, disease burden, morbidity, pain, mental health, social functioning, injuries | ||
Health behaviors and risk | Smoking, exercise, alcohol use, healthy diet, obesity | ||
Utilization of preventive services | Immunizations, dental health, appropriate screening | ||
Community health | Safety, healthy food, walkability/places to exercise, pollutants, healthy workplaces | ||
Social and economic factors | Educational attainment, literacy, poverty, unemployment, health insurance status | ||
Health Care | Effective | Adherence to guidelines, disease-specific treatment targets (e.g., cardiovascular disease: control of high blood pressure, cholesterol, aspirin use) | Promote the most effective prevention, treatment, and intervention practices for the leading causes of mortality, starting with cardiovascular disease. (National Priority 2) |
Patient-centered | Experience of care; shared decision making, shared goal setting, or patient inclusion in health care team, patient knowledge and understanding of care plan, clinical communications, supports for self-care | Ensure person- and family-centered care. (National Priority 3) |
Metric Domain | Metric Categories | Example Metrics | National Priority from National Quality Strategy |
Health Care continued | Safe | Preventable hospital admissions/readmissions, health care-associated infections, medical errors (composite measure: serious reportable events), inappropriate medication use, inappropriate maternity/newborn care, unnecessary tests, occupational safety in health care | Make care safer. (National Priority 4) |
Coordination and communication | Experience of care transitions; communication among health care team members, including patient, family, and caregivers; appropriate sharing of health records; care consistent with preferences, particularly for end-of-life care | Promote effective communication and care coordination. (National Priority 5) |
|
Equitable | Support of vulnerable populations, communication appropriate to individual and community health literacy | Elements captured in National Priorities 1, 3 and 5 | |
Efficiency and timeliness (includes operations) | Access to needed care, consistent insurance, achievement of meaningful use of health IT, appropriate nurse staffing, effective management | Elements captured in National Priorities 5 and 6 |
Metric Domain | Metric Categories | Example Metrics | National Priority from National Quality Strategy |
Cost | Affordability | Costs for households/ individuals, impact on wages/benefits, impact on other government services (local, state, national) | Make quality care affordable for people, families, employers, and governments. (National Priority 6) |
Expenditures | Overall annual spending on health care (per member per month, per capita, per episode, per service), utilization of services | ||
Indirect costs | Absenteeism, productivity | ||
Waste | Unnecessary services (includes costs due to unwarranted variation/ overuse), fraud, excessive administrative costs, inefficiently delivered services, prices that are too high, missed prevention |
Key points from the breakout group discussion chaired by David Stevens, associate chief medical officer and director of the Quality Center at the National Association of Community Health Centers and research professor at the George Washington University School of Public Health and Health Services, are summarized here. Before starting this discussion, Mary Barton, vice president for performance measurement at the National Committee for Quality Assurance, gave a brief synopsis of the state of the field.
Defining a Core Metric and Understanding Current Limitations
The group began by considering the fundamental characteristics of a core metric set. An important characteristic of a set of core metrics is that each measure should be mutually exclusive but that collectively they should be exhaustive, that is, when put together they provide a unified picture of progress on the three-part aim. The denominator of a core metric should be adaptable for different populations and population sizes so that the metric
makes it possible to drill down for more detailed analysis. The group also noted that it was important that metrics balance cost and quality of care in a way that does not overemphasize cost at the expense of the entire three-part aim. Finally, it was deemed important that any metrics of health outcomes be adjusted for risk.
The group also identified several current limitations of core measures. Many metrics today appear to be snapshots that reflect episodes of care at a specific moment in time rather than on a specific service or condition, and the breakout group expressed some concern that such metrics may not be that useful in a learning environment. A core metric should also be sensitive to change, a point that Stevens stressed, and should connect to a system that can learn over time, track results, and improve as a result of changes in the metric.
Reviewing Potential Metric Categories
Several members of the breakout group indicated their comfort with the initial set of measurement categories contained in the background material, which were largely drawn from the Quality Chasm definition of quality (IOM, 2001). To improve this initial set by reducing the number of domains, some participants proposed that a timeliness metric belonged both in patient-centered care and coordinated care and that communication should be included as part of patient-centered care. Attendees had differing views on whether an equitable care metric should span all domains or if doing so would cause it to be lost. Stevens noted the latter concern could be addressed by emphasizing that all populations should be examined across those domains.
One proposed overarching concept was overall modifiable risk. Currently this metric is used only for cardiovascular disease, but the concept could be extended to many conditions and diseases. This metric would be actionable at multiple levels, from the patient to the system level. A conceptual area that was raised in the discussion involved engaging providers. Although there has been significant attention paid to developing metrics that assess how engaged patients are in their own care, little work has been done to measure the changes and values that clinical professionals need to operate in a system designed around the three-part aim. These changes include working in teams, collaborating across organizations and regions, and being able to work with patients in self-management. Another area of need is for metrics that can assess the local capacity to assemble providers, public health structures, and community organizations to reach the three-part aim.
Examples of core metrics for the three-part aim include the composite Consumer Assessment of Healthcare Providers and Systems score for
patient experience, risk-adjusted mortality, functional status, a composite safety measure that is now under development, readmissions, and ambulatory care–sensitive readmissions. The calculation and use of many of these measures are constrained by the limited amount of consistent data collected across the wide range of health care delivery settings.
POPULATION HEALTH BREAKOUT GROUP
Participants in this group, chaired by Patrick Remington, associate dean for public health at the University of Wisconsin School of Medicine and Public Health, focused on three topics: defining populations, selecting metrics, and measuring community health. Before discussing these topics, Steven Teutsch, chief science officer of the Los Angeles County Health Department, gave a brief overview of the current state of population health measurement.
Defining Populations
Populations at risk serve as the denominator of any metric dealing with population health, making it essential to define specific populations. Geopolitical boundaries are an obvious defining feature, one with which public health is comfortable. However, health care usually thinks about populations of patients in a care system or populations of people with a particular condition, neither of which aligns with geopolitical boundaries. The definition of population, Remington said, depends on who is asking the question, whether it be policy makers, government entities, health systems, or patient groups.
Selecting Metrics
In talking about the categories for metrics of population health, participants set forth two general categories: current health, and factors that help predict health in the future. Health outcomes are essentially measures of death and disease or mortality and morbidity. The group noted that many metrics could be suitable for assessing these concepts, including health-adjusted life years, premature death rates, health-related quality of life, and disease incidence.
The group discussed briefly the notion of using rates of indicator diseases, such as heart attack rates, as sentinel indicators of health system performance. It also talked at length about whether determinants of health could be used to measure risk in populations, in terms of both factors that are intrinsic to individuals and factors that are extrinsic to individuals. Health risk appraisal was also mentioned as a potentially useful metric.
Quality of life, whether measured in terms of morbidity or judged by the individuals themselves, is an important metric, and both mental and physical health play a role. Group members noted that there may be specific measures that might be proxies of quality of life, such as the percent of 60-year-old adults who have had a knee replacement. The group also discussed the idea that the three-part aim requires measures for population health that are outside the traditional purview of the health care system and that it would be the responsibility of those communities to address those metrics.
The group also explored the concept of community health. A healthy community is one with a robust combination of supporting elements, such as safety, quality of the educational system, the availability of jobs, and others. The breakout group noted that performance metrics for community health should be measured in context, that is, not just by the health or by the length or quality of life of individuals, but relative to the conditions in which people live.
Implementation
The group’s participants conducted a substantial discussion about implementing metrics of population health. Remington captured five themes from this discussion. The first was that words matter: Measures such as health-related quality of life may be understood by policy makers, but the general public may not be familiar with this phrasing. Terms such as “healthy community” and “community well-being” are ambiguous and can have different meanings for different constituencies. Though the group did not solve the question of which words to use, it did recommend that experts in communication be brought into future discussions.
The group had a heated discussion about whether surveys or broad-based, all-inclusive individual questions are the best approach to collecting data on community health metrics. Both methods have costs associated with them that can be substantial, whether those costs are episodic in the case of surveys or ongoing for individual questionnaires given at the time an individual receives health care services. The group did agree that information needs to be collected on states of wellness and well-being and not be limited to people who are interacting with the health care system at any given moment. This is particularly important given that there are many people who are not in a structured health care system but who may constitute a substantial portion of the overall population.
While indices can serve as important metrics, it is important that they be transparent and not just a black box measurement. They can be helpful for summarizing information, but they must be amenable to drilling down. There was also a debate on when metrics should focus on everyone ver
sus the leading innovators in promoting health. By looking at innovators, metrics can provide a glimpse of the future of where the health care system could go as opposed to where it has been.
Finally, all metrics should be actionable in the short term. Long-term metrics may be interesting to researchers, but they hold little sway over policy makers. Remington noted that this is not a theoretical exercise given that there are places that already measure the health of communities or populations and where public health and health care systems are working together.
The working group participants discussed several aspects of cost as viewed from different perspectives. Kate Goodrich, senior medical advisor in the Office of Clinical Standards and Quality at the Centers for Medicare & Medicaid Services, chaired the group, and Dennis Scanlon, professor of health policy and administration at Pennsylvania State University, summarized the current state of cost measurement before the group began its work.
Purpose of Measurement
In a short, straightforward discussion, the group first addressed the issue of identifying the purposes that cost metrics should serve. Core metrics need to be related to the aim of lowering cost in general—and to lowering per capita costs specifically—and they should help explain variations in trends. Cost metrics are also needed to understand the drivers of cost and waste; to inform choices for plans, patients, and administrators; to inform value-based payments; and to fuel transparency. After a more heated discussion, participants in the group leaned toward a definition of cost that refers to health care spending within the delivery system, that is, what is spent on payments to providers and out-of-pocket expenses for patients, including the cost of insurance. This definition of cost does not include the bigger issues of public health, such as education and crime reduction.
Core Metrics
The group spent much of its time discussing high-level core metrics and outlined three of primary importance. The first two measures were straightforward: per capita costs and a measure that assesses utilization. These would be risk-adjusted measures, and they could be analyzed by delivery systems, geographies, and populations. The third metric, which several at-
tendees felt was novel and actionable, was an affordability measure, since a primary goal of the three-part aim is to make health care affordable.
While the working group participants agreed that there are many good subsets of metrics within these three core metric categories, there was not sufficient time to discuss these measures. However, the group recommended that settling on broad subsets of measures should be the next order of business.
The group also discussed how actionable these metrics would be—for example, what a provider can do once it knows its per capita costs. Several participants noted the need for core metrics and sub-metrics to be understandable at the local level and at greater levels of aggregation. The group debated whether there should be an underlying set of appropriate resource measures, but it did not reach agreement on this point and agreed that more discussion was needed to resolve this issue.
Implementation
The group identified a number of implementation issues that will need to be resolved, including antitrust issues, distrust among different organizations collecting data, the privacy provisions of the Health Insurance Portability and Accountability Act, and data standardization. The group noted that benefit design is a major issue when measuring per capita cost because of variations in attribution, legal provisions, and paying party. Fragmentation in the system across all levels is a major stumbling block to implementing meaningful cost metrics. During the general discussion, several participants remarked that data collection for cost metrics needs to be standardized at the national level in such a way that states and communities can then use this metric. It was also noted that issues over who controls cost data need to be resolved.
Following the reports from the three breakout groups, IOM staff prepared a summary of the specific metrics and categories that were discussed in the breakout sessions (see Tables 5-2, 5-3, and 5-4). These summaries were then the subject of a discussion among all the workshop participants. That discussion is summarized here.
Population Health
Multiple attendees voiced support for creating two major metric categories for current and future health. One participant suggested that example metrics in the future health category could be intrinsic risk, as measured
TABLE 5-2 Summary of Population Health Breakout Group Discussion
Metric Domain | Metric Categories | Example Metrics | Implementation |
Population Health | Current Health | Length of life: Mortality, life expectancy Quality of life: Morbidity, functional status, indicator diseases Composite: QALY, HALY |
• Defining the population • Communication/education about measures • Data collection • Transparent methods for composites/indices • Tension in targeting innovators or all actors • Actionability of measures |
Future Health | Health determinants: Health risks, health behaviors, healthy communities, and extrinsic determinants |
NOTE: HALY = health-adjusted life year; QALY = quality-adjusted life year.
at the individual level in terms of health behaviors and perhaps genetic predisposition, and extrinsic or community risk, which would be measured using social and ecological models and include some measure of place-based risk. Another participant remarked that quality-of-life metrics in the current health category should include some measure of self-reported health. There was some discussion about how to phrase metrics in the future health category so that the general public will understand the concepts involved. It was noted that the term “health determinants” failed to resonate with consumers when it was tested in Wisconsin, and instead researchers there are using the term “health factors” and explaining it as things that will predict how healthy an individual will be in the future. The term “health influences” was also suggested.
There were several comments about how to conceptualize population health as it moves from the entire population to subpopulations. One participant remarked that population health at the level of a 5,000-person accountable care organization serving Medicare patients is not going to look the same as a subpopulation of Medicaid patients or a subpopulation in the hundreds of thousands and wondered how those differences will be
TABLE 5-3 Summary of Health Care Breakout Group Discussion
Metric Domain | Metric Categories | Example Metrics | Implementation |
Health Care | Patient centered | Patient experience: HCAHPS metric Equitable Timeliness |
• Risk adjustment critical • Appropriateness of care • Timeliness of care under all metrics (both in initial access and time to return to function) |
Effective | Mortality amenable to health care Functional status Equitable |
||
Safe | Composite medical harm measure (including medical errors and health care-associated infections) Equitable |
||
Efficient | Utilization: Admissions and readmissions for ambulatory care sensitive conditions Equitable |
||
Coordination and communication | Timeliness |
NOTE: HCAHPS = Hospital Consumer Assessment of Healthcare Providers and Systems.
reflected in the implemented metrics. Participants also asked how measures of population health across subpopulations should reflect the concepts of equity, health disparities, insurance status, and access to care.
Health Care
When the breakout group reported its discussion to all workshop attendees, Marcus Thygeson, vice president for medical services at Blue Shield of California, remarked that it would be useful to have metrics that measure how patients do over time. Such metrics would assess the path to recovery and the speed of recovery, and they would differ from a timeliness
TABLE 5-4 Summary of Cost Breakout Group Discussion
Metric Domain | Metric Categories | Example Metrics | Implementation |
Cost | Total cost of care (actual costs) | Total cost of care metric, actual cost and risk-adjusted. Population-based per member per month (all conditions) |
• Antitrust • HIPAA • Proprietary interests • Standardization • Governance • Attribution • Legal • Costs |
Total cost of care (standardized costs) | Total cost of care metric, standardized costs and risk-adjusted. Population-based per member per month (all conditions) | ||
Affordability | Percent of household spending on health Percent of national GDP and/or federal government health care spending as percent of total federal government spending Percent of economy, governmental budgets, organizational budgets, or individual budgets devoted to specific programs or sources of payment, including employer-based health benefits, Medicaid, Medicare, and spending by the uninsured. Premiums |
metric or one that measured functional status. During this same discussion, Stevens noted that health care measures could be analyzed individually or be combined to support real-time improvement for individual patients, populations, providers, or health systems.
During the discussion, two participants from the health care breakout group raised suggestions for improving the summary list of measures by including several of the example measures from the background materials. Another participant suggested that the concept of appropriateness of care should be added under the effectiveness category and that a measure of composite medical harm should be included in the safety category. It was also recommended that ambulatory care admissions and readmissions be included in the efficiency category. One place to start in creating such a measure would be existing lists of conditions that should be treated on an outpatient basis.
Another participant, who had been part of the cost breakout group, asked if efficiency should be a subset of cost and if equitable care should be a metric itself rather than merely an example. A participant suggested that functional status should be broadened to that of overall health status, reflecting the changes in a patient’s overall health after receiving health care services. It was noted that patient-reported outcomes could serve as a general measure of functional status.
There were also comments about implementation, particularly concerning the settings in which the data for these metrics would be collected. It will be important going forward, one participant noted, to identify the least disruptive setting or most accessible and least expensive setting in which to collect data if these metrics are to be truly useful on a large scale. A participant commented that a number of professional organizations are developing and implementing process metrics that are condition-specific and that these metrics are being well received by the medical community.
Cost
The discussion highlighted the fact that total cost depends on two factors—the price of health care services and the utilization of those services. Unpacking overall costs in this way allows for a better understanding of which of these two factors is driving cost. However, one participant noted that this may lead to conceptual issues, as utilization can sometimes be considered in the health care quality domain of the three-part aim as opposed to the cost domain. For example, one participant noted that good health care services can directly reduce utilization, but their impact on cost may be unclear because of the multiple business layers between utilization and cost. Another participant remarked that utilization can be appropriate or inappropriate and that perhaps a metric for waste should be included
in the cost domain, while another participant suggested further delineating waste into under-utilization and over-utilization.
Concerning the affordability metric, participants noted that the examples should also include measures of health care spending by states and communities as a percentage of state and local community economic output in addition to the comparison of health care spending with national gross domestic product. Other participants noted that similar metrics could be included that illustrate the level of spending devoted to multiple programs and sources of payment, such as Medicare, Medicaid, employer-sponsored health programs, and health spending by the uninsured, with this spending normalized by the size of the economy, governmental budgets, organizational budgets, or individual budgets. One speaker suggested that premiums be included in the percentage of household spending on health and not listed separately, and another participant suggested there should be a metric involving value; while the members of this workshop all understand that value is included throughout these metrics, the participant noted, it would be useful to make this explicit for the general public.
IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.