5

Specifying the Shape of a Core Metrics Set

KEY BREAKOUT GROUP THEMES

Health Care

  • Each core metric needs to be independent of the others, and collectively they should be comprehensive.
  • Health care metrics need to be able to be properly adjusted for different populations.
  • The six domains of quality from the Quality Chasm report provide an important starting point, with one new potential concept being overall modifiable risk.

Population Health

  • Population health metrics can be divided into two categories: current health, such as length of life and quality of life, and future health, including factors that are both intrinsic and extrinsic to individuals.
  • The three-part aim requires measures for population health that are outside the traditional purview of the health care system. New responsibilities in communities will be needed to address those metrics.


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 39
5 Specifying the Shape of a Core Metrics Set KEY BREAKOUT GROUP THEMES Health Care • Each core metric needs to be independent of the others, and collectively they should be comprehensive. • Health care metrics need to be able to be properly adjusted for different populations. • The six domains of quality from the Quality Chasm report provide an important starting point, with one new potential concept being overall modifiable risk. Population Health • Population health metrics can be divided into two categories: current health, such as length of life and quality of life, and future health, including factors that are both intrinsic and ex- trinsic to individuals. • The three-part aim requires measures for population health that are outside the traditional purview of the health care system. New responsibilities in communities will be needed to address those metrics. 39

OCR for page 39
40 CORE MEASUREMENT NEEDS • Metric development should consider the eventual use of a mea- sure and its potential users and should aim for the measure to be actionable for the intended user. Cost • Cost metrics are needed to understand the drivers of cost and waste; to inform choices for plans, patients, and clinicians; to inform value-based payments; and to fuel transparency. • Three high-level concepts are of primary importance: risk- adjusted per capita costs, utilization, and affordability. • Operationalizing these metrics requires addressing issues involv- ing antitrust concerns, distrust among different organizations collecting data, the privacy provisions of the Health Insurance Portability and Accountability Act, and data standardization. An important component of this workshop was the 2-hour working group session, during which participants engaged in discussion to identify potential sets of core metrics for tracking progress toward better care, bet- ter health, and lower costs at national, state, community, organizational, and individual levels. Participants were assigned to one of three breakout groups, with each breakout group considering one dimension of the three- part aim. In addition to identifying the area’s metrics that were most im- portant to reliable assessment and monitoring of progress at these different levels, the participants were charged with identifying implementation chal- lenges associated with this dimension and proposing approaches to address those challenges. To assist each group in its deliberations, the workshop organizing committee, with the assistance of Institute of Medicine (IOM) staff, put together a packet of information on each specific aspect of measurement and included a table of potential metric categories with example metrics (see Table 5-1). The three groups took different approaches to assessing the suitability of those metric categories and to identifying making recom- mendations about additional categories and sub-categories. This chapter summarizes the discussions that took place during the breakout group discussions1 and in the subsequent discussions that involved all of the workshop participants. 1  The summaries of the working group discussions are intended to demonstrate the diver- sity of perspectives and divergent opinions and should not be construed to reflect any group consensus.

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 41 TABLE 5-1  Matrix of Potential Metric Categories Across the Three-Part Aim Metric Metric National Priority from Domain Categories Example Metrics National Quality Strategy Length of life Mortality, life expectancy, premature birth, preventable deaths Quality of life Physical health, functional status, disease burden, morbidity, pain, mental health, social functioning, injuries Health Smoking, exercise, alcohol behaviors and use, healthy diet, obesity Promote wide use of best risk Population practices to enable healthy Health Utilization Immunizations, dental living and well-being. of preventive health, appropriate (National Priority 1) services screening Community Safety, healthy food, health walkability/places to exercise, pollutants, healthy workplaces Social and Educational attainment, economic factors literacy, poverty, unemployment, health insurance status Effective Adherence to guidelines, Promote the most effective disease-specific prevention, treatment, and treatment targets (e.g., intervention practices for the cardiovascular disease: leading causes of mortality, control of high blood starting with cardiovascular pressure, cholesterol, disease. (National Priority 2) aspirin use) Health Patient- Experience of care; shared Ensure person- and family- Care centered decision making, shared centered care. goal setting, or patient (National Priority 3) inclusion in health care team, patient knowledge and understanding of care plan, clinical communications, supports for self-care continued

OCR for page 39
42 CORE MEASUREMENT NEEDS TABLE 5-1 Continued Metric Metric National Priority from Domain Categories Example Metrics National Quality Strategy Safe Preventable hospital Make care safer. admissions/readmissions, (National Priority 4) health care–associated infections, medical errors (composite measure: serious reportable events), inappropriate medication use, inappropriate maternity/newborn care, unnecessary tests, occupational safety in health care Coordination Experience of Promote effective and care transitions; communication and care communication communication among coordination. health care team members, (National Priority 5) Health including patient, family, Care and caregivers; appropriate continued sharing of health records; care consistent with preferences, particularly for end-of-life care Equitable Support of vulnerable Elements captured in National populations, Priorities 1, 3 and 5 communication appropriate to individual and community health literacy Efficiency and Access to needed care, Elements captured in National timeliness consistent insurance, Priorities 5 and 6 (includes achievement of meaningful operations) use of health IT, appropriate nurse staffing, effective management

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 43 TABLE 5-1 Continued Metric Metric National Priority from Domain Categories Example Metrics National Quality Strategy Cost Affordability Costs for households/ Make quality care affordable individuals, impact on for people, families, wages/benefits, impact on employers, and governments. other government services (National Priority 6) (local, state, national) Expenditures Overall annual spending on health care (per member per month, per capita, per episode, per service), utilization of services Indirect costs Absenteeism, productivity Waste Unnecessary services (includes costs due to unwarranted variation/ overuse), fraud, excessive administrative costs, inefficiently delivered services, prices that are too high, missed prevention HEALTH CARE BREAKOUT GROUP Key points from the breakout group discussion chaired by David S ­ tevens, associate chief medical officer and director of the Quality Center at the National Association of Community Health Centers and research professor at the George Washington University School of Public Health and Health Services, are summarized here. Before starting this discussion, Mary Barton, vice president for performance measurement at the National Com- mittee for Quality Assurance, gave a brief synopsis of the state of the field. Defining a Core Metric and Understanding Current Limitations The group began by considering the fundamental characteristics of a core metric set. An important characteristic of a set of core metrics is that each measure should be mutually exclusive but that collectively they should be exhaustive, that is, when put together they provide a unified picture of progress on the three-part aim. The denominator of a core metric should be adaptable for different populations and population sizes so that the metric

OCR for page 39
44 CORE MEASUREMENT NEEDS makes it possible to drill down for more detailed analysis. The group also noted that it was important that metrics balance cost and quality of care in a way that does not overemphasize cost at the expense of the entire three-part aim. Finally, it was deemed important that any metrics of health outcomes be adjusted for risk. The group also identified several current limitations of core measures. Many metrics today appear to be snapshots that reflect episodes of care at a specific moment in time rather than on a specific service or condition, and the breakout group expressed some concern that such metrics may not be that useful in a learning environment. A core metric should also be sensitive to change, a point that Stevens stressed, and should connect to a system that can learn over time, track results, and improve as a result of changes in the metric. Reviewing Potential Metric Categories Several members of the breakout group indicated their comfort with the initial set of measurement categories contained in the background material, which were largely drawn from the Quality Chasm definition of quality (IOM, 2001). To improve this initial set by reducing the number of domains, some participants proposed that a timeliness metric belonged both in patient-centered care and coordinated care and that communication should be included as part of patient-centered care. Attendees had differing views on whether an equitable care metric should span all domains or if doing so would cause it to be lost. Stevens noted the latter concern could be addressed by emphasizing that all populations should be examined across those domains. One proposed overarching concept was overall modifiable risk. Cur- rently this metric is used only for cardiovascular disease, but the concept could be extended to many conditions and diseases. This metric would be actionable at multiple levels, from the patient to the system level. A con- ceptual area that was raised in the discussion involved engaging providers. Although there has been significant attention paid to developing metrics that assess how engaged patients are in their own care, little work has been done to measure the changes and values that clinical professionals need to operate in a system designed around the three-part aim. These changes include working in teams, collaborating across organizations and regions, and being able to work with patients in self-management. Another area of need is for metrics that can assess the local capacity to assemble providers, public health structures, and community organizations to reach the three- part aim. Examples of core metrics for the three-part aim include the compos- ite Consumer Assessment of Healthcare Providers and Systems score for

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 45 patient experience, risk-adjusted mortality, functional status, a composite safety measure that is now under development, readmissions, and ambula- tory care–sensitive readmissions. The calculation and use of many of these measures are constrained by the limited amount of consistent data collected across the wide range of health care delivery settings. POPULATION HEALTH BREAKOUT GROUP Participants in this group, chaired by Patrick Remington, associate dean for public health at the University of Wisconsin School of Medicine and Public Health, focused on three topics: defining populations, selecting metrics, and measuring community health. Before discussing these topics, Steven Teutsch, chief science officer of the Los Angeles County Health De- partment, gave a brief overview of the current state of population health measurement. Defining Populations Populations at risk serve as the denominator of any metric dealing with population health, making it essential to define specific populations. Geopolitical boundaries are an obvious defining feature, one with which public health is comfortable. However, health care usually thinks about populations of patients in a care system or populations of people with a particular condition, neither of which aligns with geopolitical boundaries. The definition of population, Remington said, depends on who is asking the question, whether it be policy makers, government entities, health systems, or patient groups. Selecting Metrics In talking about the categories for metrics of population health, partici- pants set forth two general categories: current health, and factors that help predict health in the future. Health outcomes are essentially measures of death and disease or mortality and morbidity. The group noted that many metrics could be suitable for assessing these concepts, including health- adjusted life years, premature death rates, health-related quality of life, and disease incidence. The group discussed briefly the notion of using rates of indicator dis- eases, such as heart attack rates, as sentinel indicators of health system performance. It also talked at length about whether determinants of health could be used to measure risk in populations, in terms of both factors that are intrinsic to individuals and factors that are extrinsic to individuals. Health risk appraisal was also mentioned as a potentially useful metric.

OCR for page 39
46 CORE MEASUREMENT NEEDS Quality of life, whether measured in terms of morbidity or judged by the individuals themselves, is an important metric, and both mental and physical health play a role. Group members noted that there may be spe- cific measures that might be proxies of quality of life, such as the percent of 60-year-old adults who have had a knee replacement. The group also discussed the idea that the three-part aim requires measures for population health that are outside the traditional purview of the health care system and that it would be the responsibility of those communities to address those metrics. The group also explored the concept of community health. A healthy community is one with a robust combination of supporting elements, such as safety, quality of the educational system, the availability of jobs, and others. The breakout group noted that performance metrics for community health should be measured in context, that is, not just by the health or by the length or quality of life of individuals, but relative to the conditions in which people live. Implementation The group’s participants conducted a substantial discussion about im- plementing metrics of population health. Remington captured five themes from this discussion. The first was that words matter: Measures such as health-related quality of life may be understood by policy makers, but the general public may not be familiar with this phrasing. Terms such as “healthy community” and “community well-being” are ambiguous and can have different meanings for different constituencies. Though the group did not solve the question of which words to use, it did recommend that experts in communication be brought into future discussions. The group had a heated discussion about whether surveys or broad- based, all-inclusive individual questions are the best approach to collecting data on community health metrics. Both methods have costs associated with them that can be substantial, whether those costs are episodic in the case of surveys or ongoing for individual questionnaires given at the time an individual receives health care services. The group did agree that information needs to be collected on states of wellness and well-being and not be limited to people who are interacting with the health care system at any given moment. This is particularly important given that there are many people who are not in a structured health care system but who may constitute a substantial portion of the overall population. While indices can serve as important metrics, it is important that they be transparent and not just a black box measurement. They can be helpful for summarizing information, but they must be amenable to drilling down. There was also a debate on when metrics should focus on everyone ver-

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 47 sus the leading innovators in promoting health. By looking at innovators, metrics can provide a glimpse of the future of where the health care system could go as opposed to where it has been. Finally, all metrics should be actionable in the short term. Long-term metrics may be interesting to researchers, but they hold little sway over policy makers. Remington noted that this is not a theoretical exercise given that there are places that already measure the health of communities or populations and where public health and health care systems are working together. COST BREAKOUT GROUP The working group participants discussed several aspects of cost as viewed from different perspectives. Kate Goodrich, senior medical advisor in the Office of Clinical Standards and Quality at the Centers for Medicare & Medicaid Services, chaired the group, and Dennis Scanlon, professor of health policy and administration at Pennsylvania State University, sum- marized the current state of cost measurement before the group began its work. Purpose of Measurement In a short, straightforward discussion, the group first addressed the is- sue of identifying the purposes that cost metrics should serve. Core metrics need to be related to the aim of lowering cost in general—and to lowering per capita costs specifically—and they should help explain variations in trends. Cost metrics are also needed to understand the drivers of cost and waste; to inform choices for plans, patients, and administrators; to inform value-based payments; and to fuel transparency. After a more heated discus- sion, participants in the group leaned toward a definition of cost that refers to health care spending within the delivery system, that is, what is spent on payments to providers and out-of-pocket expenses for patients, including the cost of insurance. This definition of cost does not include the bigger issues of public health, such as education and crime reduction. Core Metrics The group spent much of its time discussing high-level core metrics and outlined three of primary importance. The first two measures were straight- forward: per capita costs and a measure that assesses utilization. These would be risk-adjusted measures, and they could be analyzed by delivery systems, geographies, and populations. The third metric, which several at-

OCR for page 39
48 CORE MEASUREMENT NEEDS tendees felt was novel and actionable, was an affordability measure, since a primary goal of the three-part aim is to make health care affordable. While the working group participants agreed that there are many good subsets of metrics within these three core metric categories, there was not sufficient time to discuss these measures. However, the group recommended that settling on broad subsets of measures should be the next order of business. The group also discussed how actionable these metrics would be—for example, what a provider can do once it knows its per capita costs. Several participants noted the need for core metrics and sub-metrics to be under- standable at the local level and at greater levels of aggregation. The group debated whether there should be an underlying set of appropriate resource measures, but it did not reach agreement on this point and agreed that more discussion was needed to resolve this issue. Implementation The group identified a number of implementation issues that will need to be resolved, including antitrust issues, distrust among different orga- nizations collecting data, the privacy provisions of the Health Insurance Portability and Accountability Act, and data standardization. The group noted that benefit design is a major issue when measuring per capita cost because of variations in attribution, legal provisions, and paying party. Fragmentation in the system across all levels is a major stumbling block to implementing meaningful cost metrics. During the general discussion, sev- eral participants remarked that data collection for cost metrics needs to be standardized at the national level in such a way that states and communities can then use this metric. It was also noted that issues over who controls cost data need to be resolved. DISCUSSION Following the reports from the three breakout groups, IOM staff pre- pared a summary of the specific metrics and categories that were discussed in the breakout sessions (see Tables 5-2, 5-3, and 5-4). These summaries were then the subject of a discussion among all the workshop participants. That discussion is summarized here. Population Health Multiple attendees voiced support for creating two major metric cat- egories for current and future health. One participant suggested that exam- ple metrics in the future health category could be intrinsic risk, as measured

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 49 TABLE 5-2 Summary of Population Health Breakout Group Discussion Metric Domain Metric Categories Example Metrics Implementation Current Health Length of life: •  efining the D Mortality, life population expectancy •  ommunication/ C Quality of life: education about Morbidity, measures functional status, indicator diseases • Data collection Composite: QALY, •  ransparent T HALY methods for Population Health composites/ Future Health Health determinants: indices Health risks, health behaviors, healthy •  ension in T communities, targeting and extrinsic innovators or all determinants actors •  ctionability of A measures NOTE: HALY = health-adjusted life year; QALY = quality-adjusted life year. at the individual level in terms of health behaviors and perhaps genetic predisposition, and extrinsic or community risk, which would be measured using social and ecological models and include some measure of place-based risk. Another participant remarked that quality-of-life metrics in the cur- rent health category should include some measure of self-reported health. There was some discussion about how to phrase metrics in the future health category so that the general public will understand the concepts involved. It was noted that the term “health determinants” failed to resonate with consumers when it was tested in Wisconsin, and instead researchers there are using the term “health factors” and explaining it as things that will predict how healthy an individual will be in the future. The term “health influences” was also suggested. There were several comments about how to conceptualize popula- tion health as it moves from the entire population to subpopulations. One participant remarked that population health at the level of a 5,000-person accountable care organization serving Medicare patients is not going to look the same as a subpopulation of Medicaid patients or a subpopulation in the hundreds of thousands and wondered how those differences will be

OCR for page 39
50 CORE MEASUREMENT NEEDS TABLE 5-3 Summary of Health Care Breakout Group Discussion Metric Domain Metric Categories Example Metrics Implementation Patient centered Patient experience: •  isk adjustment R HCAHPS metric critical Equitable •  ppropriateness of A care Timeliness •  imeliness of care T Effective Mortality amenable to under all metrics health care (both in initial access and time to Functional status return to function) Equitable Safe Composite medical harm measure Health Care (including medical errors and health care– associated infections) Equitable Efficient Utilization: Admissions and readmissions for ambulatory care sensitive conditions Equitable Coordination and Timeliness communication NOTE: HCAHPS = Hospital Consumer Assessment of Healthcare Providers and Systems. reflected in the implemented metrics. Participants also asked how measures of population health across subpopulations should reflect the concepts of equity, health disparities, insurance status, and access to care. Health Care When the breakout group reported its discussion to all workshop attendees, Marcus Thygeson, vice president for medical services at Blue Shield of California, remarked that it would be useful to have metrics that measure how patients do over time. Such metrics would assess the path to recovery and the speed of recovery, and they would differ from a timeliness

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 51 TABLE 5-4 Summary of Cost Breakout Group Discussion Metric Domain Metric Categories Example Metrics Implementation Total cost of care Total cost of care •  ntitrust A (actual costs) metric, actual cost and risk-adjusted. •  IPAA H Population-based per member per month •  roprietary P (all conditions) interests Total cost of care Total cost of care •  tandardization S (standardized costs) metric, standardized costs and risk- •  overnance G adjusted. Population- based per member per •  ttribution A month (all conditions) Affordability Percent of household •  egal L spending on health •  osts C Percent of national GDP and/or federal government health Cost care spending as percent of total federal government spending Percent of economy, governmental budgets, organizational budgets, or individual budgets devoted to specific programs or sources of payment, including employer- based health benefits, Medicaid, Medicare, and spending by the uninsured. Premiums

OCR for page 39
52 CORE MEASUREMENT NEEDS metric or one that measured functional status. During this same discussion, Stevens noted that health care measures could be analyzed individually or be combined to support real-time improvement for individual patients, populations, providers, or health systems. During the discussion, two participants from the health care breakout group raised suggestions for improving the summary list of measures by including several of the example measures from the background materials. Another participant suggested that the concept of appropriateness of care should be added under the effectiveness category and that a measure of composite medical harm should be included in the safety category. It was also recommended that ambulatory care admissions and readmissions be included in the efficiency category. One place to start in creating such a measure would be existing lists of conditions that should be treated on an outpatient basis. Another participant, who had been part of the cost breakout group, asked if efficiency should be a subset of cost and if equitable care should be a metric itself rather than merely an example. A participant suggested that functional status should be broadened to that of overall health status, reflecting the changes in a patient’s overall health after receiving health care services. It was noted that patient-reported outcomes could serve as a general measure of functional status. There were also comments about implementation, particularly con- cerning the settings in which the data for these metrics would be collected. It will be important going forward, one participant noted, to identify the least disruptive setting or most accessible and least expensive setting in which to collect data if these metrics are to be truly useful on a large scale. A participant commented that a number of professional organizations are developing and implementing process metrics that are condition-specific and that these metrics are being well received by the medical community. Cost The discussion highlighted the fact that total cost depends on two factors—the price of health care services and the utilization of those ser- vices. Unpacking overall costs in this way allows for a better understand- ing of which of these two factors is driving cost. However, one participant noted that this may lead to conceptual issues, as utilization can sometimes be considered in the health care quality domain of the three-part aim as opposed to the cost domain. For example, one participant noted that good health care services can directly reduce utilization, but their impact on cost may be unclear because of the multiple business layers between utilization and cost. Another participant remarked that utilization can be appropriate or inappropriate and that perhaps a metric for waste should be included

OCR for page 39
SPECIFYING THE SHAPE OF A CORE METRICS SET 53 in the cost domain, while another participant suggested further delineating waste into under-utilization and over-utilization. Concerning the affordability metric, participants noted that the ex- amples should also include measures of health care spending by states and communities as a percentage of state and local community economic output in addition to the comparison of health care spending with national gross domestic product. Other participants noted that similar metrics could be included that illustrate the level of spending devoted to multiple programs and sources of payment, such as Medicare, Medicaid, employer-sponsored health programs, and health spending by the uninsured, with this spending normalized by the size of the economy, governmental budgets, organiza- tional budgets, or individual budgets. One speaker suggested that premiums be included in the percentage of household spending on health and not listed separately, and another participant suggested there should be a metric involving value; while the members of this workshop all understand that value is included throughout these metrics, the participant noted, it would be useful to make this explicit for the general public. REFERENCE IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.

OCR for page 39