2

Criteria for Selecting Measures

The committee was asked to describe methods for selecting quality measures for the Leading Health Indicators (LHIs) and for the work of improving the public’s health more broadly. In this chapter, the committee offers guidance for selecting measures to assess and improve quality in the multisectoral health system, and especially among health care and public health participants.

To make progress toward long, healthy lives for all, pertinent stakeholders need to create healthy communities and address the underlying factors that influence health outcomes and disparities in those outcomes among different subgroups. Measurement is an essential ingredient of such efforts. The committee believes that a summary measure of population health is needed along with or as a part of a set of measures of quality. Such a measure is needed for every community and must be consistently defined in order for public health officials and members of the public to understand the quality of their public health system and to compare it with national and other appropriate benchmarks of quality (e.g., summary scores measuring system quality for peer counties or states and for the nation), and to support mutual accountability among partners and contributors in the multisectoral health system. The committee reviewed the 2011 Institute of Medicine (IOM) report For the Public’s Health: The Role of Measurement in Action and Accountability and endorses its rationale and recommendation for the national adoption of a summary measure of population health equivalent to health-adjusted life years (HALYs) or health-adjusted life expectancy (HALE) (IOM, 2011a).

It is incumbent on decision makers and professionals working in quality improvement to consistently answer the question “Quality for whom?” Furthermore, once this question is answered, the public health agencies and the health care delivery organizations need policies to trans-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 27
2 Criteria for Selecting Measures The committee was asked to describe methods for selecting quality measures for the Leading Health Indicators (LHIs) and for the work of improving the public’s health more broadly. In this chapter, the commit- tee offers guidance for selecting measures to assess and improve quality in the multisectoral health system, and especially among health care and public health participants. To make progress toward long, healthy lives for all, pertinent stake- holders need to create healthy communities and address the underlying factors that influence health outcomes and disparities in those outcomes among different subgroups. Measurement is an essential ingredient of such efforts. The committee believes that a summary measure of popula- tion health is needed along with or as a part of a set of measures of quali- ty. Such a measure is needed for every community and must be consistently defined in order for public health officials and members of the public to understand the quality of their public health system and to compare it with national and other appropriate benchmarks of quality (e.g., summary scores measuring system quality for peer counties or states and for the nation), and to support mutual accountability among partners and contributors in the multisectoral health system. The commit- tee reviewed the 2011 Institute of Medicine (IOM) report For the Pub- lic’s Health: The Role of Measurement in Action and Accountability and endorses its rationale and recommendation for the national adoption of a summary measure of population health equivalent to health-adjusted life years (HALYs) or health-adjusted life expectancy (HALE) (IOM, 2011a). It is incumbent on decision makers and professionals working in quality improvement to consistently answer the question “Quality for whom?” Furthermore, once this question is answered, the public health agencies and the health care delivery organizations need policies to trans- 27

OCR for page 27
28 TOWARD QUALITY MEASURES FOR POPULATION HEALTH late into action the broad goal of addressing disparities in health status. It is not enough to monitor and report disparities; measurement can also serve as a tool to ascertain whether inputs (e.g., resources and capacity) and processes (e.g., policies, programs, and services) are succeeding in moving health improvement efforts toward the desired health outcomes and greater equity. Equity across all groups by age, socioeconomic sta- tus, race, gender, and sexual orientation can be measured as ranges (i.e., differences among groups), and using such metrics as the Gini coeffi- cient, the index of dissimilarity, the slope index of inequality, and the index of disparity developed for Healthy People 2010 (Pearcy and Keppel, 2002). It is possible to publish health-adjusted life expectancy data by quintiles of income, sex, race, and ethnicity (see, for example, the evidence reviewed by Clarke et al. [2010]). One strategy for ensuring that equity is a guiding principle from the beginning is to apply a con- sistent and scientifically justifiable approach to race/ethnicity and socio- economic classification in the process of population measurement. Also, it is not enough to look at descriptive data; rather, it is important that sys- tem inputs be used to drive improvement in achieving equitable out- comes. Asada and colleagues (2013) have proposed a potentially useful analytic approach to measuring disparities, using functional limitation data (i.e., activities of daily living) from the 2009 American Community Survey to develop disparity profiles by states, showing whether dispari- ties were associated primarily with race and ethnicity, socioeconomic factors, or both. Because the social determinants chapter in Healthy Peo- ple 2020 was under development during the deliberations leading to se- lection of the LHIs, those factors are minimally reflected in the LHIs. Although it is beyond this report’s scope to discuss the social determi- nants of health in depth, measuring the inequities (in inputs) that lead to disparities in outcome provides essential information about systemic challenges or obstacles at the level of government policy and institutional practices (see, for example, the IOM [2003c] report Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care). The committee attempted to apply a coherence-creating, unifying, standardized quality measurement approach to the LHIs, but the LHIs were not designed primarily from a quality perspective. The final selec- tion of LHIs for Healthy People 2020 was made by HHS with input from the IOM (2011b)1 and the Secretary’s Advisory Committee on Health 1 The IOM’s 2011 report Leading Health Indicators for Healthy People 2020: Letter Report does provide that report’s authoring committee’s rationale for selecting candidate LHIs for HHS (IOM, 2011b).

OCR for page 27
CRITERIA FOR SELECTING MEASURES 29 Promotion and Disease Prevention (Honore et al., 2011). The present IOM committee recognizes that HHS and its advisers had to reconcile many considerations in a complex process that included considering the burden of disease; managing mandates, resources, current priorities, and capabilities; weighing and balancing stakeholder interests and expecta- tions; and making important tradeoffs. This array of factors and forces explains both the very large and heterogeneous set of Healthy People 2020 objectives (approximately 1,200 that cover many but not necessari- ly all top burden of disease priorities) and the heterogeneity or dissimi- larity of the LHIs (population vs. clinical/individual, disease specific vs. risk behavior, related to one outcome/endpoint vs. pertinent to a dozen outcomes/endpoints), which presented a challenge to the IOM committee in its attempt to identify a coherent set of quality measures. There were also some constraints on the HHS advisory committee’s work. For example, one requirement was that the LHIs be drawn from the Healthy People 2020 objectives. As a result of the many considera- tions and constraints made in selecting them, the resulting set of 26 LHIs (see Box 1-1 for the complete list of LHIs) is highly heterogeneous: some LHIs relate to outcomes, while others relate to processes. Four of the LHIs—fatal injuries, homicides, infant deaths, and suicides—are themselves outcome measures. Another four are intermediate outcome measures (preterm births, adolescents with major depressive episodes, adults with controlled hypertension, and adult diabetics with poor glucose control) and 12 LHIs could be classified as measures of the effectiveness of an intervention. When the LHIs are organized according to the logic model above, most measures fit in the categories of Healthy Conditions and Health Outcomes. Few LHIs fit in the category of Interventions (one example is receiving primary care services), and none fits under Resources and Ca- pacity. Despite the lack of LHIs in this last category, measures in this area are an important contributor to a system’s success in moving toward desired outcomes. For example, research exploring the relationship be- tween public health funding and outcomes is still in its early stages, but Mays and Smith (2011) have provided suggestive evidence that public health funding is correlated with better health outcomes. Some aspects of Resources and Capacity were discussed in a previous IOM report (2012), and reviewing that report’s discussion of the “foundational capabilities” of public health agencies shows a great deal of congruence with the six drivers of quality improvement in public health. The committee did not set out to replicate the many causal networks that could be developed to show how various system inputs lead to cer-

OCR for page 27
30 TOWARD QUALITY MEASURES FOR POPULATION HEALTH tain intermediate outcomes that result in ultimate population health out- comes. This has been done by others for various chronic diseases, includ- ing the Centers for Disease Control and Prevention (CDC)/ National Institutes of Health (NIH) Prevention Impacts Simulation Mod- el (PRISM) described in Homer and colleagues (2010) and also the work of Jones and colleagues (2006), and Wolfson and Rowe (2001), and al- though many of the models available have specific purposes and limita- tions, they illustrate the complexity of causal factors and the interactions among them. They also suggest important methods for structuring and understanding the relationship of specific interventions to specific health outcomes, and they help quantify components and indicate which inter- ventions might be best in a specific place to achieve a desired set of out- comes. A previous IOM committee called for modeling to help inform decision makers about interventions that are likely to have the greatest impact (IOM, 2011a). That committee recommended “that the Depart- ment of Health and Human Services (HHS) coordinate the development and evaluation and advance the use of predictive and system-based simu- lation models to understand the health and consequences of underlying determinants of health. HHS should also use modeling to assess intended and unintended outcomes associated with policy, funding, investment, and resource options” (IOM, 2011a, p. 103 [Recommendation 6]). Such work would also help identify, develop, and refine measures of quality. In Chapter 3 (see Table 3-1), the committee offers sample measures along the trajectory of the various LHIs. For example, the update of childhood vaccines is an intermediate outcome, whose ultimate outcomes are the rates of morbid- ity and mortality from certain infectious diseases. GENERAL CONSIDERATIONS FOR SELECTING AND PRESENTING MEASURES OF QUALITY The committee provides some preliminary considerations for select- ing measures of quality related to the LHIs and for population health more broadly. Also, as previously noted, the quality characteristics do not apply directly to the LHIs or to measures of quality. Measures cannot in and of themselves be thought of as effective, population-centered, or transparent. However, several of the characteristics are embedded in the criteria for measure selection as described below. Measures of quality may measure outcomes or intermediate outcomes, may reflect on effi-

OCR for page 27
CRITERIA FOR SELECTING MEASURES 31 ciency or on cost-effectiveness2 and will ideally be associated with evidence-based interventions (e.g., programs and policies). In its deliberations, the committee agreed that although measures of quality may be presented as long lists or catalogues (similar to the Na- tional Quality Forum–endorsed system of almost 700 measures, or as an extension of the 1,200 Healthy People 2020 objectives themselves), a set or portfolio of measures is likely to be more useful to—and thus, more used by—practitioners involved in quality improvement, and also more informative to communities and other audiences wishing to understand how the system is performing. There are at least two such comprehen- sive, but parsimonious, measure portfolios currently in use: America’s Health Rankings (AHR), which uses 44 measures, and the County Health Rankings (CHR), which uses 30 measures in 5 domains (Remington and Booske, 2011). The AHR has been published since 1990 (United Health Foundation, 2012) and represents an effort to quantify and analyze the status and changes in health in the U.S. population from year to year. The information used is from publicly accessible data, collected mostly by the federal government. A major data source is Behavioral Risk Factor Surveillance System (BRFSS) of the Centers for Disease Control and Prevention (CDC), but it has limited usefulness at the local level. AHR reports the rankings by state and includes sections on health disparities and how the United States compares to other countries. More recently, CHR began rankings by county within each state. CHR is based at the University of Wisconsin Population Health Institute, which has ranked Wisconsin’s counties since 2003. CHR uses data from publicly available local and federal government sources. Other pertinent measurement ac- tivities include the CDC Community Health Status Indicators, which provides a portfolio of 200 measures, with a set for each of the 3,141 U.S. counties, and allows comparisons between peer counties to inform quality improvement efforts and the sharing of best practices. The 2011 IOM report For the Public’s Health: The Role of Measurement in Action and Accountability provides additional descriptions of indicator efforts. In the realm of health care delivery there also are multiple ongoing ef- forts, ranging from the National Committee on Quality Assurance HEDIS (Healthcare Effectiveness Data and Information Set) measures, to the hundreds of additional measures from many other organizations provided as part of the National Quality Forum’s Quality Positioning System, its measure endorsement process. Finally, the Agency for 2 The “extent to which the results are achieved at a lower cost compared with alterna- tives” (World Bank, 2007, p. 65).

OCR for page 27
32 TOWARD QUALITY MEASURES FOR POPULATION HEALTH Healthcare Research and Quality (ARHQ) National Quality Measures Clearinghouse contains a large collection of measures (more than 2,000)—the vast majority of measures belong to the health care delivery domain and small number are listed under the population health domain.3 CRITERIA FOR MEASURE SELECTION In comments made at the committee’s first meeting, the Assistant Secretary for Health suggested the committee provide methodological guidance that could be used in selecting measures of quality. In response, the committee reviewed a number of existing resources for sample crite- ria (see Table 2-1 below and Table B-1 in Appendix B for a description of criteria across several different sources). The committee’s review in- cluded the National Quality Forum’s criteria for evaluating measures, the Institute for Healthcare Improvement’s key measurement principles that apply to the Triple Aim, the HHS Secretary’s Advisory Committee on Health Promotion and Disease Prevention Objectives for 2020 operation- al criteria for selection of LHIs, the 2003 IOM report Priority Areas for National Action: Transforming Health Care Quality and the 2010 IOM report Future Directions for the National Healthcare Quality and Dis- parities Reports. The AHRQ National Quality Measures Clearinghouse (not in the table) provides the following criteria for measures included in their database: (1) the measure is cited in peer-reviewed/National Library of Medicine journal, (2) the measure has documented evidence of relia- bility and validity, and (3) the measure has been developed, adopted, adapted, or endorsed by an appropriate organization.4 With the exception of the criteria for selecting LHIs, the sets of crite- ria listed were developed for measures largely pertinent to health care delivery, and the committee did not feel that any existing set of measures was sufficiently specific for selecting measures of quality for the multisectoral health system. Because none of the sets of criteria listed is primarily oriented toward population health, they miss some of the nu- ances that are important in population health interventions. For example, data availability at the state and local level is a critical issue for popula- tion health measures, but not for clinical care measures, where data are intrinsically part of the medical record. 3 The population health domain includes dimensions such as population health state, so- cial determinants of health, and environment. 4 See http://www.qualitymeasures.ahrq.gov/about/inclusion-criteria.aspx (accessed May 31, 2013).

OCR for page 27
CRITERIA FOR SELECTING MEASURES 33 Topically similar criteria in these sets were grouped into five catego- ries: (1) impact or importance of the condition or outcome to be meas- ured; (2) improvability, or the extent of the gap between current practice and evidence-based best practice and the likelihood that the gap can be closed; (3) scientific soundness of the measure, including validity and reliability; (4) geographic, temporal, and population coverage to ensure that the measure has sufficient granularity to be useful in monitoring ac- tions to improve health at different geographic levels in important popu- lation subgroups; and (5) data availability to ensure that data are readily available in a form useful for quality and performance measurement (see Table 2-1 and Appendix B). The first two categories—impact and improvability—refer to characteristics of condition(s), outcome(s), and associated interventions that a measure would address, while the final three categories—scientific soundness, coverage, and data availability— address characteristics of the measures themselves. Thus, the committee developed a set of criteria that embody most characteristics of these ex- isting sets, while using words and providing definitions that indicate characteristics that are more relevant for population health, and empha- sizing certain criteria that are particularly important for population health, such as coverage at the state and local level, where many population-based interventions are implemented. The committee also recognizes the potential usefulness of a stepwise approach to applying TABLE 2-1 Quick Comparison of Published Criteria for Measure Selection (Detailed Table Provided in Appendix B) Published Criteria Category of HHS SAC, Criteria NQF, 2012b 2011 IOM, 2010 IOM, 2003b Impact (importance) X X X X Improvability X unclear X X Scientifically sound measure X n/a X n/a Geographic, tem- poral, and popula- tion coverage n/a X X unclear Data availability X n/a X n/a NOTES: X indicates that the published criteria included one or more items in the category listed in the first column; n/a = not applicable.

OCR for page 27
34 TOWARD QUALITY MEASURES FOR POPULATION HEALTH these criteria, in which the target of potential measures is first evaluated for importance and improvability (seen through a population health lens), followed by an evaluation of the measures characteristics (see IOM, 2010, pp. 69-74). Finding 2-1: The committee finds that partners in the multisectoral health system currently use a vast and com- plex array of measures of quality in a manner that seems uncoordinated. This finding refers to measures of public health relevance, thus ex- cluding measures that are largely pertinent to clinical care. Developing a more coordinated approach would include paying attention to the three purposes of measurement: assessment, improvement, and accountability. A previous IOM committee provided a measurement framework for ac- countability that acknowledged the different forces at work when ac- countability is contractually required compared to when accountability is informal or “soft”—what that committee termed “compact accountabil- ity” (IOM, 2011a). A challenge identified by the previous committee also persists, in the proliferation of metrics, with limited effort to coordinate, consolidate, and organize in a manner that increases coherence and re- duces overlap and duplication. RECOMMENDATION 2-1: The committee recommends that the Department of Health and Human Services and its partners in population health improvement (e.g., public health agencies, health care organizations, and communi- ties) adopt a portfolio of measures of the quality of the multisectoral health system. The portfolio of measures should a. include summary scores reflecting population-level health outcomes and healthy conditions. b. balance parsimony with sufficient breadth. c. inform assessment, improvement, and accountability of the multisectoral health system. When it refers to summary scores, the committee envisions  a “healthy outcomes” summary score that is a composite of (the quantitative values for) all outcome measures selected in a port-

OCR for page 27
CRITERIA FOR SELECTING MEASURES 35 folio (the summary measures of population health, such as HALE, could serve as such a score, but others could be devel- oped to reflect several different health outcomes of interest); and  a “healthy conditions” summary score (e.g., such as the commu- nity well-being indicator described in IOM [2012]) that is a composite of all intermediate outcomes (or determinants of health) measures selected in a portfolio. These scores and values could be made publicly available through regu- lar reports and other channels of communication. A parsimonious (or manageable) portfolio will not be exhaustive or exhausting, it will instead consist of a small number of the most im- portant things, up to, say, four or five per LHI topic. The portfolio will also reflect relevant different areas of the logic model across measures and within each topic, and the measures will not be redundant or over- lapping. Measurement has three somewhat overlapping purposes. Measures are needed for the assessment of overall quality in the multisectoral health system, beginning with a focus on governmental public health and health care (where the areas of responsibility and ac- countability are more clearly delineated) and moving on to the roles and responsibilities of other stakeholders. Measures are also necessary to in- form quality improvement and to demonstrate accountability for popula- tion health improvement. Measures of quality will inform clinicians and health care organizations, public health agencies, and others in their community health improvement process work at the state and local lev- els. Measures of quality will also assist members of the public and their elected officials to hold public health officials and other stakeholders, involved in communitywide programs and efforts to improve health, ac- countable for the quality and effectiveness of their actions (IOM, 2006). RECOMMENDATION 2-2: The committee recommends that the Department of Health and Human Services and other relevant organizations adopt the following set of cri- teria for selecting and prioritizing measures of quality for use in population health improvement, including the Lead- ing Health Indicators:

OCR for page 27
36 TOWARD QUALITY MEASURES FOR POPULATION HEALTH Criteria for conditions or outcomes to be measured a. Reflective of a high preventable burden5 b. Actionable at the appropriate level for intervention Criteria for the measures c. Timely d. Usable for assessing various populations e. Understandable f. Methodologically rigorous g. Accepted and harmonized Judging the criterion “reflective of a high preventable burden” re- quires estimates of the frequency of the condition or disease entity and the effectiveness of the interventions. This means that the measures need to refer to interventions and outcomes related to health conditions that account for considerable morbidity and mortality and that are also pre- ventable. A standard measure is needed to determine what is a large bur- den and what is not, but context matters as well. As an example, the preventable burden of infant mortality is likely relatively small since in- fant mortality, however catastrophic, is still relatively low. It is likely on the LHI list for several reasons: these are catastrophic events, rates in some groups are much higher than in others, it is an indicator of many issues in the clinical care system and in society as a whole, and it is a measure used in international comparisons. Assessing the evidence of effectiveness may be done by reviewing published findings of the Guide to Community Preventive Services, the Cochrane Collaboration, and sim- ilar entities. The following characteristics of interventions are important to consider: the feasibility of implementing the intervention (particularly the feasibility in different settings); the potential for improvement (e.g., for a specific health condition, an intervention already delivered at a very high level has less potential for improvement than an equally effective intervention delivered at a lower level); and the extent to which the inter- vention has externalities (i.e., benefits other conditions or has non-health benefits). The corollary to the “effectiveness of the intervention” condi- 5 The concept of high preventable burden has two components: high burden and existence of effective interventions. This concept (burden × effectiveness), refers to burden as the absolute burden, not relative burden. In other words, a condition like phenylketonuria (PKU) has a high preventable burden if one thinks of the denominator as all people with PKU, but it is a low absolute preventable burden if one uses the entire population.

OCR for page 27
CRITERIA FOR SELECTING MEASURES 37 tion is a case where the magnitude of a problem is such that action is necessary though there is inadequate information on effective interven- tions (see IOM, 2011a). For example, evidence may be limited to lessons learned or preliminary information on best practices, or there may be multiple potential interacting interventions and insufficient evidence to help pinpoint the most effective. Also, the context of interventions is a critical factor to consider. There are different considerations for measures of relevance to the local level compared to those with relevance to the national level. At the local level, interventions informed by limited evi- dence, including perhaps emerging best practices or the equivalent, may be tested, so measures of quality are needed. In the absence of evidence- based recommendations from, for example, the Guide to Community Preventive Services, interventions or combinations of interventions must have an associated rigorous evaluation. Although the committee concluded that the nine characteristics of quality in public health do not directly align with measures of quality, it recognizes that a measure that reflects a high preventable burden enables organizations to assess whether their interventions are effective, efficient, equitable (because vulnerable populations may bear a higher burden of, for example, infant mortality) and risk reducing. The criterion “actionable at the appropriate level of intervention” reflects whether a measure provides sufficient information about a prob- lem to help identify a way to address it, and whether there are effective programs and policies that can be adopted by relevant stakeholders (e.g., local jurisdictions). Finding an effective programs may be done both by referring to systematic reviews, such as those provided by the Guide to Community Preventive Services and the Cochrane Collaboration, or by looking to other rigorous efforts to identify best practices, such as the work of the Public Health Law Research program in establishing the Law Atlas, which could in the future help identify associations between changes in policy and health outcomes.6 The criterion “timely” refers both to data for a measure (1) being collected frequently enough to make it possible to track changes in the measure that reflect actions intended to affect the outcome or condition and (2) being made available quickly enough (e.g., within 6 months of collection) to be acted on. “Usable for assessing various populations” means that data are avail- able and can be used to assess different populations (e.g., defined by de- 6 See http://lawatlas.org/about.

OCR for page 27
38 TOWARD QUALITY MEASURES FOR POPULATION HEALTH mographics or defined by living in a certain zip code) and at different levels (national, state, local). 1. At the total population level and also, to allow levels of equity to be observed, at the subpopulation level (including disparities in every measure). 2. At national, state, and local levels, depending on where policy, programmatic, system, or clinical action is needed. National data include those collected in such systems as the BRFSS, vital rec- ords, the American Community Survey, and the Small Area Health Insurance Estimates, while state and local data include those used in the Community Health Status Indicators, drawn from electronic health records, and from other sources used by America’s Health Rankings and the County Health Rankings. 3. At levels applicable to the public health system and to the clini- cal care system where possible. There are several data-related challenges related to “usable” criteri- on. There are considerable data limitations at state and local levels and there is a great need for investment in better data infrastructure (an issue that also relates to one of the six drivers of public health quality—metrics and information technology). For example, data needed for all the clini- cal parameters (e.g., controlled blood pressure, hemoglobin A1c) are un- available at the local level. Even assuming universal electronic health records (and attaining this was not within reach at the time this report was written), there are many individuals who will remain outside the clinical care system and thus for whom there are no data. For example, uninsured individuals receiving emergency department care may have their blood pressure captured, but without a data generation process simi- lar to that in the National Health and Nutrition Examination Survey, it may not be possible to find out blood pressure levels in a community. Virtually no county has adequate data from BRFSS, and even in the case of large counties, the county-level data are of little value in dealing with the many sub-county issues. “Understandable” means that no great level of sophistication is re- quired from decision makers, including public health and health care practitioners, policy makers, and the public to understand the criteria. Furthermore, do the measures have face validity? Is the selection process transparent to users and other audiences? The criterion “methodologically rigorous” refers to the measures having suitable methodological and quantitative characteristics, such as

OCR for page 27
CRITERIA FOR SELECTING MEASURES 39 sensitivity, specificity, reliability, validity, and consistency over time and being managed by an established, regularly updated process. The com- mittee believes that a broader criterion is needed because common methodologic criteria such as validity and reliability do not cover issues such as representativeness and consistency. For example, blood pressure control may be a good measure for a clinical care system (i.e., valid and reliable), but a poor measure if the goal is control in the population (which deals with those with poor care, poor adherence, with population- level issues such as physical activity and sodium restriction to reduce population blood pressure levels). Finally, “accepted and harmonized” refers to measures such as those endorsed by the NQF (primarily focused on health care), or that are in standard use (e.g., used by America’s Health Rankings and the County Health Rankings). Difficulties arise, and harmonization is needed, in cases where there are many commonly used measures for the same phe- nomenon, such as binge drinking and defining an appropriate “norm” in consumption level, or conditions that are not well (or easily) measured, such as major depression in adolescents. ENDORSING QUALITY MEASURES FOR THE FIELD In its review of available measures, the committee was unable to find many measures of quality that reside outside the clinical sector. At its information-gathering meeting, the committee learned from participants that the universe of quality measures seems to include very few such measures compared to metrics that aggregate individual-level data related to specific disease states or clinical interventions (Jarris and Stange, 2012). The report Priorities for the 2011 National Quality Strategy (Nation- al Priorities Partnership, 2011) contains a table that details priorities, goals, and sample measures organized according to the HHS Three-Part Aim. The section of the table with the heading Population Health is sub- divided into three columns: clinical preventive services, healthy lifestyle behaviors, and community health index. The last item refers to truly population health measures (consistent with the Jacobson and Teutsch [2012] definition of “total population health”), and the two sources cited are the County Health Rankings and the AHRQ Prevention Quality Indi- cators. The former is a portfolio of measures that includes several population-based measures, including health behaviors (e.g., motor vehi- cle crash death rate and adult obesity), social and economic factors (e.g., high school graduation rates and percentage of children in poverty), and

OCR for page 27
40 TOWARD QUALITY MEASURES FOR POPULATION HEALTH of the physical environment (e.g., drinking water safety and fast food restaurants). The AHRQ Prevention Quality Indicators is a set of indi- vidual and composite measures derived from entirely clinical data and designed to assess the effectiveness of a local community’s ambulatory health care delivery system.7 This situation concerning population-based measures suggests that key national efforts to describe health priorities and measures have been predominantly clinical in orientation, as has much of the national effort on quality and prevention, and more at- tention is warranted for measures related to population-based preventive interventions. The first step in finding ways to measure quality that are relevant to many different system actors, beginning with public health agencies and health care organizations, could be to find a shared quality language that both health care and public health partners understand (and that can ulti- mately be understood by stakeholders outside the health sector). For ex- ample, the committee began the search for quality measures for the LHIs by reviewing the measures relevant to the LHIs that have been endorsed by the NQF. Where appropriate NQF-endorsed measures do exist, not only are they useful for improving the quality of care, but they can also further progress on the LHIs. This is a linkage that is sometimes not rec- ognized, i.e., that a pediatrician working in a HEDIS-compliant practice setting is not merely working to improve care, but may also be contrib- uting to improving health outcomes at a level far beyond the individual patient. The advantage offered by having an endorsing entity with an ac- cepted endorsing process is that organizations that want to use measures of quality in their work can simply look to the endorsed sets and select appropriate measures. An endorsement process is one of the first steps in improving measurement and is a prerequisite to the use of standard- ized measures and measure sets. In order to ensure progress in the for- mal adoption of population-based measures of quality, an entity charged with endorsing population health measures will need the following characteristics:  Be nongovernmental, to ensure independence;  Have the appropriate high-level leadership, organization, and expertise to enable review and endorsement measures of quality intended for population health improvement (e.g., measures of 7 See http://www.qualityindicators.ahrq.gov/Downloads/Modules/PQI/V44/Composite_ User_ Technical_Specification_PQI%20V4.4.pdf (accessed June 26, 2013).

OCR for page 27
CRITERIA FOR SELECTING MEASURES 41 the social and environmental determinants of health), not just measures of clinical care;  Have processes designed and resourced to evaluate and endorse measures of quality for population health improvement (a pro- cess separate and distinct from any existing process for endors- ing measures of clinical care); and  Include expert panel membership and staff support to identify and assess measures of quality for the multisectoral health sys- tem, with consideration of data sources, methodology, and other issues that span sectors and disciplines. Measuring the quality of population-based nonclinical interventions, such as policies, presents more challenges than measuring the quality of individual-based clinical interventions. Clinical interventions can demon- strate improved outcomes in the short term, while population-level action (e.g., clean air laws or universal preschool) may take a generation to bear fruit. For example, evaluating new population health measures will re- quire expertise in interacting with city councils, with planners and land use experts, with educators, and with community organizations. The re- port For the Public’s Health: The Role of Measurement in Action and Accountability (IOM, 2011a) asserted that improving population health will require entirely new kinds of measures and data. For example, the authoring committee wrote that there currently is no coordinated, standard set of true measures of a community’s health—not aggregated information about the health of individuals residing in a community, but rather measures of green space, availability of healthy foods, land use and zoning practices that are supportive of health, safety, social capital, and social cohe- sion, among many other determinants of health. (IOM, 2011a, p. 5) There currently is no organization that endorses measures of quality to be used for population health (i.e., measures for the multisectoral health system). However, NQF endorses measures of quality for the health care delivery system, and it or a similar entity, appropriately con- stituted, could perform the same role for the universe of measures that go beyond the health care delivery system. Given the conclusions of IOM committees and other groups that health care is only responsible for a modest proportion of the factors that influence population health, the committee calls for changes in the ap- proach to measurement of quality.

OCR for page 27
42 TOWARD QUALITY MEASURES FOR POPULATION HEALTH RECOMMENDATION 2-3: The committee recommends that the Department of Health and Human Services ensure the implementation of a systematic approach to develop and manage a portfolio of measures of quality for the multisectoral health system. HHS also should establish or designate a nongovernmental and appropriately equipped entity to endorse measures of quality. An entity endorsing measures of quality for the multisectoral health system would need to be guided by a strong research infrastructure, ele- ments of which were described in a recommendation in an earlier report (IOM, 2012) that called for a research agenda and funding to support the public health research and evaluation infrastructure. Also, an endorsing entity would not have to be, and ideally will not be, an organization that develops measures; although the two skillsets overlap somewhat, the roles and purposes are very different. With regard to the role of HHS in systematizing the approach to measures of quality, the committee has learned about two evolving HHS activities in the area of quality meas- urement. These are the Centers for Medicare & Medicaid Services (CMS) Quality Measures Task Force and the HHS Interagency Meas- urement Policy Council. The CMS Task Force is charged with “develop- ing recommendations on CMS measure implementation with the goal of aligning and prioritizing measures across programs and avoidance of du- plication or conflict among developing and implemented measures” and one of its goals is to coordinate “measure implementation, development and measurement policies” with other agencies in HHS (Goodrich, 2012). The Measurement Policy Council, which was established in 2012 as a subgroup of the HHS National Quality Strategy effort, is focused on policies for measure development, implementation, and alignment across HHS. The council includes AHRQ, CMS, the Office of the National Co- ordinator for Health Information Technology, the Substance Abuse and Mental Health Services Administration, the HHS Office of the Assistant Secretary for Planning and Evaluation, the Health Resources and Ser- vices Administration, CDC, the Office of Minority Health, the Food and Drug Administration, and others. Its initial focus is on “alignment and prioritization of measures in six major areas: hypertension, smoking ces- sation, depression, health care acquired conditions, patient experience, and care coordination” (Goodrich, 2012). It appears that the HHS-wide council, like the CMS Task Force, is largely oriented toward clinical care. Moreover, if the objective is to improve the health of the population by creating healthy conditions, coordination and measures are needed

OCR for page 27
CRITERIA FOR SELECTING MEASURES 43 that involve sectors of government beyond HHS, for example, the array of executive branch leaders (including the Departments of Education, Housing, and Transportation) participating in the National Prevention, Health Promotion, and Public Health Council. Data and information are needed to identify and develop measures of quality, and to support the development and management of the portfolio of measures described in Recommendation 2-3. Strengthening public health agency capacity in this area will also spur progress in the priority area for quality improvement in public health “Metrics and Information Technology.” To this end, the committee endorses both Recommenda- tions 1 and 2 from the 2011 IOM report For the Public’s Health: The Role of Measurement in Action and Accountability, and Recommenda- tion 6 from the 2012 IOM report For the Public’s Health: Investing in a Healthier Future. The former called for strengthening the population health information infrastructure, and for integrating, aligning, and standardizing health data and health outcome measurement at all geo- graphic levels.8 The latter called for a research infrastructure to establish the value of public health and prevention strategies, mechanisms for their effective implementation, health and economic outcomes derived, and the comparative effectiveness and impact of those strategies.9 8 Recommendation 1: The committee recommends that: (1) The Secretary of Health and Human Services transform the mission of the National Center for Health Statistics to provide leadership to a renewed population health information system through enhanced coordination, new capacities, and better integration of the determinants of health. (2) The National Prevention, Health Promotion, and Public Health Council include in its annual report to Congress on its national prevention and health-promotion strategy an update on the progress of the National Center for Health Statistics transformation. Recommendation 2: The committee recommends that the Department of Health and Hu- man Services support and implement the following to integrate, align, and standardize health data and health-outcome measurement at all geographic levels: a. A core, standardized set of indicators that can be used to assess the health of communities. b. A core, standardized set of health-outcome indicators for national, state, and lo- cal use. c. A summary measure of population health that can be used to estimate and track health-adjusted life expectancy for the United States. 9 Recommendation 6: The committee recommends that Congress direct the Department of Health and Human Services to develop a robust research infrastructure for establishing the effectiveness and value of public health and prevention strategies, mechanisms for effective implementation of these strategies, the health and economic outcomes derived from this investment, and the comparative effectiveness and impact of this investment. The infrastructure should include  A dedicated stream of funding for research and evaluation.  A national research agenda.

OCR for page 27
44 TOWARD QUALITY MEASURES FOR POPULATION HEALTH RECOMMENDATION 2-4: The Department of Health and Human Services should develop, implement, and support data collection, analysis, and dissemination mechanisms and infrastructure for the portfolio of quality measures so they are usable for health assessment and improvement at the national, state, and local levels. The committee hopes that the implementation of all IOM recom- mendations in this area will contribute to future data systems that address current limitations. The ability to develop good quality measures requires ensuring that timely data are available at national, state and, in particular, local levels, and that these data can be stratified for vulnerable sub- populations to assess changes in health disparities during improvement efforts.  Development of data systems and measures to capture research-quality infor- mation on key elements of public health delivery, including program imple- mentation costs.  Development and validation of methods for comparing the benefits and costs of alternative strategies to improve population health.