Initiatives are under way in all regions of the United States to improve health care quality, improve the health of the American population, and reduce health care costs. These initiatives take on increased urgency in the face of shortfalls with respect to what is possible in health and health care. Despite spending almost one-fifth of the economy’s output on health care, the quality and safety of care remains uneven (Hartman et al., 2013; IOM, 2012). Patient harm remains too common, care is frequently uncoordinated and fragmented, care quality varies significantly across the country, and overall health outcomes are not commensurate with the extraordinary level of investment (Bastian et al., 2010; Classen et al., 2011; IOM, 2012; Landrigran et al., 2010; Levinson, 2010, 2012; McGlynn et al., 2003).
This profound disconnect between potential performance and current reality exists despite the best efforts of many individuals and organizations to close the gap. As a result of concentrated efforts, some areas of the health system have been able to perform impressively and lead the world in science, innovation, and outcomes. Yet, results remain variable, and the health of the public varies from state to state, city to city, and even neighborhood to neighborhood (Fisher et al., 2003; McCarthy et al., 2009; RWJF and UWPHI, 2013; Schoenbaum et al., 2011; United Health Foundation et al.,
__________________
1 The planning committee’s role was limited to planning the workshop, and the workshop summary has been prepared by the workshop rapporteurs as a factual summary of what occurred at the workshop. Statements, recommendations, and opinions expressed are those of individual presenters and participants and are not necessarily endorsed or verified by the Institute of Medicine, and they should not be construed as reflecting group consensus.
2012). The challenges stem largely from the structure of the health system, which adds unnecessary burdens; organizes its activities into silos that do not communicate or coordinate with one another; and does not center itself on the needs of patients, consumers, and the broader public. Overcoming these obstacles requires restructuring the current system into one that continuously learns, improves, and focuses its efforts on the health and well-being of patients and the public (IOM, 2012).
THE ROLE OF MEASUREMENT IN A LEARNING HEALTH SYSTEM
While there are multiple obstacles to improving the nation’s health care system, one essential element for sustained progress is the capacity to reliably and consistently measure progress across all aspects of health and the health care system. Accurate, reliable, and valid measurements are a prerequisite for achieving and assessing progress in areas such as improving the quality of health care delivered to patients, reporting on the status of the health care system, and developing payment policies and financial incentives that reward improvement (IOM, 2006). Without a strong measurement capability, the nation cannot learn what initiatives and programs work best, resources cannot be guided toward the most promising strategies, and there is little ability to promote accountability in results.
One of the major questions concerning measurement is its scope. Current measurement initiatives focus on health care quality as it affects individuals, often on narrow or technical aspects of care, which encourages improvement only on those areas being measured. Yet the goals of the health system are broader, including health outcomes at the individual and population level, the quality of care that is delivered, cost and resource use by the system, and engagement of patients and the public (Berwick et al., 2008). These areas are interconnected, and changes to any particular area would likely have effects on the others. Furthermore, there are multiple factors that influence a person’s health, many of which lie outside the traditional health system (IOM, 2011b; Kindig and Stoddart, 2003; McGinnis and Foege, 1993; McGinnis et al., 2002).
Developing a more robust measurement enterprise will require overcoming several key challenges. Given the number of organizations involved in measurement and the large number of metrics currently in play, a key challenge is harmonization among the multiple metric development efforts that already are under way (AHRQ, 2013; Hussey et al., 2009; IOM, 2006; NQF, 2013; Wold, 2008). The current proliferation of measure sets and reporting requirements in health and health care can place a serious burden on individuals providing health services. These measurement requirements
can require substantial effort, time, and resources while potentially diverting attention from addressing higher health priorities.
Similarly, the logistical challenges for routine measurement are significant. The data needed to populate measures can be lacking, especially when paper health records are used. It thus can be difficult to track metrics in real time and to provide routine feedback to clinicians on their care processes and outcomes. These data challenges intensify when moving beyond clinical care to assess the efforts of public health agencies, community-based organizations, and others in improving the health of all Americans (IOM, 2011b).
Yet, new opportunities exist. The increased use of electronic health records and other digital tools has enhanced the ability to collect data routinely (IOM, 2011a). Beyond improving data collection, these tools also provide a means for measurement results to be fed back into clinical practice, patient self-management, and other care uses in near real time, allowing for regular, fine-tuned adjustments. Additionally, changes in payment and reporting policies have emphasized the importance of measurement and have increased interest in its advancement (Schneider et al., 2011). Thus, the country is poised for transformative change. By identifying current capacity for measurement and developing a shared strategy for future development, further progress will be made toward achieving a continuously learning health system.
THE ROUNDTABLE AND THE LEARNING HEALTH SYSTEM SERIES
The Roundtable on Value & Science-Driven Health Care has, since its founding at the Institute of Medicine (IOM) in 2006, brought together leaders from throughout the health system to accelerate the development of a continuously learning health system. A learning health system is one in which science, informatics, incentives, and culture are aligned to create a continuous learning loop, with evidence and best practices embedded in health and health care services and new knowledge routinely captured as a byproduct from each interaction with the system. Multiple steps have been taken to make progress toward this ambitious goal, including convening meetings of key health leaders, holding public workshops, stewarding collaborative projects that advance a learning system, and authoring reports and related publications.
Over the past 7 years, 13 volumes have been produced in the Learning Health System series of publications, including this publication. These publications have spanned a number of elements necessary for system transformation, including clinical research, the digital infrastructure, engaging patients and the public, focusing on value and financial incentives,
and applying lessons from other industries to health and health care. The publications have explored stakeholder perspectives on each issue, explored priorities for advancement, and discussed areas in need of collaborative action.
Another vehicle for this work is a series of Innovation Collaboratives that engage key health leaders in collaborative activities that advance the science and value in the health system. The Innovation Collaboratives currently focus on six overlapping and complementary areas: clinical effectiveness research, digital infrastructure, best practices, evidence communication, value, and systems approaches to improving health. These collaboratives foster information sharing and cooperation across the health and health care system, explore emerging issues facing particular sectors of the health system, and harness the talent and expertise of the participants in practical efforts to advance the field.
Building on previous work to advance the learning health system concept, the IOM held a 2-day workshop to explore in depth the core measurement needs for population health, health care quality, and health care costs. This workshop drew participation from across the measurement landscape, including perspectives from health care delivery organizations, clinicians, patients and consumers, public health experts, researchers, payers, health economists, measure developers, standard-setting organizations, regulators, clinical research, health information technology, and community organizations.
The goal of this workshop was to understand how to improve the nation’s measurement capacity to track progress in a core measure set for better care quality, lower cost, improved patient and public engagement, and better health outcomes. Furthermore, the workshop sought to consider the implementation of core measure sets, including the measurement burden, a measure’s actionability, and its accuracy when used in regular practice. The workshop statement of task, shown in Box 1-1, guided the objectives for the workshop:
- Discuss the vision for the nature, use, and impact of core health metrics.
- Identify the important principles, targets, infrastructure, processes, strategies, and policies.
- Describe lessons from efforts at national, state, community, and organization levels.
- Specify core needs and requirements, and propose priority metric categories that will most reliably measure care outcomes, care costs, and health improvement.
- Consider specific examples of metric options within categories.
- Describe the implementation strategies—national, state, community, organizational.
To address these objectives, the workshop was divided into a series of sessions that explored different aspects of measurement. The workshop began with an exploration of the vision for the use of core metrics, the current capabilities for the use of core metrics, and lessons learned from current measurement initiatives. The workshop attendees then divided into smaller groups to consider categories of measures, along with example measures within each category, that could help to achieve the vision of a core metric set. As measurement requires many support structures, the second day of the workshop explored the infrastructure, resources, and policies that are needed to support the use of core metrics. Throughout the discussions the workshop considered the differing measurement needs for different levels of the health system, from the local level to the national level, as well as the needs for the diverse set of stakeholders involved in measurement.
An expert planning committee will guide the development of a two-day workshop to examine the elements necessary for progress toward, and achievement of, a truly learning health system that achieves the three-part aim: better care for individuals, better health for a population, and lower costs. Fundamental to a learning health system is measurement of health outcomes and cost, delivered in a fashion that allows accurate, actionable, real-time, and continuous use of that information. The committee will steer development of the agenda for the workshop, including selection of speakers and discussants. The workshop will feature invited presentations and discussions that will provide participants an opportunity to engage representatives from federal, state, and local governments and the nonprofit and private sectors. The discussions will highlight lessons learned from existing data and measurement systems and the needs and opportunities for future measurement capacity across all sectors. The focus of the sessions will be on practical approaches to capacity building to ensure not only that options are considered for the critical analysis of progress toward the three-part aim but also that achievement of a learning health system is extended through seamless availability of health care data.
One of the challenges revealed by the workshop discussions was providing consistent terminology. For example, in some cases the term “better health” referred to population health, while in others it referred to clinical or disease outcomes for individuals. This diversity of meanings reflects the numerous perspectives in play when measuring the performance of the health system. For clarity, when different definitions are used for the same term, this publication includes the presenter’s intended meaning.
This publication summarizes the discussions that occurred throughout the workshop, highlighting the key lessons presented, practical strategies, and the needs and opportunities for improving future measurement capacity. Chapter 2 explores a vision for core metric sets, Chapter 3 considers current measurement capabilities, and Chapter 4 highlights example core measure sets that are currently in use. Chapter 5 covers the discussions from the breakout groups that surveyed potential metric categories and example metrics for population health, health care quality, and health care costs. Chapters 6 and 7 focus on implementation issues, including the implementation challenges faced by example initiatives and the data infrastructure needs for measurement. Chapter 8 concludes the report with a summary of common themes that emerged from the workshop discussions.
The workshop discussions are intended to be a first step in understanding the many factors affecting the development of a core measure set. The meeting revealed the many issues that must be considered in order to comprehensively assess the performance of the health system in improving overall health, care quality, cost and resource use, and patient and public engagement. Further work will be needed to resolve the issues raised and synthesize these discussions into a formal set of recommendations.
AHRQ (Agency for Healthcare Research and Quality). 2013. National Quality Measures Clearinghouse. http://www.qualitymeasures.ahrq.gov/index.aspx (assessed April 9, 2013).
Bastian, H., P. Glasziou, and I. Chalmers. 2010. Seventy-five trials and eleven systematic reviews a day: How will we ever keep up? PLoS Medicine 7(9):e1000326.
Berwick, D. M., T. W. Nolan, and J. Whittington. 2008. The triple aim: Care, health, and cost. Health Affairs (Millwood) 27(3):759–769.
Classen, D. C., R. Resar, F. Griffin, F. Federico, T. Frankel, N. Kimmel, J. C. Whittington, A. Frankel, A. Seger, and B. C. James. 2011. “Global trigger tool” shows that adverse events in hospitals may be ten times greater than previously measured. Health Affairs 30(4):581–589.
Fisher, E. S., D. E. Wennberg, T. A. Stukel, D. J. Gottlieb, F. L. Lucas, and É. t. L. Pinder. 2003. The implications of regional variations in Medicare spending. Part 2: Health outcomes and satisfaction with care. Annals of Internal Medicine 138(4):288–298.
Hartman, M., A. B. Martin, J. Benson, A. Catlin, and the National Health Expenditure Accounts Team. 2013. National health spending in 2011: Overall growth remains low, but some payers and services show signs of acceleration. Health Affairs (Millwood) 32(1):87–99.
Hussey, P. S., H. de Vries, J. Romley, M. C. Wang, S. S. Chen, P. G. Shekelle, and E. A. McGlynn. 2009. A systematic review of health care efficiency measures. Health Services Research 44(3):784–805.
IOM (Institute of Medicine). 2006. Performance Measurement: Accelerating Improvement, Pathways to Quality Health Care. Washington, DC: The National Academies Press.
———. 2011a. Digital Infrastructure for the Learning Health System: The Foundation for Continuous Improvement in Health and Health Care: Workshop Series Summary. Washington, DC: The National Academies Press.
———. 2011b. For the Public’s Health: The Role of Measurement in Action and Accountability. Washington, DC: The National Academies Press.
———. 2012. Best Care at Lower Cost: The Path to Continuously Learning Health Care in America. Washington, DC: The National Academies Press.
Kindig, D., and G. Stoddart. 2003. What is population health? American Journal of Public Health 93(3):380–383.
Landrigan, C. P., G. J. Parry, C. B. Bones, A. D. Hackbarth, D. A. Goldmann, and P. J. Sharek. 2010. Temporal trends in rates of patient harm resulting from medical care. New England Journal of Medicine 363(22):2124–2134.
Levinson, D. R. 2010. Adverse Events in Hospitals: National Incidence Among Medicare Beneficiaries. Washington, DC: U.S. Department of Health and Human Services, Office of Inspector General.
———. 2012. Hospital incident reporting systems do not capture most patient harm. Washington, DC: U.S. Department of Health and Human Services, Office of Inspector General.
McCarthy, D., S. How, C. Schoen, J. Cantor, and D. Belloff. 2009. Aiming Higher: Results from a State Scorecard on Health System Performance. New York: Commonwealth Fund Commission on a High Performance Health System.
McGinnis, J. M., and W. H. Foege. 1993. Actual causes of death in the United States. JAMA 270(18):2207–2212.
McGinnis, J. M., P. Williams-Russo, and J. R. Knickman. 2002. The case for more active policy attention to health promotion. Health Affairs (Millwood) 21(2):78–93.
McGlynn, E. A., S. M. Asch, J. Adams, J. Keesey, J. Hicks, A. DeCristofaro, and E. A. Kerr. 2003. The quality of health care delivered to adults in the United States. New England Journal of Medicine 348(26):2635–2645.
NQF (National Quality Forum). 2013. Report from the National Quality Forum: 2012 NQF Measure Gap Analysis. http://www.qualityforum.org/Publications/2013/03/2012_NQF_Measure_Gap_Analysis.aspx (accessed April 9, 2013).
RWJF (Robert Wood Johnson Foundation) and UWPHI (University of Wisconsin Population Health Institute). 2013. County Health Rankings & Roadmaps. http://www.countyhealthrankings.org/app/home (accessed April 9, 2013).
Schneider, EC, PS Hussey, and C. Schnyer. 2011. Payment Reform: Analysis of Models and Performance Measurement Implications. Santa Monica, CA: RAND Corporation.
Schoenbaum, S. C., C. Schoen, J. L. Nicholson, and J. C. Cantor. 2011. Mortality amenable to health care in the United States: The roles of demographics and health systems performance. Journal of Public Health Policy 32(4):407–429.
United Health Foundation, American Public Health Association, and Partnership for Prevention. 2012. America’s Health Rankings: A Call to Action for Individuals and their Communities. Minnetonka, MN: United Health Foundation.
Wold, C. 2008. Health Indicators: A Review of Reports Currently in Use. Washington, DC: State of the USA.
