National Academies Press: OpenBook

Performance Measurement: Accelerating Improvement (2006)

Chapter: 3 Achieving a National System for Performance Measurement and Reporting

« Previous: 2 Current and Future State of Performance Measurement and Reporting
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

3
Achieving a National System for Performance Measurement and Reporting

CHAPTER SUMMARY

This chapter presents the committee’s recommendation for the establishment of an independent board to oversee and coordinate the functions of a national system for performance measurement and reporting. Guidelines for the design and operation of the board to ensure its authority and sustainability are discussed. The chapter closes with a discussion of potential concerns about this newly proposed entity and the committee’s responses to those concerns.

ALTERNATIVES TO ACHIEVING A NATIONAL SYSTEM FOR PERFORMANCE MEASURMENT AND REPORTING

A strong evidence base does not yet exist to support the value of a well-coordinated national system for performance measurement and reporting relative to the status quo described in Chapter 2. However, some promising examples of how performance measurement and public reporting have been linked to improved quality helped shape the committee’s decision making. For example, the Veterans Health Administration (VHA) has implemented several system changes targeted at improving the quality of care, including measuring and tracking performance on a comprehensive set of indicators. In a cross-sectional comparison of 12 VHA health care systems versus a representative national sample, VHA patients were found to receive higher-quality care than their counterparts in the areas of overall quality, chronic disease care, and preventive care. The greatest differences were seen in areas in which the VHA had well-established performance measures in place and was aggressively monitoring performance and providing feedback (Asch et al., 2004). With regard to public reporting, a recent study comparing hospitals that reported data publicly and those that shared data only privately

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

found that the former hospitals significantly improved their performance in the clinical area studied relative to those that did not use public reports. The results of this study appear to suggest that public reports stimulate quality improvement interventions (Hibbard et al., 2005). In the absence of a strong evidence base, the committee considered three factors in developing its recommendations: (1) the scope of efforts currently under way in public reporting, payment incentives, and quality improvement that are dependent on performance measures; (2) the consequences of inaction; and (3) the merits of alternative approaches to the development of an effective national system for performance measurement and reporting.

The scope of current efforts and expectations for performance measurement are high. The Centers for Medicare and Medicaid Services (CMS), the private sector, and Congress are all committing substantial resources to performance measurement, public reporting, and quality improvement initiatives, and are now embarking on pay for performance initiatives. All of these efforts are being undertaken with the strong expectation of improving the quality of care. Indeed, it was this expectation that led Congress to request that the Institute of Medicine (IOM) provide guidance to CMS to support these efforts.

After reviewing current efforts (see Chapter 2), the committee concluded that extensive resources exist on which to build a national system, including those of national organizations such as the National Committee for Quality Assurance (NCQA), the National Quality Forum (NQF), and the Joint Commission on Accreditation of Healthcare Organizations (JCAHO), as well as numerous state and regional entities. Notwithstanding these considerable accomplishments, however, the nation does not yet have a coherent, integrated system for establishing, measuring, and tracking the performance of the health care system. In fact, there is growing concern in some quarters that the current fragmented approach could create a confusing, duplicative, yet still incomplete set of activities that would absorb too many resources. As the overall enterprise is still young, it is worth asking whether the current efforts can and will evolve into the system that is needed—one in which performance is tied to national health care goals; is viewed as credible, objective, and grounded in evidence; is comprehensive in covering all of the six aims identified in the Quality Chasm report; is coordinated and forward looking; and is transparent and accessible to all stakeholders.

In its deliberations, the committee also discussed the potential consequences of failing to move forward and to capitalize on existing resources in a more systematic way. One such consequence is that the pace of change will remain slow, thus not reflecting the sense of urgency the committee views as essential to reach the point at which consistent delivery of high-quality care will become accelerated and more widespread. Another con-

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

cern is that without a mechanism in place for continuously evaluating the impact of performance measurement initiatives and guarding against unintended consequences, the much-needed evidence base for identifying effective interventions will be less likely to evolve.

ASSESSMENT OF ALTERNATIVES

In the above context, the committee considered four alternatives for achieving a high-functioning national system for performance measurement and reporting, as illustrated in Figure 2-1 in Chapter 2: (1) establishing a large federal entity, (2) establishing an office within CMS or the Agency for Healthcare Research and Quality (AHRQ), (3) delegating functions to existing stakeholder groups, and (4) establishing a new independent board.

Alternative 1: Large Federal Entity

A new federal entity could be established to assume responsibility for the entire spectrum of activities shown in Figure 2-1 in Chapter 2. The advantage of this option is that all of the resources needed to create a national system for performance measurement and reporting would be housed under one roof and supported by a single stream of funding. However, the committee believes this option is not preferable for several reasons. First, creating such a federal entity would duplicate the work already being performed by a host of reputable stakeholder groups. Incurring the high cost of assuming the tasks currently conducted by those existing groups would be imprudent, particularly in the current fiscal environment. Moreover, the transition to a large federal bureaucracy could disrupt current activities in the private sector, thus having the unintended consequence of setting back progress made to date that has been shown to be of value.

Alternative 2: Office Within the Centers for Medicare and Medicaid Services or the Agency for Healthcare Research and Quality

Current public–private, largely voluntary efforts could be sustained with the addition of a special office within CMS or AHRQ that would be charged with encouraging existing players to align those efforts more directly with a specific set of national health goals. The committee believes this option would be an improvement over the status quo, but lacks the capacity to achieve the vision of a full-fledged national system for performance measurement and reporting. The committee believes such an office would be unlikely to have the authority to establish national goals and aims for improvement, and would lack the clout and the resources to convince stakeholder groups to move beyond the sphere of their own special interests.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Additionally, this type of structure could be susceptible to undue political influences that would threaten its independence.

Alternative 3: Other Stakeholder Groups

The responsibility for a national system could be assigned to one of the existing major stakeholder groups, such as NQF, NCQA, or JCAHO. As described in Chapter 2, current private-sector efforts have made substantial progress in shaping and advancing the field of performance measurement in health care. For example, NQF has a reputable track record in endorsing standardized performance measures, while NCQA and JCAHO have rich histories in the development of standardized measures and the public reporting of comparative quality data.1 In addition to these competencies, however, the committee identified additional functions of a national system that are unlikely to be realized through private-sector efforts alone, as they are framed from the perspective of a public good. These include (1) specifying a purpose and national aims for the health care system, (2) setting national goals, (3) establishing and funding a national research agenda, and (4) evaluating the effectiveness of the performance measurement system and reporting in its entirety. The committee believes that to achieve a broad-based performance measurement and reporting system that is capable of fulfilling all these requirements, public-sector leadership and oversight are essential.

Alternative 4: Independent Board

The committee concludes that a well-functioning national system for performance measurement and reporting is most likely to arise through creation of an entity built on the accomplishments and ongoing efforts of existing organizations. While not impossible, the committee doubts that such a system would evolve on its own from the vast array of efforts by public, for-profit, and not-for-profit organizations currently under way. This assertion is based on history and a number of realities that characterize the present situation:

  • As noted above national goals are unlikely to be set and translated into measures, since existing entities have neither the authority nor the overarching leadership required to formulate such goals.

1  

NCQA publicly reports Health Plan Employer Data and Information Set measures for participating health plans; JCAHO’s ORYX measures are used by CMS for its Hospital Compare Website.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
  • Gaps in performance measurement, such as the capacity to measure equity and access, are unlikely to be filled because of the lack of clear ownership of these aspects of the nation’s quality improvement agenda.

  • Wasteful duplication and inconsistencies among measures will continue, since no single stakeholder group has the standing to require others to use specific, standardized definitions and measurements.

  • Measures may not be viewed as authoritative, credible, or objective since the measures developed by most stakeholders are more apt to reflect the interests of their constituencies than those of others.

  • Public goods, such as investments in better risk adjustment methodologies and data aggregation methods, are unlikely to be addressed adequately in a competitive market among current developers of measures.

  • Making all information fully transparent and available to the public is unlikely, since much of the technology and data on performance measurement is currently held as proprietary.

Table 3-1 provides a synopsis of the four alternatives discussed above. Although a large federal entity could assume all the necessary functions for a national performance measurement and reporting system, the committee

TABLE 3-1 Comparison of Alternatives for Achieving a National System for Performance Measurement and Reporting

Key Functions of a National System

Alternative 1: Large Federal Entity

Alternative 2: Office Within CMS or AHRQ

Alternative 3: Other Stakeholder Groups

Alternative 4: Independent Board

Specify purpose and aims

 

Prioritize national goals

Promulgate standardized measures

Ensure data collection, validation, and aggregation

Establish public reporting methods responsive to the needs of all stakeholders

Identify a research agenda

 

Evaluate impact of overall system

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

endorses the model of an independent board with contracting authority to capitalize on existing resources as the most appropriate and judicious approach. As illustrated, public and private entities are currently performing several key tasks, but have not been able to meet national needs and expectations because of the difficulty of relying on consensual measures, the fluctuating nature of private support for these efforts, and the lack of clear goals.

Therefore, the committee recommends a strategy based on the establishment of a new independent board, the National Quality Coordination Board (NQCB), that would be recognized by all public and private stakeholders as the lead agency responsible for ensuring the creation of a national system for performance measurement and reporting. In addition to carrying out general management and coordinating functions, the board would provide leadership and policy guidance that would support existing efforts, and seek to align those efforts with national health goals through contractual agreements, educational programs, and consensus-building initiatives.

Recommendation 1: Congress should establish a National Quality Coordination Board (NQCB) with seven key functions:

  • Specify the purpose and aims for American health care.

  • Establish short- and long-term national goals for improving the health care system.

  • Designate, or if necessary develop, standardized performance measures for evaluating the performance of current providers, and monitor the nation’s progress toward these goals.

  • Ensure the creation of data collection, validation, and aggregation processes.

  • Establish public reporting methods responsive to the needs of all stakeholders.

  • Identify and fund a research agenda for the development of new measures to address gaps in performance measurement.

  • Evaluate the impact of performance measurement on pay for performance, quality improvement, public reporting, and other policy levers.

The NQCB should be composed of health care leaders capable of understanding the diverse sectors within the health care system, such as consumers, purchasers, educators, clinicians from all disciplines (medicine, nursing, pharmacy), and agencies and research centers with expertise in performance measurement. Expert staff will be needed to monitor routine data collection, coordinate standards-setting efforts, and assess national

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

progress toward the implementation of measures that reflect the six quality aims for the health care system.

To operate effectively, the NQCB must have the authority to function independently while working in close collaboration with public- and private-sector health agencies and health care providers. The board is intended to supplement and strengthen—but not replace—ongoing data collection, standardization, and reporting activities in both the private and public sectors. Ideally, the board will enable these key players to perform their jobs more effectively by ensuring a unified and coordinated approach to performance measurement within a framework of clearly articulated national goals. Additionally, the committee views the NQCB as a federal entity that will complement and support current efforts to guide the development of health care performance measures among professional societies, trade associations, health plans, consumer groups, and other elements of the health care enterprise.

ESSENTIAL ATTRIBUTES OF THE NQCB

The need for a national system for performance measurement and reporting provides the rationale for creating the NQCB. While the development of a detailed operational plan for the board’s implementation was beyond the scope of the committee’s work, its functions are clear. This section provides further details on what the committee identified as the essential attributes of the NQCB.

Standardizing performance measurement and establishing a useful, bold, and transparent national system for setting goals and reporting on progress will require a difficult transition for many stakeholders in American health care. Although almost all parties—especially patients—will benefit in the end from better measurement and reporting, many of the current stakeholders can initially be expected to defend the status quo.

The sources of resistance will be many. For example, vendors and consultants now maintaining or developing their own measurement systems will regard a more uniform national approach as a potential threat to their current designs and market niche. Organizations and individuals that provide health care will be concerned that bold and transparent measurement may divert them from their current strategic agendas, reveal hidden defects in their care, pose uncomfortable threats to marketplace competition, add new costs for data collection and reporting, and, if not properly adjusted for case mix, be unfair and misleading. Governmental and nongovernmental agencies now charged with surveying and assessing care may be biased toward their own current formats, data definitions, and reporting schemes, and will be concerned about the changes to their current systems and processes required by standardization. If, as the committee recommends, a na-

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

tional system for performance measurement and reporting begins with bold aims for the improvement of care and efficiency, stakeholders lacking confidence that such improvements are achievable can be expected to resist.

History bears this out. In the past, the threshold effect—the pain of transition—has prevented progress at the scale recommended by the committee. Consensus-driven processes have made progress, but not enough to meet the social need. Despite numerous efforts over several decades to improve the nation’s health care performance assessment, no effective system yet exists.

An NQCB that can lead the development of a truly effective system of performance measurement and reporting will have to possess a high level of independence and authority if it is to accomplish its purpose. At the same time, the board will need to collaborate with major stakeholders, building upon their good will and social vision, if this initiative is to be more successful than earlier attempts. This means that the NQCB should not ignore or replicate the existing measurement capability represented by these stakeholders. Indeed, the committee recommends that, at the outset, the board seek to accomplish its aims to the extent possible by convening capable existing entities, encouraging them to achieve new levels of cooperation, creating appropriate contractual relationships, and assigning projects and deliverables. In attempting to make use of existing organizations and resources, the NQCB must insist on levels of cooperation, standardization, and transparency not currently characteristic of this array of actors. The committee is hopeful that such cooperation—which represents the most parsimonious way to meet the nation’s needs for measurement and reporting—can and will emerge.

Recommendation 2: The NQCB’s membership and procedures should be designed to ensure that the board has structural independence, protection from undue special interests, substantive expertise drawn from the public and private sectors (including not-for-profit entities), contract authority, standards-setting authority, financial strength, and external accountability.

The board proposed by the committee should have the capacity to perform the multiple functions outlined in Recommendation 1. These functions require that the board have sufficient independence to ensure its objectivity and the capacity to develop and rely upon evidence-based knowledge to guide its recommendations. At the same time, the board needs to be located within a constellation of governmental agencies so it can coordinate its work with appropriate organizations having the operational resources necessary to implement recommended measurement standards, data collection and reporting procedures, and research and evaluation efforts.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Given these competing needs, the committee has opted to recommend locating the NQCB as an independent entity within the U.S. Department of Health and Human Services (DHHS), reporting directly to the DHHS Secretary. This arrangement will allow the board to perform its functions while preserving its structural independence from other agency priorities that could impede its activities.

The committee believes that an entity without adequate authority and protection cannot succeed in this endeavor. The chance of succeeding through new relationships with and among existing players will depend on the board’s ability to withstand the intense short-term political pressures that without doubt will arise as current stakeholders perceive the threats enumerated above. Change will not be accomplished without discomfort, and the NQCB must be able to ride out that discomfort and adhere to the goals articulated in this report. Therefore, the committee proposes that the NQCB be armed with at least the following forms of authority and protection:

  • Structural independence. The NQCB should have the capacity to move the health care system beyond the status quo. The committee recommends that the board be housed within the DHHS and report directly to the Secretary.

  • Protection from undue influence. The membership of the NQCB should be appointed by the President, with terms that are staggered and long enough to protect the board against short-term political influence and major stakeholder interests.

  • Substantive expertise. As noted above, the committee’s intention is not to supplant or duplicate the often outstanding work of the many organizations currently involved in developing, evaluating, vetting, and implementing performance measures in health care. Rather, the goal is to accelerate progress through coordination and direct financial support for these current activities. Thus the membership of the NQCB should encompass the technical competence needed to assess and guide that work.

  • Contract authority. In the event that the major organizations currently engaged in measurement development, implementation, and reporting prove unwilling or unable to undertake the activities outlined by NQCB or to deliver under contract the required levels of standardization, analysis, and reporting, the board should have the backup authority and sufficient funding to broaden the array of contractors through which it can execute its key functions.

  • Standards-setting authority. The Secretary of DHHS should direct CMS (including Medicare, Medicaid, and the State Children’s Health Insurance Program), the Health Resources and Services Administration, and AHRQ to focus on the achievement of all applicable national goals established by the NQCB through public reporting, payment reform and other

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

incentives, health care improvement programs, health professions education, and organizational and systems capacity building. The Secretary should also direct CMS to require that providers submit to the NQCB (or its designee) performance data that can be used by Medicare for public reporting and quality improvement activities or as a basis for payment. In addition, Congress should activate an interagency task force to explore mechanisms for aligning other government health care programs with these efforts—including the Department of Defense (DoD) TRICARE program and DoD-operated clinical facilities, the Federal Employees Health Benefits Program, and the programs of the Veterans Health Administration and the Indian Health Service.

  • Financial strength. The NQCB should have sufficient, stable funding to contract for services with outside groups and organizations so it can perform its designated functions effectively. The board should be funded directly from the Medicare Trust Fund and have bypass authority to request an appropriation directly from Congress. This bypass authority would free the NQCB from the unpredictable budgetary cycles commonly associated with preparing discretionary budgets that are subject to review and modification on the basis of other departmental, executive, and legislative priorities. Congress should authorize and appropriate funds to support the work of the NQCB and to implement its recommendations in Medicare and other government programs by the end of fiscal year 2007. More specifically, Congress should authorize an annual allocation from the Medicare Trust Fund, initially in the range of $100–200 million (see the discussion in the next section). This level of investment is based on an analysis of resources that currently support related but more limited activities led by NQF, NCQA and JCAHO (as described below). This figure constitutes approximately 0.1 percent of the Medicare annual budget,2 a relatively small investment with great potential to enhance value and improve efficiency throughout the health care delivery system. The committee envisions substantial staff requirements to support the functions of the board delineated in Recommendation 1 and substantial costs related to contracts with existing entities to carry out tasks pursuant to the mission of the board. Although the federal government should provide up front the funding needed for the NQCB to become fully operational, particularly with regard to its public-good functions, public–private partnerships could be formed over time to support this ongoing work (see Chapter 5).

  • External accountability. The NQCB should be required to provide an annual report to Congress on its progress toward implementing an effective system for performance measurement and reporting. In addition, the

2  

$278 billion in 2003 (CMS, 2004).

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

board should undergo periodic independent assessments performed by an external organization such as the Medicare Payment Advisory Commission, the IOM, or the Government Accountability Office.

FUNDING FOR THE NQCB

To estimate the level of resources that may be required to support the work of the NQCB, the board’s functions were compared with those of other organizations focused on quality improvement, as previously summarized in Table 3-1. As discussed earlier in the chapter, the primary role of NQF, as now structured, is to endorse standardized performance measures—an important function for a national system. NQF has relatively small revenues ($4 million annually), more than half of which are derived from external sources (Guidestar, 2004c). As the scope of the committee’s proposed functions for the NQCB is much broader (see Figure 2-1 in Chapter 2), far greater resources would be needed for its operational and oversight functions.

A better model from which to draw inferences about the funding required for the NQCB is NCQA, as it currently performs three key functions of a national system for performance measurement and reporting: (1) development of performance measures; (2) data collection, validation, and aggregation; and (3) public reporting of performance patterns in various regions and across the country. NCQA’s revenues totaled $24 million in 2003 (Guidestar, 2004b). Given the administrative tasks and responsibilities of the NQCB beyond NCQA’s existing activities—specifying a purpose and aims for the health care system, setting national goals, establishing and funding a national research agenda, and monitoring the impacts of the overall system—a funding estimate of $100–200 million annually is reasonable.

As described in detail in Chapter 2, JCAHO has been extensively involved in performance measure development and reporting activities. JCAHO’s revenues for 2003 totaled $85 million (Guidestar, 2004a). Although a large portion of those revenues is associated with accreditation-related activities, the much broader functions of the NQCB argue for funding within the recommended range.

GUIDELINES FOR THE DESIGN AND OPERATION OF THE NQCB

Establishing the NQCB will be a complex and challenging task that will evolve over time. As noted earlier, preparing a detailed blueprint for the operation of such a system was beyond the scope of this study; however, the committee has developed an overall framework and guidance for the initial steps.

The careful design, operationalization, and management of the NQCB will be critical to its success. The Strategic Framework Board focused special attention on three areas that are applicable to the NQCB (Kizer, 2003).

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

First, the NQCB decision-making process should be evidence based and continually updated to reflect changes in knowledge. Second, the Board should reinforce local improvement efforts, not superimpose new structures on local communities. Third, the NQCB process should be responsive to the challenges and concerns of health care providers, health plans, consumers, and purchasers alike.

Grounded in Evidence

The decision-making processes of the NQCB will benefit greatly from a strong evidence base and access to specialized expertise (McGlynn, 2003). The board should be a learning system that is supported by and contributes to the generation of evidence relevant to (1) making key decisions, such as setting national goals and specifying performance measures; and (2) evaluating the selected measures and understanding their impact on various types of stakeholders.

A great deal of evidence, much of which is currently unavailable, will be needed to support the work of the NQCB. In setting national goals, the board will need epidemiological evidence (e.g., leading causes of death and disability), clinical evidence (e.g., efficacy or effectiveness of various interventions in curing or slowing the progression of a particular disease), and health services research evidence (e.g., feasibility of successful implementation and cost-effectiveness of various interventions) (McGlynn, 2003). In promulgating standardized performance measures, the NQCB will need evidence on the scientific soundness of the various measures under consideration (e.g., whether claims data can be used to assess whether a particular clinical process was performed on those patients who would likely benefit from it or was performed properly). The NQCB will also need evidence on what measures are most important to patients with various types of preferences and needs (see Chapter 5).

Evidence to support both the setting of goals and the promulgation of measures will have to come from a variety of sources. The Centers for Disease Control and Prevention plays a central role in generating epidemiological evidence. Clinical evidence is produced by both the public sector (e.g., National Institutes of Health) and the private sector (e.g., academic health centers, pharmaceutical companies). Specialty societies and others synthesize this evidence into practice guidelines. Health services research, funded by AHRQ, private foundations, and many health care organizations, is often conducted by researchers located in academic settings or research institutes or within health care delivery systems.

The NQCB should be designed to support the decision making of many different stakeholders and to generate evidence on the impacts of measurement and reporting. The board is intended to generate information that will

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

influence health and health care through three pathways: accountability, quality improvement, and population health. Ultimately, the efforts of the many stakeholders involved in these three areas should result in achievement of the six quality aims of the Quality Chasm report (IOM, 2001). Data and analyses required to assess impact will be necessary for determining whether the NQCB and the many other quality-related activities now under way are having their intended effect. Specifically, the NQCB should provide the information necessary to evaluate:

  • Changes in the environment of care (e.g., consumer and purchaser selection of providers, pay for performance incentives, use of performance data for public reporting and in accreditation and credentialing decisions).

  • Changes in the capacity of the delivery system to provide high-quality care (e.g., changes in care coordination mechanisms, use of multidisciplinary teams, implementation of systematic processes to increase adherence to practice guidelines).

In other words, the NQCB should generate information on areas of progress as well as factors that contribute to or impede the rate of change.

A carefully crafted research agenda will be needed to ensure the availability of the evidence necessary for the NQCB to function as a learning system that incorporates new advances. The development and maintenance of this research agenda will require collaboration among the epidemiological, clinical, consumer, purchaser, and health services research communities. Adequate and ongoing funding will require commitments on the part of public- and private-sector funding agencies.

Supportive of Local Improvement Efforts

Although many important aspects of the U.S. health care system are national in scope (e.g., Medicare payment policies, accreditation and certification programs), the delivery of health care services is for the most part a local enterprise. Where applicable, the design and operation of the NQCB should respond to local goals and improvement priorities. Whenever possible, the board should specify the use of standardized measures and reporting requirements that will yield useful information for the purposes of accountability, quality improvement, and population health at the community, regional, and state levels.

In setting national goals, the NQCB should provide opportunities for local input into the agenda-setting process by giving communities an opportunity to comment (McGlynn et al., 2003). Local communities might also be encouraged to undertake locally driven quality improvement initiatives in addition to pursuing national goals.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Recommendation 3: Local innovation in pursuit of national goals for improving health care quality should be encouraged. Performance measurement, improvement, and reporting activities—including those of public and private purchasers; accreditation and certification entities; and federal, state, and local government programs—should be substantially aligned with the national goals and standardized measures established by the NQCB, but local communities should also be encouraged to identify and pursue local priorities, in addition to helping to achieve national goals.

Improvement efforts will likely be more successful and the reporting burden on providers far less onerous if the performance measures and data requirements specified by the NQCB produce information that is useful at all levels of the care system—from the individual provider (whose efforts to improve require performance measures) to the national level (James, 2001). Overall, the committee believes the NQCB will be far more effective in achieving its purpose and aims if standardized performance measure sets are comprehensive enough to support the efforts of many stakeholders—both those external to health care organizations (i.e., purchasers, planners) and those engaged in health care delivery. The NQCB will also be more efficient and timely if the data used to calculate measures are, to the extent possible, generated in real time as a byproduct of the patient care process, rather than retrospectively.

Responsive to Stakeholder Concerns

The NQCB is a potentially powerful tool intended to support the efforts of all stakeholders to achieve a fundamental redesign of the health care delivery system. As with any powerful tool, it must be used wisely and cautiously, and balance the needs of various stakeholders.

The pace of change and burden of data collection should not over-whelm the provider community, yet it should be rapid enough to address the most important unmet needs of consumers and purchasers. The NQCB should develop a reasonable and prompt schedule for the implementation of various measurement and data submission requirements and for the achievement of its specified goals. Requirements should be phased in and communicated in advance to all stakeholders. The failure to develop a reasonable and prompt plan for implementation (e.g., 1-, 3-, and 5-year requirements) and to communicate this plan to providers, consumers, and purchasers could generate a backlash that would impede progress toward a nationally coherent measurement system.

The stewardship responsibilities of the NQCB should be well defined and carried out with the utmost integrity. To be successful, the board

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

must earn the trust and respect of all stakeholders, but especially the provider and patient communities. As noted above, the board’s decision making must be grounded in scientific evidence. Auditing mechanisms must be established to ensure data quality. Adequate data protections must be in place to ensure patient confidentiality. Public reports must provide fair comparisons.

Finally, in any complex system, the change process produces both intended and unintended consequences. An early warning system will be needed to identify unintended consequences of the NQCB and take mitigating action. The unintended consequences of the goal-setting and standardized measurement and reporting processes of the NQCB might include the following:

  • Adverse selection—In the absence of adequate risk-adjustment techniques, providers who care for some of those patients most in need may appear to be poor performers compared with their peers who treat healthier patients (Werner and Asch, 2005). Some providers may try to improve their performance scores by engaging in adverse selection. As a consequence, patients most likely to experience poor health outcomes, such as those most severely ill or with poor health behaviors, may experience difficulty in gaining access to the health system.

  • Data manipulation—Providers may engage in data recording and coding practices designed to inflate their performance ratings. For example, if performance measures are adjusted for a patient’s complicating and comorbid conditions, providers may inflate the list of secondary diagnoses to include conditions that are inactive or those yet to be confirmed.

  • Stifled innovation—There is always the potential for innovation to be stifled through the imposition of a more structured process for setting goals and focusing quality improvement efforts. As provider attention becomes focused on the national goals and measurement requirements established by the NQCB, providers may divert resources from other promising quality measurement and improvement activities that could yield even greater returns. Private-sector organizations may reduce investments in the development of new quality measures, survey instruments, and tools, some of which could represent breakthrough technologies.

Recognizing the potential for undesirable consequences such as those described above, the committee included efforts to identify solutions to these problems in the comprehensive research agenda proposed in Chapter 5. That research agenda places particular emphasis on the need to address methodological issues, such as risk adjustment, and to perform an impact assessment to monitor and correct for unintended consequences.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

POTENTIAL CONCERNS: THE RATIONALE FOR THE NQCB

While the need for the proposed NQCB is clear and compelling, the committee anticipates understandable concerns regarding the potential repercussions of implementing such a system. However, it is possible to make a strong case for the NQCB that addresses these concerns. The concerns most likely to be raised are summarized in Table 3-2 and discussed below.

The NQCB Will Be Too Bureaucratic

In recommending the NQCB, the committee is not suggesting yet another bureaucratic structure, but a centralized mechanism to promote standardization. The goal of standardization is to ensure a level information

TABLE 3-2 Concerns Regarding the Proposed NQCB and Responses to Those Concerns

Potential Concerns

Responses

The NQCB is too bureaucratic.

The NQCB is a centralized mechanism to promote standardization and a level information playing field.

The NQCB duplicates current functions.

The NQCB is more comprehensive in its proposed measure set, stakeholder groups, and reporting functions than what currently exists.

The NQCB is too costly.

The NQCB is a plausible approach to identifying waste in the health care system and improving efficiency.

The NQCB is too complicated.

The NQCB will simplify performance measurement by providing clear goals, a phased approach to implementation, and alignment of measures.

The NQCB is too burdensome for providers.

The NQCB will decrease the reporting burden by substituting a single data set and reporting format for the multiple data sets and formats currently requested by various stakeholder groups.

The NQCB could result in worse quality.

The NQCB will be responsive to the complexities of good clinical care.

The NQCB is a threat to patient privacy.

The NQCB will ensure appropriate confidentiality protections for patient data in strict compliance with regulations of the Health Insurance Portability and Accountability Act.

The NQCB will stifle local innovation.

The NQCB will serve as a foundation upon which local efforts can build.

The NQCB could pose undue hardship on local providers in underserved areas.

The NQCB will support performance measurement and reporting at the population level. Underserved areas will be carefully monitored for unintended consequences, particularly with regard to access issues related to providers.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

playing field for comparison among providers, as well as to promote efficient information collection and transfer analogous to standards setting for transportation and financial systems. Performance measures and quality information represent public goods regardless of one’s political perspective or preferred policy approach: a competitive market driven by consumer choice, regulatory approaches based on provider accreditation, or self-motivated efforts by providers to improve. Valid measures of performance are an essential foundation for improving quality and efficiency. Because the unfettered market cannot produce standardized measures, this is a legitimate arena for the government to assume a leadership role. However, the board’s focus will be, to the extent possible, on coordinating and building on existing efforts so as to avoid becoming a large bureaucracy.

The NQCB Will Be Too Costly

Some will argue that the NQCB will be too costly. Conversely, the committee argues that not establishing the NQCB will be too costly as the negative consequences of inaction are too great to be ignored. The nation’s current health care system is riddled with waste and duplication. A performance measurement system that supports fair comparisons on costs and quality offers a plausible approach to identifying waste and improving efficiency. Present approaches to quality measurement are also wasteful: duplicative, distracting, and sometimes misleading, consuming precious consumer and provider time. The goal of the proposed new measurement system is to provide a common and efficiently collected body of useful and meaningful information for all stakeholders, including providers, payers, and consumers. As stated earlier, the initial investment the committee is recommending is only 0.1 percent of Medicare’s annual budget.

The NQCB Will Be Too Complicated

At first blush, the NQCB may appear too complicated. However, current approaches to measurement are both complicated and fragmented. The NQCB, if properly implemented, should provide (1) clear goals for measurement, alleviating the problem of competing measure sets; (2) a phased approach to implementation to ensure that data collection tools and approaches are efficient and supported by electronic health records (EHRs); and (3) alignment of nearly identical measures (similar measures with different data element definitions) that currently require duplicative record collection.

The NQCB Will Impose Too Great a Burden on Providers

A major concern is that the NQCB will be too burdensome for already overtaxed providers. On the contrary, the NQCB will eliminate multiple,

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

often conflicting requests for data from private and public purchasers, accrediting bodies, and others, thus decreasing the burden of data collection. It is anticipated that the NQCB will drive the use of EHRs, enabling data collection to become part of the routine care process rather than an additional task as it is today, with the long-term goal that the data collected will support local quality improvement efforts led by providers. Caution must be exercised, however, with regard to EHRs. Their adoption rate is slow—currently estimated at 27 percent (Bates, 2005)—impeded by costs, privacy issues, lack of national data standards, and physician culture. The NQCB should monitor the adoption rate of EHRs, remaining cognizant of these impediments, and adjust its expectations and timeline accordingly. As the capacity of EHRs to support reporting of performance data is currently uncertain, the NQCB should contribute to the gathering of evidence for evaluating their effectiveness for this purpose (Baron et al., 2005; IOM, 2003; Miller et al., 2005; Sprague, 2004).

The NQCB May Result in Worse Quality

Concern might be raised that a focus on technical process measures may in some cases result in worse rather than better quality. This concern stems from the belief that current technical process measures do not adequately capture the complexity of clinical care, as in the case of frail elderly patients who often have multiple chronic conditions. Good care in this instance requires that physicians prioritize treatment objectives or in some cases choose to focus on functional improvement or quality of life rather than disease treatment. A well-functioning performance measurement and reporting system should be designed to address this concern by (1) ensuring that measures exclude groups or populations of patients for whom the guidelines (and related measures) are inappropriate; (2) allowing evidence-based and verifiable exclusions by practitioners where measures are imperfect; and (3) fostering the development of patient-centered measures of decision quality.

The NQCB Threatens Patient Privacy

Issues concerning the privacy and confidentiality of patient health information warrant heightened attention by the NQCB, particularly with regard to data aggregation. The NQCB will need to be diligent in ensuring that appropriate confidentiality protections are in place for the submission of patient data that are in strict compliance with the regulations of the Health Insurance Portability and Accountability Act. The board will also need to address the potential problem of patients opting not to have their data included in a data repository and the impact this would have on the

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

ability to accurately assess the quality of care both nationally and across communities. These issues are explored further in Chapter 4.

The NQCB Will Stifle Local Quality Improvement Efforts

As with any national compulsory structure, there could be concern that the NQCB will pose a threat to innovative local quality improvement initiatives and programs. Many regions of the country are developing advanced performance measurement systems, and some stakeholders may be concerned that the NQCB will establish a ceiling, thus precluding their own quality improvement targets and local priorities. The NQCB may require local efforts to make some modifications so that common definitions are used by all. However, the board should be flexible enough to serve as a foundation upon which local efforts can build.

The NQCB May Impose Undue Hardship on Local Providers in Underserved Areas

There may be concern that the NQCB will have a negative impact on some communities, particularly those with a shortage of providers, such as in rural and urban areas. The demands of data collection and the impact of public reporting could inadvertently influence providers to leave such underserved areas. This, coupled with emigration of more mobile residents, could force smaller clinics or hospitals to close before they can be competitive on key quality measures. The NQCB will support the collection of data at the population level, as well as the development of public reports appropriate for these communities, and will be flexible in addressing the unique needs of this constituency.

REFERENCES

Asch SM, McGlynn EA, Hogan M, Hayward R, Shekelle P, Rubenstein L, Keesey J, Adams J, Kerr E. 2004. Comparison of quality of care for patients in the Veterans Health Administration and patients in a national sample. Annals of Internal Medicine 141(12):938–945.


Baron RJ, Fabens EL, Schiffman M, Wolf E. 2005. Electronic health records: Just around the corner? Or over the cliff? Annals of Internal Medicine 143(3):222–226.

Bates DW. 2005. Physicians and ambulatory electronic health records. Health Affairs 24(5):1180–1189.


CMS (Centers for Medicare and Medicaid Services). 2004. 2004 CMS Statistics. Washington, DC: CMS.


Guidestar. 2004a. 2003 Tax Form 990 for JCAHO Surveyor and QHR Consultant Corporation. [Online]. Available: http://www.guidestar.org/FinDocuments/2003/363/673/2003-363673595-1-9.pdf [accessed July 8, 2005].

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Guidestar. 2004b. 2003 Tax Form 990 for National Committee for Quality Assurance. [Online]. Available: http://www.guidestar.org/FinDocuments/2003/521/191/2003-521191985-1-9.pdf [accessed July 8, 2005].

Guidestar. 2004c. 2003 Tax Form 990 for the National Quality Forum. [Online]. Available: http://www.guidestar.org/FinDocuments/2003/522/175/2003-522175544-1-9.pdf [accessed July 8, 2005].


Hibbard JH, Stockard J, Tusler M. 2005. Hospital performance reports: Impact on quality, market, share, and reputation. Health Affairs 24(4):1150-1160.


IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Committee on Quality of Health Care in America. Washington, DC: National Academy Press.

IOM. 2003. Key Capabilities of an Electronic Health Record System: Letter Report. Committee on Data Standards for Patient Safety. Washington, DC: The National Academies Press.


James BC. 2001. Making it easy to do it right. New England Journal of Medicine 345(13):991–993.


Kizer KW, ed. 2003. Putting the ideas into practice. Medical Care 41(Suppl. 1):I-87–I-89.


McGlynn EA. 2003. An evidence-based national quality measurement and reporting system. Medical Care 41(Suppl. 1):I-8–I-15.

McGlynn EA, Cassel CK, Leatherman ST, DeCristofaro A, Smits HL. 2003. Establishing national goals for quality improvement. Medical Care 41(Suppl. 1):I-16–I-29.

Miller RH, West C, Brown TM, Sim I, Ganchoff C. 2005. The value of electronic health records in solo or small group practices. Health Affairs 24(5):1127–1137.


Sprague L. 2004. Electronic health records: How close? How far to go? National Health Policy Forum Issue Brief (800):1–17.


Werner RM, Asch DA. 2005. The unintended consequences of publicly reporting quality information. Journal of the American Medical Association 293(10):1239–1244.

Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 63
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 64
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 65
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 66
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 67
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 68
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 69
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 70
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 71
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 72
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 73
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 74
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 75
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 76
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 77
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 78
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 79
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 80
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 81
Suggested Citation:"3 Achieving a National System for Performance Measurement and Reporting." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 82
Next: 4 Moving Forward: What Should Be Measured? »
Performance Measurement: Accelerating Improvement Get This Book
×
Buy Hardback | $75.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Performance Measurement is the first in a new series of an ongoing effort by the Institute of Medicine (IOM) to improve health care quality. Performance Measurement offers a comprehensive review of available measures and introduces a new framework to examine these measures against the six aims of the health care system: health care should be safe, effective, patient-centered, timely, efficient, and equitable. This new book also addresses the gaps in performance measurement and introduces the need for measures that are longitudinal, comprehensive, population-based, and patient-centered. This book is directed toward all concerned with improving the quality and performance of the nation's health care system in its multiple dimensions and in both the public and private sectors.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!