National Academies Press: OpenBook

Performance Measurement: Accelerating Improvement (2006)

Chapter: 5 Research Agenda

« Previous: 4 Moving Forward: What Should Be Measured?
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

5
Research Agenda

CHAPTER SUMMARY

This chapter recommends an aggressive research agenda for the National Quality Coordination Board (NQCB) with four primary components: (1) development, implementation, and evaluation of new performance measures; (2) applied research to address underlying methodological issues; (3) design and testing of reporting formats for consumer usability; and (4) evaluation of a performance measurement and reporting system. A collaborative effort among private and public stakeholder groups led by the NQCB will be necessary to develop and fund this agenda.

In Chapter 4, the committee identifies significant gaps in current performance measurement and reporting capabilities. We argue for an accelerated effort to move beyond the status quo to ensure a broader and deeper understanding of how well the health care system is performing across all six aims of the Quality Chasm report (IOM, 2001) and, most important, where the system can be improved. This chapter focuses on the development of a research agenda that can help realize the kind of performance measurement and reporting system proposed by the committee.

One primary component of the necessary research agenda involves the development, implementation, and evaluation of performance measures. Second is applied research to address methodological issues related to data analysis, including how to minimize the effects of confounders and safeguard against misclassification of providers. Third, research is needed to determine the best formats for public reporting of performance data so that the data can be used by consumers as a decision tool in selecting high-quality providers. Finally, on a broader front, the committee has asserted the need for a national system for performance measurement and reporting to improve the quality of care for all Americans. This assertion is based on the committee’s expert analysis and lessons learned from past experience, but is not as yet supported by an evidence base. Therefore, research is needed

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

to develop a business case either supporting or refuting the need for the National Quality Coordination Board (NQCB). Moreover, the NQCB as envisioned by the committee will be a learning system. Thus it will be necessary to understand how well the entire system is functioning and to what extent these efforts to improve quality are affecting health and processes of care.

Recommendation 5: The NQCB should formulate and promptly pursue a research agenda to support the development of a national system for performance measurement and reporting. The board should develop this agenda in collaboration with federal agencies and private-sector stakeholders. The agenda should address the following:

  • Development, implementation, and evaluation of new measures to address current gaps in performance measurement.

  • Applied research focused on underlying methodological issues, such as risk adjustment, sample size, weighting, and models of shared accountability.

  • Design and testing of reporting formats for consumer usability.

  • Evaluation of the performance measurement and reporting system.

Advances in the quality of health care delivery will be markedly slower without a performance measurement and reporting system that articulates a focused research agenda. The NQCB should take responsibility for leading efforts to develop such a research agenda and to ensure its timely implementation. In this role the board will need to have contracting and grantmaking authority to support external research as well as the internal capacity to perform this function. To provide a base for these efforts, the Agency for Healthcare Research and Quality (AHRQ) and other stakeholders—both public and private—should take steps now to assess and sponsor developmental work addressing current barriers to performance measurement and reporting. The following sections address how action on the four fronts enumerated above can advance a national performance measurement and reporting system designed to enhance the quality of health care delivery.

DEVELOPMENT, IMPLEMENTATION, AND EVALUATION OF NEW MEASURES

Current efforts to develop performance measures to fill some of the gaps identified in Chapter 4 are unlikely to succeed without the more coordinated and effective leadership that the NQCB can provide in prioritizing and adequately funding a targeted research agenda to address those gaps.

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

TABLE 5-1 Priority Areas for Measure Development

Approach

Research Focus

Areas for Measure Development

Comprehensive measurement

Extend quality domains through the development of new measures.

  • Efficiency

  • Equity

  • Patient-centeredness

Longitudinal measurement

Expand a longitudinal perspective to encompass other care settings and clinical conditions.

  • Longitudinal experiences of care

  • Outcomes and efficiency of care

Patient-level, population-based, and systems-level measurement

Develop measures and approaches to measurement that support decision making by leaders at the physician group, hospital, and community levels.

  • Systems-level measures

Shared accountability

Develop measures and methods that foster shared accountability.

This is a cross-cutting approach that will be fostered by measures in the above six areas.

As recommended in Chapter 3, the research agenda developed by the NQCB should be linked to well-specified goals and aims of the health care delivery system. In Chapter 4, the committee highlights four approaches that could be taken to achieve a high-quality performance system: (1) comprehensive measurement; (2) longitudinal measurement; (3) patient-level, population-based, and systems-level measurement; and (4) shared accountability. Table 5-1 identifies six priority areas for future development of performance measures within these broad approaches. The committee believes measures developed in these areas have the potential for yielding the greatest impact on quality of care within the next 3 years.

The NQCB should identify short- and long-term goals for the development of measures in these six areas by 2008 and beyond. The NQCB should work with public and private stakeholders to support the development and promulgation of measures in these six areas. Additionally, as the identification of measurement gaps is a dynamic process, the priority areas of focus should be updated periodically.

Comprehensive Measurement

In the short term, comprehensive measurement can best be achieved by developing adequate measures that address the all of the six aims identified

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

in the Quality Chasm report. The most important gaps identified by the committee are measures of efficiency, equity, and patient-centeredness.

Efficiency

Substantial work is under way on the development of measures of efficiency that can represent the value of medical care. Prior Institute of Medicine (IOM) studies have endorsed the basic concept of avoiding waste: a more efficient care process or delivery system will produce an equal or better outcome at lower cost. The key is to be able to measure both quality and resource use for well-defined episodes of care. The following principles guided the committee’s thinking in this area: (1) measures of efficiency should be based on episodes of care of adequate duration so that the quality of care and/or outcomes of treatment can be reliably determined; (2) the scope of services and time window of observation should be broad and long enough to ensure that providers being evaluated cannot improve their apparent efficiency simply by shifting costs to other providers or to periods outside the window of observation; (3) multiple measures of efficiency (i.e., of costs and quality) for a given provider are preferable because performance may vary across the types of service provided (e.g., care for diabetes versus congestive heart failure); and (4) when possible, reliance should be placed on measures that have been reported in the peer-reviewed literature to enhance both affordability and validity.

Two broad types of efficiency measures warrant consideration: longitudinal and episodic. The committee recommends an aggressive research agenda to develop and pilot test efficiency measures of both types.

Longitudinal efficiency An example of measures of longitudinal efficiency for defined populations over relatively prolonged periods is 1-year mortality and resource use for acute conditions. The feasibility of collecting these data has been demonstrated for different types of care delivery systems (Tarlov et al., 1989; Ware et al., 1996), hospitals (Fisher et al., 2004; Guadagnoli et al., 1995), and for regional care systems within the United States (Fisher et al., 2003a,b). Such data have also been used to monitor the impact of the introduction of a prospective payment system on hospitalized patients (Kahn et al., 1990). In addition to the measure of longitudinal efficiency recommended by the committee for the starter set of measures detailed in Chapter 4—1-year mortality, resource use, and functional status after acute myocardial infarction—attention should be paid to collecting long-term follow-up data on additional conditions for which longitudinal outcomes and costs can be reliably assessed. Candidates include hip fracture and colorectal cancer, given their relative frequency, the high rates of hospitalization associated with these conditions (allowing population-based comparisons of outcomes at the com-

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

munity level), and the potential for these data to provide insight into multiple care systems (orthopedics and oncology) and care settings (rehabilitation, ambulatory care, and acute hospital).


Episodic efficiency Measurement of the efficiency of episodic care refers to a unit of analysis that reflects the level of resources used in the care of a specific, relatively brief episode (e.g., acute back pain) as part of the total care received by patients. Examples of such measures are methods for calculating adjusted average payments for all patient refined-diagnosis related groups and episode treatment groups. Many researchers have identified the need for measuring episodic efficiency to address issues ranging from cost containment and attribution to reduction in waste. Issues such as nonstandardized use of these measures, validity of and availability of data sources, and risk adjustment hinder progress in this area, however. (For further discussion, refer to Appendix H.)

Equity

Multiple studies have demonstrated marked variations in access to health care (Cassil and Sorian, 2002; IOM, 2002, 2003, 2005; Isaacs and Schroeder, 2004; Sheikh and Bullock, 2001). As equity is a cross-cutting quality aim, it is important that it be measured not only to achieve comprehensive measurement, but also to test how well the health care system is functioning on all other quality aims. The committee was thus concerned by the relatively few measures available for evaluating equity, particularly with regard to issues of access and disparities in care. Greater attention should be focused on these issues, with consideration of the following measures and methods.


Access An important area of disparity in care is health insurance coverage. Ambulatory care measures, which reflect the quality of care for individuals in any ambulatory care setting, are one useful kind of equity measure. Yet they are obtained most easily by sampling only insured populations. Thus greater use should be made of hospital-based measures, which include all patients at a given institution regardless of their payer or insurance status. Other important issues of access include those related to transportation, service hours, and manpower. Rural communities are a particularly critical population to assess, as they often have limited access to high-quality care (IOM, 2005). The committee believes that in the short term, it will be necessary to identify representative samples of patients from all sites where the uninsured may receive care—whether uncompensated care from physicians’ office-based practices or emergency rooms, or care provided by established safety net providers.

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Disparities The committee strongly advocates the collection of performance measures addressing health care services at the individual level, allowing for aggregation to various levels of providers, geographic regions, and demographics. The critical issue in measuring racial, ethnic, or socioeconomic disparities is the need for data aggregation and reporting systems that can provide this stratification when sample sizes are large enough to yield reliable estimates. These efforts should be coordinated with those of other organizations characterizing the equity of care. The National Healthcare Disparities Report has made significant progress in this area, but much research remains to be done (AHRQ, 2003).

Additionally, equity should be assessed using measures that can capture variations in care by (1) region of the country; (2) type of community (i.e., rural versus urban, as the former tend to comprise sicker and poorer populations); (3) availability of care; and (4) patient race, ethnicity, and class. The committee views such research as a top priority of the NQCB so as to minimize the detrimental impacts of inequity in health care on underserved and disadvantaged populations (AHRQ, 2003; IOM, 2001, 2002, 2004). Although data systems may not be sufficient to support reporting of such measures within the next year, the committee believes such systems can be in place by 2008, and calls for an aggressive effort to that end.

Patient-Centered Care

The committee recommends expanding the current repertoire of patient-centered measures so as to gain insight based on patients’ experiences, as patients are a valuable part of the interconnected chain of care delivery. In accordance with design principle 6 for a national system for performance measurement and reporting—a central role for the patient’s voice (see Chapter 2)—the committee defines patient-centered care as “providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions” (IOM, 2001:6). Data that capture measures of health care that matter to patients can be a powerful influence on how care is delivered by providers, purchased by payers, and adhered to by patients.

The CAHPS initiative, discussed in Chapter 2, has been a forerunner in systematically capturing patients’ perspectives on their care. The committee endorses use of the entire CAHPS family of surveys, which have been developed for a variety of settings—ambulatory care, hospital, health plan, in-center dialysis center, and nursing home—as part of the NQCB’s measurement strategy. In addition, three promising dimensions of patient-centeredness that require attention are (1) self-assessment of patients’ level of engagement in their care (Hibbard, 2004; Hibbard et al., 2004); (2) patients’ input on the quality of the delivery of their chronic disease care

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

(Bodenheimer et al., 2002; Glasgow et al., 2005; Lorig et al., 2001, 2004); and (3) information on whether patients who faced major treatment choices received accurate information and support in making choices aligned with their values, a parameter termed “decision quality” (Mulley, 1989, 2004; Sepucha et al., 2004). The committee views the incorporation of additional patient-centered measures as a top priority for achieving a more balanced, less provider-centric approach to performance measurement.

Longitudinal Measurement

As suggested earlier, longitudinal measurement will help break down the boundaries created by the current silos of the health care system. Two sets of measures are needed: measures of care transitions, or how well patients’ care is coordinated as they enter into and out of different health care settings, and measures of longitudinal efficiency. As the latter was discussed above, this section focuses on measures of care transitions, characterized as both longitudinal experiences of care and outcomes of care. While gains have been made in care transitions within the hospital setting, further emphasis should be placed on patients transferring from ambulatory to other care settings.

Longitudinal Experiences of Care

The assessment of care transitions is critical as it is at these points that breakdowns and errors in care are most likely to occur (Coleman and Berenson, 2004). Evaluation of transitions requires longitudinal measurement to determine whether the health care needs of patients have been met irrespective of where their care is delivered—hospital, nursing home, or home care. Measures in this area should provide the impetus for moving toward care that transcends the various care settings and the fragmentation promoted by disease-specific care.

Four measure sets for care transitions are candidates for implementation within the next 3 years: (1) the California Healthcare Foundation’s Patients’ Evaluation of Performance in California Survey, (2) the University of Colorado Health Sciences Center’s Care Transitions Measure, (3) Hospital Consumer Assessment of Healthcare Providers and Systems, and (4) the Assessing Care of Vulnerable Elders measure. All of these measures reflect the patient’s experiences and rely on self-reported responses to items during either a telephone or written survey. (For further discussion of care transitions and these measures, see Appendix I.) The committee proposes that the NQCB evaluate which of these candidate measures would be most appropriate for immediate use.

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Outcomes of Care

Patient outcomes are the ultimate indicator of the quality of care received. These important measures reflect the extent to which providers are delivering high- or low-quality care, as well as the functioning of the broader health care system. However, outcomes measures are often difficult to use for quality improvement purposes. Multiple confounders are associated with health outcomes, such as patient adherence, societal factors, and the long time frames required to yield significant results. Health services researchers have wrestled for decades with models for statistical adjustment that can protect against holding providers accountable for such confounding determinants of outcome. Much progress has been made in understanding simple adjustments, such as for age, as well as more difficult ones, such as for comorbidity. In addition, more is now understood about the policy implications of the choices of adjusters. For example, race may correlate with outcome. A mortality measure that adjusts for race in effect excuses the care system from responsibility for race-related factors—a decision with profound policy implications.

The committee believes consideration should be given to measures of two outcomes of care particularly salient to patients: disease-specific mortality and pain control.


Disease-specific mortality The committee proposes the following 30-day and 1-year disease-specific mortality measures for consideration: acute myocardial infarction, coronary artery bypass graft, percutaneous coronary intervention, and end-stage renal disease. Sufficient epidemiologic work has been done on these measures to permit both appropriate adjustments for demographics and comorbidities and widespread practical adoption. As always, consideration must be given to the available sample sizes and to the expression of risks in terms of confidence intervals. It is likely that small hospitals and individual practitioners simply have too few relevant cases to be included in a disease-specific mortality measurement strategy.


Pain control Qualitative studies have demonstrated that recognition and treatment of pain are important priorities for patients receiving palliative care; however, validated measures of pain control are not yet available for widespread use (Lynn, 2000). The committee proposes that the NQCB consider which measures of pain assessment and changes in pain management are the most promising candidates for implementation, with evidence for their reliability and validity. (For more discussion of the evidence base and areas for further research with respect to pain control, see Appendix J.)

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Individual-Patient-Level, Population-Based, and Systems-Level Measurement

The importance of individual-patient-level and population-based measures was highlighted above. In addition, systems-level measures are needed to assess the performance of both the smaller entities constituting the overall health care delivery system (such as hospitals and health plans) and the overall system itself. Measures of this type therefore have implications for purchasers and providers wishing to compare the performance of these smaller systems relative both to each other and to larger systems. The committee believes that with targeted attention, systems-level measures could be ready for implementation by 2008.

An example of such measures is mortality measures, which entail some controversy. The primary issue is the classic problem of severity or case-mix adjustment. Hospitals and other care providers facing comparisons of outcome measures, especially those as significant as mortality, understandably become concerned about fairness with respect to severity variables beyond their control. Unmeasured determinants of outcome, unevenly distributed among providers, may masquerade as effects of care itself, thus penalizing providers who simply are dealing with more problematic, higher-risk patients at the outset.

The committee considered two types of mortality measures: (1) disease-specific mortality, such as from cancer or ischemic heart disease, discussed in the previous section, and (2) hospitalwide mortality, summing experience over many diagnoses. The former could, in principle, characterize the quality of specialty care or clinical services, while the latter might reflect systemic and organizational characteristics with broader impact, such as teamwork, supervision, information management, or adequacy of the physical plant.

The committee achieved consensus on disease-specific mortality measures, but was unable to do so with respect to hospitalwide mortality measures. The Hospital Standardised Mortality Rate model has been widely discussed in the peer-reviewed literature and is now in significant use in the United Kingdom and, in earlier stages, in the United States (Jarman et al., 1999). Several committee members suggested that this model should be included in the initial set of performance measures to provide additional experience with its use, as well as information on its correlations with other structural, process, and outcome measures. These members suggested that mortality, as a systemic characteristic, is simply too important to ignore when initiating a consolidated measurement system, and that the use of a recognized approach, even if still developmental, would be prudent. Other committee members expressed skepticism about the technical aspects of the Hospital Standardised Mortality Rate

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

model in particular and about the more general theoretical foundation for attempting to measure mortality as a hospitalwide characteristic, given how hospital-specific variations in end-of-life care can influence such measures (Fisher et al., 1994). The NQCB will need to address these issues as the measurement development effort goes forward.

Shared Accountability

Shared accountability is cross-cutting in that it holds all providers who partake in a patient’s care responsible for the outcomes of that care. There is no single method for achieving shared accountability. Assessing care longitudinally across time and space can require the evaluation of care for a patient from the hospital to the nursing home. Composite measures of care reinforce this overall approach by focusing on treatment for all aspects of a patient’s condition. Measurement at both the population and systems levels addresses the larger health care system and includes the societal factors that contribute to the health of the general public. The committee therefore believes that development and promulgation of measures in all of these areas foster shared accountability. It will become increasingly necessary to develop models of shared accountability as the focus shifts away from measuring care by setting, as discussed in the next section.

APPLIED RESEARCH TO ADDRESS UNDERLYING METHODOLOGICAL ISSUES

The NQCB should support research aimed at resolving key methodological issues surrounding performance measurement so as to enhance the accuracy and integrity of the data obtained. If measurement methodologies are flawed, data can be misleading, potentially threatening providers’ reputations and falsely portraying the quality of care provided. The committee calls particular attention to the following issues:

  • Risk adjustment—This statistical tool allows data to be modified to control for variations in patient populations. For example, risk adjustment could be used to ensure a fair comparison of the performance of two providers: one whose caseload consists mainly of elderly patients with multiple chronic conditions and another who treats a patient population with a less severe case mix. Risk adjustment makes it possible to take these differences into account when resource use and health outcomes are compared.

  • Sample size—Small sample sizes may make conclusions statistically invalid, particularly when used for ranking individual providers. For instance, depictions of a physician’s performance may be inaccurate if she has

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

treated only five patients for congestive heart failure, because if a random event has occurred, her performance rating will be skewed.

  • Weighting of elements for composite measures—The issue here is whether to place more emphasis on any particular component of a composite measure, as discussed in Chapter 4. For example, weighting of components of the prevention composite measure would address whether a physician treating a woman with a history indicating an increased risk of breast cancer should be scored higher for providing this screening as opposed to giving smoking cessation advice.

  • Shared accountability—As noted, the committee espouses the concept of shared accountability as a way to encourage better care coordination and to shift away from measuring and rewarding care by setting. Models are needed for determining how best to hold accountable all providers involved in a patient’s care—e.g., a group of providers who prevented hospital readmission for a typical Medicare beneficiary with four chronic conditions—and to reward high-quality care.

Resolution of these methodological issues is critical for accurate data reporting. The NQCB should therefore ensure that these issues, as well as others it deems important, are promptly addressed.

DESIGN AND TESTING OF REPORTING FORMATS FOR CONSUMER USABILITY

If performance measures are to have the intended effects on the way care is provided, as well as on the health outcomes of patients, it is essential that they be reported so as to be clear and meaningful for those who wish to use the data. There is a broad audience for public reports on care, ranging from providers and purchasers to patients. To date, attention has focused mainly on how purchasers and providers respond to public reports and how their responses affect their behaviors; little attention has been focused at the patient level. Additionally, reports often are not tailored to the needs of special populations who may vary widely in their specific health information needs, language, and level of health literacy. The committee believes the usability of public reports of comparative health care performance data needs to be a focus of further research, as these reports currently are not produced in formats that resonate with consumers. Inadequate or inaccurate public reports can undermine the confidence of both consumers and clinicians in the value of public disclosure of performance information. Knowing what measures are meaningful to consumers is also important. Reports need to be produced so they can be understood by consumers and assist those searching for a provider (Farley Short et al., 2002; Hibbard et

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

al., 2002). If these goals are not met, then public reports will have little effect, if any, on consumers and their choice of care. Emphasis in this area of research should be placed on how best to design and test formats for public reporting for consumers of health care (Shaller et al., 2003; Vaiana and McGlynn, 2002).

EVALUATION OF A SYSTEM FOR PERFORMANCE MEASUREMENT AND REPORTING

The NQCB should not be a static entity, but rather a dynamic learning system that continually evaluates itself and advances understanding of the impact of performance measurement. The committee proposes that assessment of the NQCB be carried out at time intervals that allow for continual improvement and midcycle corrections as needed (Deming, 1994; Langley et al., 1996). It is critical to determine whether the NQCB is having the intended consequences—ultimately the attainment of the six quality aims of the Quality Chasm—through intermediate outcomes such as better care processes. Just as important, the ongoing monitoring of the NQCB should serve to safeguard against unintended consequences, such as adverse selection. Table 5-2 presents a summary of what should be encompassed by an impact assessment of the NQCB, as discussed in detail below.

Intended Consequences

Performance measurement should yield knowledge and enable inferences about the effects of health system changes in such areas as payment policies, public reporting, benefit design, accreditation/certification, and quality oversight. Assessment of the NQCB should elucidate whether these changes to the health care system are inducing behaviors, particularly among providers, that result in improved patient care. For example, it should be possible to address the following key questions more fully as a result of the performance measurement activities overseen by the NQCB:

  • Is performance measurement contributing to a closer evaluation of care processes so that providers are capable and desirous of changing the way they organize and deliver care to achieve improved quality?

  • Does performance measurement assist providers in making wiser choices concerning the allocation of resources by addressing efficiency and the overuse of services that have been demonstrated to show no benefit or possibly even harm to patients?

  • Does performance measurement encourage more rapid uptake of information technology by physician practices, thus facilitating the exchange of patient information among multiple providers?

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

TABLE 5-2 Impact Assessment of the NQCB

Intended Consequences

Unintended Consequences

Gain knowledge of important health system changes

  • Payment policies of public and private purchasers

  • Pay for performance

  • Public reporting programs

  • Benefit design

  • Accreditation and certification programs

  • Quality oversight processes

Foster belief that measurement in and of itself can improve care

  • Investment in measurement without a focus on improving care

  • Data collection burden

  • Misclassification

Induce desirable provider behaviors

  • Better understanding of care processes, leading to actions that improve quality

  • Wise use of resources

  • Investment in information technology infrastructure

  • Enhanced cooperation among providers

  • Accelerated innovation

Induce undesirable provider behaviors

  • Gaming

  • Adverse selection

Close known quality gaps

  • 22 priority areas

  • Populations of focus (equity)

Sustain quality gaps

  • Patient harm

  • Community harm

  • Does performance measurement foster cooperation among providers so that care is better integrated and more patient-centered?

  • Does performance measurement spur innovation rather than stifle creativity?

In addition to lessons learned about interventions introduced into the health care system to improve care (e.g., pay for performance) and influence provider behavior, the NQCB needs to be assessed to determine whether it is indeed closing known quality gaps, as well as eliminating disparities in health care. A potential risk of not doing so is that measurement will be done simply for its own sake, without serving the primary purpose of moving closer to achieving the six quality aims.

Unintended Consequences

In addition to assessing whether the NQCB is having the intended consequences or desired outcomes, it will be equally important to identify any

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

unintended consequences of the demands imposed by the system. As noted above, measurement itself must not be viewed as capable of improving care, but as a catalyst for actions that can do so. Other potential consequences that warrant close monitoring include the potential burden of data collection on the health care system, as well as on individual providers; misclassification of providers, particularly if data are publicly reported; gaming of the system; and adverse selection of healthier patients to improve scores. Perhaps the most serious unintended consequence is that quality gaps will persist, resulting in harm to both patients and communities.

FUNDING

Recommendation 6: Congress should provide the financial resources needed to carry out the research agenda developed by the NQCB. The AHRQ should collaborate with Grantmakers in Health and others that have ties to local foundations to convene public- and private-sector stakeholders currently investing in various aspects of this research agenda for the purpose of identifying complementary investment strategies.

Achieving the goal of a comprehensive national system for performance measurement and reporting will require the development and implementation of new measures, methodologies, and reporting formats, as well as thorough evaluation of the system. Accomplishing these tasks will in turn require commitment from both public and private stakeholders. Collaboration among these stakeholders could jumpstart much-needed development of measures to fill the gaps identified in this report, as well as the formulation of evidence-based consensus guidelines to serve as the basis for measure development. The NQCB should receive adequate funding to ensure the implementation of a robust research agenda, such as that proposed in this chapter. The committee recommends that the NQCB work closely with AHRQ, who has an established track record in funding evidence-based health services research, and other groups that can provide linkages between foundations and community collaborations, such as Grantmakers in Health, to align investment strategies for carrying out this agenda.

REFERENCES

AHRQ (Agency for Healthcare Research and Quality). 2003. National Healthcare Disparities Report. Rockville, MD: AHRQ.


Bodenheimer T, Lorig K, Holman H, Grumbach K. 2002. Patient self-management of chronic disease in primary care. Journal of the American Medical Association 288(19):2469–2475.

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Cassil A, Sorian R. 2002. 1 in 4 Uninsured Americans with Chronic Conditions Can’t Get Needed Care. Washington, DC: Center for Studying Health System Change.

Coleman EA, Berenson RA. 2004. Lost in transition: Challenges and opportunities for improving the quality of transitional care. Annals of Internal Medicine 141(7):533–536.


Deming WE. 1994. The New Economics for Industry, Government, and Education (2nd Edition). Cambridge, MA: Massachusetts Institute of Technology Center for Advanced Engineering Study.


Farley Short P, McCormack L, Hibbard J, Shaul JA, Harris-Kojetin L, Fox MH, Damiano P, Uhrig JD, Cleary PD. 2002. Similarities and differences in choosing health plans. Medical Care 40(4):289–302.

Fisher ES, Wennberg JE, Stukel TA, Sharp SM. 1994. Hospital readmission rates for cohorts of Medicare beneficiaries in Boston and New Haven. New England Journal of Medicine 331(15):989–995.

Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. 2003a. The implications of regional variations in Medicare spending. Part 1: The content, quality, and accessibility of care. Annals of Internal Medicine 138(4):273–287.

Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ, Lucas FL, Pinder EL. 2003b. The implications of regional variations in Medicare spending. Part 2: Health outcomes and satisfaction with care. Annals of Internal Medicine 138(4):288–298.

Fisher ES, Wennberg DE, Stukel TA, Gottlieb DJ. 2004. Variations in the longitudinal efficiency of academic medical centers. Health Affairs Suppl. Web Exclusive: VAR19-32.


Glasgow RE, Wagner EH, Schaefer JM, Mahoney LD, Reid RJ, Greene SM. 2005. Development and validation of the patient assessment of chronic illness care (PACIC). Medical Care May 43(5):436–444.

Guadagnoli E, Hauptman PJ, Ayanian JZ, Pashos CL, McNeil BJ, Cleary PD. 1995. Variation in the use of cardiac procedures after acute myocardial infarction. New England Journal of Medicine 333(9):573–578.


Hibbard J. 2004. Measuring Patient Activation. PowerPoint Slides, Alliance of Community Health Plans Advancing Better Care Conference, February 9–10.

Hibbard JH, Slovic P, Peters E, Finucane ML. 2002. Strategies for reporting health plan performance information to consumers: Evidence from controlled studies. Health Services Research 37(2):291–313.

Hibbard JH, Stockard J, Mahoney ER, Tusler M. 2004. Development of the Patient Activation Measure (PAM): Conceptualizing and measuring activation in patients and consumers. Health Services Research 39 (4 Pt 1):1005–1026.


IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.

IOM. 2002. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care. Smedley BS, Stith AY, Nelson BD, eds. Washington, DC: The National Academies Press.

IOM. 2003. Insuring Health—Hidden Costs, Value Lost: Uninsurance in America. Washington, DC: The National Academies Press.

IOM. 2004. Insuring Health—Insuring America’s Health: Principles and Recommendation. Committee on the Consequences of Uninsurance, eds. Washington, DC: The National Academies Press

IOM. 2005. Quality Through Collaboration: The Future of Rural Health. Washington, DC: The National Academies Press.

Isaacs SL, Schroeder SA. 2004. Class: The ignored determinant of the nation’s health. New England Journal of Medicine 351(11):1137–1142.


Jarman B, Gault S, Alves B, Hider A, Dolan S, Cook A, Hurwitz B, Iezzoni LI. 1999. Explaining differences in English hospital death rates using routinely collected data. British Medical Journal 318(7197):1515–1520.

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Kahn KL, Keeler EB, Sherwood MJ, Rogers WH, Draper D, Bentow SS, Reinisch EJ, Rubenstein LV, Kosecoff J, Brook RH. 1990. Comparing outcomes of care before and after implementation of the DRG-based prospective payment system. Journal of the American Medical Association 264(15):1984–1988.


Langley G, Nolan KM, Norman CL, Provost LP, Nolan TW, Norman CL. 1996. The Improvement Guide: A Practical Approach to Enhancing Organizational Performance. San Francisco, CA: Jossey-Bass.

Lorig KR, Sobel DS, Ritter PL, Laurent D, Hobbs M. 2001. Effect of a self-management program on patients with chronic disease. Effective Clinical Practice 4(6):256–262.

Lorig KR, Ritter PL, Laurent DD, Fries JF. 2004. Long-term randomized controlled trials of tailored-print and small-group arthritis self-management interventions. Medical Care 42(4):346–354.

Lynn J. 2000. Learning to care for people with chronic illness facing the end of life. Journal of the American Medical Association 284(19):2508-2511.


Mulley AG Jr. 1989. Assessing patients’ utilities. Can the ends justify the means? Medical Care 27(Suppl. 3):S269–S281.

Mulley AG Jr. 2004. Performance Measurement Workshop: Patient-Centered Measurement of Decision Quality. Presentation to the IOM Committee on Performance Measures, Payment and Performance Improvement Programs, December 1–3, 2004, Washington, DC.


Sepucha KR, Fowler FJ Jr, Mulley AG Jr. 2004. Policy support for patient-centered care: The need for measurable improvements in decision quality. Health Affairs var.54.

Shaller D, Sofaer S, Findlay SD, Hibbard JH, Delbanco S. 2003. Perspective: Consumers and quality-driven health care: A call to action. Health Affairs 22(2):95–101.

Sheikh K, Bullock C. 2001. Urban-rural differences in the quality of care for Medicare patients with acute myocardial infarction. Archives of Internal Medicine 161(5):737–743.


Tarlov, AR, Ware JE Jr., Greenfield S, Nelson EC, Perrin E, Zubkoff M. 1989. The Medical Outcomes Study. An application of methods for monitoring the results of medical care. Journal of the American Medical Association 262(7):925–930.


Vaiana ME, McGlynn EA. 2002. What cognitive science tells us about the design of reports for consumers. Medical Care Research and Review 59(1):35–59.


Ware, JE Jr., Bayliss MS, Rogers WH, Kosinski M, Tarlov AR. 1996. Differences in 4-year health outcomes for elderly and poor, chronically ill patients treated in HMO and fee-for-service systems: Results from the Medical Outcomes Study. Journal of the American Medical Association 276(13):1039–1047.

Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 113
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 114
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 115
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 116
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 117
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 118
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 119
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 120
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 121
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 122
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 123
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 124
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 125
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 126
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 127
Suggested Citation:"5 Research Agenda." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 128
Next: Appendix A Glossary »
Performance Measurement: Accelerating Improvement Get This Book
×
Buy Hardback | $75.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Performance Measurement is the first in a new series of an ongoing effort by the Institute of Medicine (IOM) to improve health care quality. Performance Measurement offers a comprehensive review of available measures and introduces a new framework to examine these measures against the six aims of the health care system: health care should be safe, effective, patient-centered, timely, efficient, and equitable. This new book also addresses the gaps in performance measurement and introduces the need for measures that are longitudinal, comprehensive, population-based, and patient-centered. This book is directed toward all concerned with improving the quality and performance of the nation's health care system in its multiple dimensions and in both the public and private sectors.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!