National Academies Press: OpenBook

Performance Measurement: Accelerating Improvement (2006)

Chapter: Appendix E Methodology and Analytic Frameworks

« Previous: Appendix D Ten Design Principles
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Appendix E
Methodology and Analytic Frameworks

The committee’s selection of performance measures began with identifying leading performance measure sets1 and classifying these measures within nationally endorsed frameworks for quality assessment and evaluation of health system performance. Table E-1 presents the frameworks that informed the committee’s deliberations on the selection of performance measures. These analytic frameworks, briefly described below, are as follows: (1) Donabedian’s model for assessing quality; (2) the Institute of Medicine’s (IOM) six aims for quality improvement in health care; (3) the Foundation for Accountability’s (FACCT) domains of consumer needs for health care; (4) the IOM’s priority areas for national action, as adapted by the National Quality Forum (NQF); and (5) the Centers for Medicare and Medicaid Services’ (CMS’) priority chronic conditions for adults 65 and older.

The Donabedian Model

Donabedian’s (1988) classic paradigm for assessing quality of care is based on a three-component approach—structure, process, and outcomes

1  

The committee defined leading performance measure sets as (1) those currently being used for public reporting, pay-for-performance, or quality improvement efforts at the national or regional level; (2) those recognized by leading national stakeholder groups; and (3) those whose owner had a rigorous process in place for assessing validity and reliability, as well as a mechanism for updating or retiring measures.

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

TABLE E-1 Analytic Frameworks Used by the Committee

Analytic Frameworks

Framework Components

Donabedian

Structure → Process → Outcomes

IOM six aims

Safe, effective, patient-centered, timely, efficient, and equitable

FACCT domains of consumer needs

Staying healthy, getting better, living with illness or disability, and coping with end of life

IOM priority areas as expanded by NQFa

Infrastructure: information technology (standardization and capacity); patient safety (including but not limited to health care–acquired infections and medication management and adherence)

 

Processes of care: care coordination and communication, care at the end of life (focus on congestive heart failure and chronic obstructive pulmonary disease), immunizations (all ages), pain management, self-management/health literacy

 

Health care conditions: asthma; cancer; pneumonia; depression; diabetes; children with special health care needs; frailty associated with old age (preventing falls and pressure ulcers, maximizing function, and developing advanced-care plans); hypertension; ischemic heart disease; kidney disease; mental illness; obesity; pregnancy, childbirth, and newborn care; stroke; and tobacco dependence (prevention and treatment)

CMS priority chronic conditions for adults 65 and over

Ischemic heart disease; cancer; chronic obstructive pulmonary disease/asthma; stroke, including hypertension; arthritis and nontraumatic joint disorders; diabetes mellitus; dementia, including Alzheimer’s disease; pneumonia; peptic ulcer/dyspepsia; and depression and other mood disorders

aNQF endorsed the IOM’s original 20 priority areas, and added the areas of kidney disease and information technology infrastructure.

(See Figure E-1). Donabedian’s model proposes that each component has a direct influence on the next, as represented by the arrows in the following schematic (Donabedian, 1980):

FIGURE E-1 Donabedian’s model.

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Structure refers to the attributes of the settings in which providers deliver health care, including material resources (e.g., electronic health records), human resources (e.g., staff expertise), and organizational structure (e.g., hospitals vs. clinics). For example, a cardiologist may use a disease registry to track whether a patient with cardiovascular disease is receiving drugs for lowering cholesterol.

Process of care denotes what is actually done to the patient in the giving and receiving of care. Building on the example above, the provider could review whether an eligible patient has been placed on an angiotensin-converting enzyme inhibitor to help prevent future heart attacks.

Health outcomes are the direct result of a patient’s health status as a consequence of contact with the health care system. In the above example, the patient’s receiving the preventive medications mentioned above could decrease the chance of dying from a heart attack.

IOM’s Six Aims

The report Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001) calls for national action to address serious and well-documented quality shortcomings in the U.S. health care system. The report proposes a restructuring of the health care delivery system so that Americans will consistently receive the quality of care they deserve. To this end, the report recommends the adoption of six quality aims for improvement, defined as follows:

  • Safe—avoiding injuries to patients from the care that is intended to help them

  • Effective—providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoiding underuse and overuse, respectively)

  • Patient-centered—providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions

  • Timely—reducing waits and sometimes harmful delays for both those who receive and those who give care

  • Efficient—avoiding waste, including waste of equipment, supplies, ideas, and energy

  • Equitable—providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status.

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

FACCT’s Consumer Information Framework

FACCT, closed in 2004, was a nonprofit organization committed to measuring health care quality and communicating the results to consumers in a meaningful way (FACCT, 1997). In 1997, FACCT developed a customer-centered framework for quality measurement that is based on what consumers conveyed as their health care needs across the lifespan, encompassing the following four domains:

  • Staying healthy—helping people avoid illness and stay healthy through preventive care, reduction of health risks, early detection of illness, and education

  • Getting better—helping people recover when they are sick or injured through appropriate treatment and follow-up

  • Living with illness or disability—helping people with ongoing, chronic conditions (such as diabetes or asthma) take care of themselves, control symptoms, avoid complications, and maintain daily activities

  • Coping with the end of life—caring for people and their families when needs change dramatically because of disability or terminal illness, with comprehensive services, caregiver support, and hospice care.

IOM’s Priority Areas

The Quality Chasm report (IOM, 2001) recommended that no fewer than 15 priority areas be identified as the focus of quality improvement efforts, based on the premise that a limited number of chronic conditions account for the majority of the nation’s health care burden and resource use. It was argued that by focusing on a discrete set of common chronic conditions, sizable improvements in the quality of care could be made over the next decade (IOM, 2003). An IOM committee was formed to respond to this recommendation. The committee selected 20 clinical priority areas on the basis of three overarching criteria (IOM, 2003):

  • Impact—the extent of the burden—disability, mortality, and economic costs—imposed by a condition within the populations

  • Improvability—the extent of the gap between current practice and evidence-based best practice and the likelihood that the gap could be closed and conditions improved through change in an area, and the opportunity to achieve improvements in the six quality aims

  • Inclusiveness—the relevance of an area to a broad range of individuals with regard to age, gender, socioeconomic status, and ethnicity/race (equity); the generalizability of associated quality improvement strategies across

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

the spectrum of health care conditions; and the capability for change across a range of health care settings and providers.

NQF subsequently added two areas to the IOM’s original list of 20—kidney disease and information technology infrastructure—and endorsed the resulting list of 22 areas.

CMS’s Priority Chronic Conditions

A collaborative effort involving CMS, the Agency for Healthcare Research and Quality (AHRQ), the Food and Drug Administration, and other stakeholders recently identified 10 priority conditions (listed in Table E-1) that account for the majority of disease burden and service utilization for the Medicare population aged 65 and older (AHRQ, 2004). AHRQ has launched a $15 million initiative, with funding authorized under the Medicare Prescription Drug, Improvement, and Modernization Act of 2003, supporting research to investigate the effectiveness of interventions targeting these conditions, including prescription drugs. As a result of these efforts, CMS hopes ultimately to help providers and patients make more informed health care decisions.

COMMITTEE’S PROCESS FOR SELECTING PERFORMANCE MEASURES

The committee’s analysis involved a series of steps that are listed below sequentially, although the actual process was far more iterative than linear:

  1. Map measures from leading performance measure sets to a two-dimensional matrix.

  2. Assess the current state of performance measurement and identify gaps.

  3. Review measures in the matrix against the priority clinical areas.

Map Existing Measures to Matrix

To guide the selection of performance measures, the committee adopted a matrix building upon the IOM six aims and the FACCT domains representing patients’ needs across the lifespan, as described above (FACCT, 1999). More than 800 measures of structure, process, and outcomes from more than 50 leading measurement sets were pooled and mapped against this two-dimensional matrix. When appropriate, the individual measures were maintained within the context of the original measure sets.

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Assess Current State of Performance Measurement and Identify Gaps

The assignment of individual measures to matrix cells facilitated assessment of the current state of performance measurement and the identification of major gaps in existing measurement sets. The committee found that the majority of currently available measures evaluated effectiveness of care largely in acute or ambulatory settings. Conversely, there was a particular paucity of measures addressing the IOM aims of equity, efficiency, and patient-centeredness.

Review Measures Against Priority Clinical Areas

Following the initial mapping exercise, the committee took the additional step of checking against the 22 priority clinical areas to ensure the comprehensiveness of the performance measures now populating the matrix. Although some of these areas—such as pregnancy, childbirth, and newborn care—involve individuals outside the traditional boundaries of the Medicare population, the committee concluded that the recommended measurement system should represent the entire lifespan and spectrum of care to achieve the downstream goal of healthier older adults. The committee was also diligent in ensuring that the top chronic conditions affecting Medicare beneficiaries aged 65 and over were given due consideration.

Limitations to the Matrix

The matrix was a useful tool for cataloging leading performance measurement sets and identifying gaps in needed areas. However, the committee’s analytic approach has some limitations. Assigning the measures to only one category or cell in the matrix often involved arbitrary judgment, since many of the measures, such as pain control, were applicable to multiple settings of care. Additionally, while most process and outcome measures fit neatly into the predefined categories of the matrix, some measures did not. An additional column was necessary to include structural measures characterizing the care system, such as a disease registry system. Thus enhanced, the matrix served as a functional starting point to help inform the committee’s deliberations.

REFERENCES

AHRQ (Agency for Healthcare Research and Quality). 2004. List of Priority Conditions for Research under Medicare Modernization Act Released. [Online]. Available: http://www.ahrq.gov/news/press/pr2004/mmapr.htm [accessed February 1, 2005].

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×

Donabedian A. 1980. Methods for deriving criteria for assessing the quality of medical care. Medical Care Review 37(7):653–698.

Donabedian A. 1988. The quality of care. How can it be assessed? Journal of the American Medical Association 260(12):1743–1748.


FACCT (Foundation for Accountability). 1997. The FACCT Consumer Information Framework: Comparative Information for Better Health Care Decisions. [Online]. Available: http://www.facct.org/information.html [accessed June 4, 2002].

FACCT. 1999. FACCT: Quality Measures. [Online]. Available: http://www.facct.org/facct/site/facct/facct/Measures [accessed September 17, 2004].


IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.

IOM. 2003. Priority Areas for National Action: Transforming Health Care Quality. Adams K, Corrigan JM, eds. Washington, DC: The National Academies Press.

Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 170
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 171
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 172
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 173
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 174
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 175
Suggested Citation:"Appendix E Methodology and Analytic Frameworks." Institute of Medicine. 2006. Performance Measurement: Accelerating Improvement. Washington, DC: The National Academies Press. doi: 10.17226/11517.
×
Page 176
Next: Appendix F Commissioned Paper: Improving the Quality of Quality Measurement--John D. Birkmeyer, Eve A. Kerr, and Justin B. Dimick »
Performance Measurement: Accelerating Improvement Get This Book
×
Buy Hardback | $75.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Performance Measurement is the first in a new series of an ongoing effort by the Institute of Medicine (IOM) to improve health care quality. Performance Measurement offers a comprehensive review of available measures and introduces a new framework to examine these measures against the six aims of the health care system: health care should be safe, effective, patient-centered, timely, efficient, and equitable. This new book also addresses the gaps in performance measurement and introduces the need for measures that are longitudinal, comprehensive, population-based, and patient-centered. This book is directed toward all concerned with improving the quality and performance of the nation's health care system in its multiple dimensions and in both the public and private sectors.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!