Appendix E
Methodology and Analytic Frameworks
The committee’s selection of performance measures began with identifying leading performance measure sets1 and classifying these measures within nationally endorsed frameworks for quality assessment and evaluation of health system performance. Table E-1 presents the frameworks that informed the committee’s deliberations on the selection of performance measures. These analytic frameworks, briefly described below, are as follows: (1) Donabedian’s model for assessing quality; (2) the Institute of Medicine’s (IOM) six aims for quality improvement in health care; (3) the Foundation for Accountability’s (FACCT) domains of consumer needs for health care; (4) the IOM’s priority areas for national action, as adapted by the National Quality Forum (NQF); and (5) the Centers for Medicare and Medicaid Services’ (CMS’) priority chronic conditions for adults 65 and older.
The Donabedian Model
Donabedian’s (1988) classic paradigm for assessing quality of care is based on a three-component approach—structure, process, and outcomes
TABLE E-1 Analytic Frameworks Used by the Committee
Analytic Frameworks |
Framework Components |
Donabedian |
Structure → Process → Outcomes |
IOM six aims |
Safe, effective, patient-centered, timely, efficient, and equitable |
FACCT domains of consumer needs |
Staying healthy, getting better, living with illness or disability, and coping with end of life |
IOM priority areas as expanded by NQFa |
Infrastructure: information technology (standardization and capacity); patient safety (including but not limited to health care–acquired infections and medication management and adherence) |
|
Processes of care: care coordination and communication, care at the end of life (focus on congestive heart failure and chronic obstructive pulmonary disease), immunizations (all ages), pain management, self-management/health literacy |
|
Health care conditions: asthma; cancer; pneumonia; depression; diabetes; children with special health care needs; frailty associated with old age (preventing falls and pressure ulcers, maximizing function, and developing advanced-care plans); hypertension; ischemic heart disease; kidney disease; mental illness; obesity; pregnancy, childbirth, and newborn care; stroke; and tobacco dependence (prevention and treatment) |
CMS priority chronic conditions for adults 65 and over |
Ischemic heart disease; cancer; chronic obstructive pulmonary disease/asthma; stroke, including hypertension; arthritis and nontraumatic joint disorders; diabetes mellitus; dementia, including Alzheimer’s disease; pneumonia; peptic ulcer/dyspepsia; and depression and other mood disorders |
aNQF endorsed the IOM’s original 20 priority areas, and added the areas of kidney disease and information technology infrastructure. |
(See Figure E-1). Donabedian’s model proposes that each component has a direct influence on the next, as represented by the arrows in the following schematic (Donabedian, 1980):
Structure refers to the attributes of the settings in which providers deliver health care, including material resources (e.g., electronic health records), human resources (e.g., staff expertise), and organizational structure (e.g., hospitals vs. clinics). For example, a cardiologist may use a disease registry to track whether a patient with cardiovascular disease is receiving drugs for lowering cholesterol.
Process of care denotes what is actually done to the patient in the giving and receiving of care. Building on the example above, the provider could review whether an eligible patient has been placed on an angiotensin-converting enzyme inhibitor to help prevent future heart attacks.
Health outcomes are the direct result of a patient’s health status as a consequence of contact with the health care system. In the above example, the patient’s receiving the preventive medications mentioned above could decrease the chance of dying from a heart attack.
IOM’s Six Aims
The report Crossing the Quality Chasm: A New Health System for the 21st Century (IOM, 2001) calls for national action to address serious and well-documented quality shortcomings in the U.S. health care system. The report proposes a restructuring of the health care delivery system so that Americans will consistently receive the quality of care they deserve. To this end, the report recommends the adoption of six quality aims for improvement, defined as follows:
-
Safe—avoiding injuries to patients from the care that is intended to help them
-
Effective—providing services based on scientific knowledge to all who could benefit and refraining from providing services to those not likely to benefit (avoiding underuse and overuse, respectively)
-
Patient-centered—providing care that is respectful of and responsive to individual patient preferences, needs, and values and ensuring that patient values guide all clinical decisions
-
Timely—reducing waits and sometimes harmful delays for both those who receive and those who give care
-
Efficient—avoiding waste, including waste of equipment, supplies, ideas, and energy
-
Equitable—providing care that does not vary in quality because of personal characteristics such as gender, ethnicity, geographic location, and socioeconomic status.
FACCT’s Consumer Information Framework
FACCT, closed in 2004, was a nonprofit organization committed to measuring health care quality and communicating the results to consumers in a meaningful way (FACCT, 1997). In 1997, FACCT developed a customer-centered framework for quality measurement that is based on what consumers conveyed as their health care needs across the lifespan, encompassing the following four domains:
-
Staying healthy—helping people avoid illness and stay healthy through preventive care, reduction of health risks, early detection of illness, and education
-
Getting better—helping people recover when they are sick or injured through appropriate treatment and follow-up
-
Living with illness or disability—helping people with ongoing, chronic conditions (such as diabetes or asthma) take care of themselves, control symptoms, avoid complications, and maintain daily activities
-
Coping with the end of life—caring for people and their families when needs change dramatically because of disability or terminal illness, with comprehensive services, caregiver support, and hospice care.
IOM’s Priority Areas
The Quality Chasm report (IOM, 2001) recommended that no fewer than 15 priority areas be identified as the focus of quality improvement efforts, based on the premise that a limited number of chronic conditions account for the majority of the nation’s health care burden and resource use. It was argued that by focusing on a discrete set of common chronic conditions, sizable improvements in the quality of care could be made over the next decade (IOM, 2003). An IOM committee was formed to respond to this recommendation. The committee selected 20 clinical priority areas on the basis of three overarching criteria (IOM, 2003):
-
Impact—the extent of the burden—disability, mortality, and economic costs—imposed by a condition within the populations
-
Improvability—the extent of the gap between current practice and evidence-based best practice and the likelihood that the gap could be closed and conditions improved through change in an area, and the opportunity to achieve improvements in the six quality aims
-
Inclusiveness—the relevance of an area to a broad range of individuals with regard to age, gender, socioeconomic status, and ethnicity/race (equity); the generalizability of associated quality improvement strategies across
-
the spectrum of health care conditions; and the capability for change across a range of health care settings and providers.
NQF subsequently added two areas to the IOM’s original list of 20—kidney disease and information technology infrastructure—and endorsed the resulting list of 22 areas.
CMS’s Priority Chronic Conditions
A collaborative effort involving CMS, the Agency for Healthcare Research and Quality (AHRQ), the Food and Drug Administration, and other stakeholders recently identified 10 priority conditions (listed in Table E-1) that account for the majority of disease burden and service utilization for the Medicare population aged 65 and older (AHRQ, 2004). AHRQ has launched a $15 million initiative, with funding authorized under the Medicare Prescription Drug, Improvement, and Modernization Act of 2003, supporting research to investigate the effectiveness of interventions targeting these conditions, including prescription drugs. As a result of these efforts, CMS hopes ultimately to help providers and patients make more informed health care decisions.
COMMITTEE’S PROCESS FOR SELECTING PERFORMANCE MEASURES
The committee’s analysis involved a series of steps that are listed below sequentially, although the actual process was far more iterative than linear:
-
Map measures from leading performance measure sets to a two-dimensional matrix.
-
Assess the current state of performance measurement and identify gaps.
-
Review measures in the matrix against the priority clinical areas.
Map Existing Measures to Matrix
To guide the selection of performance measures, the committee adopted a matrix building upon the IOM six aims and the FACCT domains representing patients’ needs across the lifespan, as described above (FACCT, 1999). More than 800 measures of structure, process, and outcomes from more than 50 leading measurement sets were pooled and mapped against this two-dimensional matrix. When appropriate, the individual measures were maintained within the context of the original measure sets.
Assess Current State of Performance Measurement and Identify Gaps
The assignment of individual measures to matrix cells facilitated assessment of the current state of performance measurement and the identification of major gaps in existing measurement sets. The committee found that the majority of currently available measures evaluated effectiveness of care largely in acute or ambulatory settings. Conversely, there was a particular paucity of measures addressing the IOM aims of equity, efficiency, and patient-centeredness.
Review Measures Against Priority Clinical Areas
Following the initial mapping exercise, the committee took the additional step of checking against the 22 priority clinical areas to ensure the comprehensiveness of the performance measures now populating the matrix. Although some of these areas—such as pregnancy, childbirth, and newborn care—involve individuals outside the traditional boundaries of the Medicare population, the committee concluded that the recommended measurement system should represent the entire lifespan and spectrum of care to achieve the downstream goal of healthier older adults. The committee was also diligent in ensuring that the top chronic conditions affecting Medicare beneficiaries aged 65 and over were given due consideration.
Limitations to the Matrix
The matrix was a useful tool for cataloging leading performance measurement sets and identifying gaps in needed areas. However, the committee’s analytic approach has some limitations. Assigning the measures to only one category or cell in the matrix often involved arbitrary judgment, since many of the measures, such as pain control, were applicable to multiple settings of care. Additionally, while most process and outcome measures fit neatly into the predefined categories of the matrix, some measures did not. An additional column was necessary to include structural measures characterizing the care system, such as a disease registry system. Thus enhanced, the matrix served as a functional starting point to help inform the committee’s deliberations.
REFERENCES
AHRQ (Agency for Healthcare Research and Quality). 2004. List of Priority Conditions for Research under Medicare Modernization Act Released. [Online]. Available: http://www.ahrq.gov/news/press/pr2004/mmapr.htm [accessed February 1, 2005].
Donabedian A. 1980. Methods for deriving criteria for assessing the quality of medical care. Medical Care Review 37(7):653–698.
Donabedian A. 1988. The quality of care. How can it be assessed? Journal of the American Medical Association 260(12):1743–1748.
FACCT (Foundation for Accountability). 1997. The FACCT Consumer Information Framework: Comparative Information for Better Health Care Decisions. [Online]. Available: http://www.facct.org/information.html [accessed June 4, 2002].
FACCT. 1999. FACCT: Quality Measures. [Online]. Available: http://www.facct.org/facct/site/facct/facct/Measures [accessed September 17, 2004].
IOM (Institute of Medicine). 2001. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, DC: National Academy Press.
IOM. 2003. Priority Areas for National Action: Transforming Health Care Quality. Adams K, Corrigan JM, eds. Washington, DC: The National Academies Press.