BOX 4-1

The IOM 2001 Recommendations for Measure Selection Criteria for the NHQR and NHDR

In the IOM’s 2001 report Envisioning the National Healthcare Quality Report, three major criteria were proposed for measure selection:

  1. Importance of what is being measured

    • Impact on health. What is the impact on health associated with this problem?

    • Meaningfulness. Are policy makers and consumers concerned about this area?

    • Susceptibility to being influenced by the health care system. Can the health care system meaningfully address this aspect or problem?

  1. Scientific soundness of the measure

    • Validity. Does the measure actually measure what it is intended to measure?

    • Reliability. Does the measure provide stable results across various populations and circumstances?

    • Explicitness of the evidence base. Is there scientific evidence available to support the measure?

  1. Feasibility of using the measure

    • Existence of prototypes. Is the measure in use?

    • Availability of required data across the system. Can information needed for the measure be collected in the scale and time frame required?

    • Cost or burden of measurement. How much will it cost to collect the data needed for the measure?

    • Capacity of data and measure to support subgroup analyses. Can the measure be used to compare different groups of the population?

The 2001 IOM report stipulated that it is desirable for a measure to meet all 10 elements within the three overall criteria, but noted that it is not required that all 10 apply in order for a given measure to be considered for inclusion in the NHQR and NHDR.


The 2001 IOM committee indicated that the three criteria, as listed above, provide a hierarchy by which measures should be considered, with priority to be given to measures evaluated for importance and scientific soundness and then by feasibility. For example, the committee stated:

Measures that address important areas and are scientifically sound, but are not feasible in the immediate future, deserve potential inclusion in the data set and further consideration. However, measures that are scientifically sound and feasible, but do not address an important problem area, would not qualify for the report regardless of the degree of feasibility or scientific soundness.

SOURCE: IOM, 2001, pp.83 and 87.

chosen for Healthy People 2010 (cancer, diabetes, end-stage renal disease, heart disease, HIV/AIDS, maternal and child health, mental health, respiratory disease, and nursing home and home health care) (AHRQ, 2003a; HHS, 2009b).

AHRQ’s Current Measure Selection Process and Criteria

AHRQ reduced the number of measures presented in subsequent editions in response to criticisms that the first edition was unwieldy (Gold and Nyman, 2004). The intent was to be able to “highlight measures with in-depth analysis, rather than broad, but sparse, coverage of all 179 measures” (AHRQ, 2004).2 That basic format is maintained by AHRQ today, with a set of approximately 46 core measures presented in the body of the reports and more detailed tables available online for a larger set of measures. To select the 46 core measures for the NHQR and NHDR, AHRQ staff and the HHS Interagency Workgroup prioritized measures by the three original IOM criteria and several additional ones. Usability was added as a new primary criterion—one that is also articulated

2

Additional measures were added to the initial full measure set.



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement