Criteria for Selection
The committee’s focus in determining selection criteria for the priority areas was on identifying a few core criteria that would lead to a list of areas representing opportunities to close gaps between known best practice and usual practice. Consistent with the Quality Chasm report, the committee started with the assumption that health care quality is a systems property that requires redesigning systems of care instead of just expecting health care workers to try harder (Institute of Medicine, 2001a). To date, most quality improvement efforts have been focused on changing the behavior of individual clinicians, often neglecting systems interventions (Mechanic, 2002). This report emphasizes such systems-level changes, although the committee also recognizes that improving quality requires work at multiple levels, including direct contact with clinicians and patients.
As noted in Chapter 1, the committee’s immediate focus was on identifying the most promising opportunities for improving the delivery of existing best-practice treatments through changes in health care systems and policies, as distinct from efforts to improve the efficacy of existing best-practice treatments through continued biomedical research or technological innovation. Clearly, efforts to generate new scientific information for health care are critical. But unless we can develop more effective and efficient ways of delivering health care based on the best available known treatments, it is unlikely that such new knowledge will be used to its full benefit. As stated in the Quality Chasm report:
At no time in the history of medicine has the growth in knowledge and technologies been so profound….[But] research on the quality of care reveals a health care system that frequently falls short in its ability to translate knowledge into practice, and to apply new technology safely and effectively…. If the health care system cannot consistently deliver today’s science and technology, we may conclude that it is even less prepared to respond to the extraordinary scientific advances that will surely emerge in the first half of the 21st century (Institute of Medicine, 2001a:2–3).
Determining the criteria for the selection of priority areas required making choices and value judgments based on available evidence while taking into account principles of improving population health and effecting social justice. Several key questions had to be addressed. Should the emphasis be on conditions for which the potential for improvement through systems change is greatest, or those for which the clinical burden and quality deficits are greatest? What evidence is available to inform these decisions, and what are the gaps in that evidence? And what are the trade-offs between choosing priority areas for which the benefits are more modest but predictable and those for which the benefits are more uncertain but potentially greater?
The core issue underlying these questions— how best to allocate limited resources—is a question with which policy makers in the public and private sectors must grapple every day. The Institute of Medicine (IOM) has previously addressed priority setting in three other contexts—technology assessment (Institute of Medicine, 1992), clinical practice guidelines (Institute of Medicine, 1995), and research funding (Institute of Medicine, 1998a). However, this committee’s charge was distinct from these previous efforts: to identify a set of conditions for which quality improvement within the health system could have the most benefit. Thus for candidate priority areas, the committee needed to examine:
Not only the clinical burden of illness, but also the extent of preventable burden attributable to the failure to deliver best-practice care.
Not just the degree to which the current system is broken, but also the extent to which known health care system changes could close gaps between best practice and usual care.
Not only how to improve clinical care for the population as a whole, but also how to address unacceptable disparities in health care.
Not only how to improve clinical care for individual conditions, but also how to identify an initial set of conditions for which improving care could be broadly transformative across the U.S. health care system.
General Considerations in Determining Criteria
The committee considered three general issues when discussing potential selection criteria. First, to what degree could existing evidence be a guide in assessing candidate priority areas according to the criteria? Second, how broadly should individual priority areas be defined? And finally, where should the line be drawn between the health care system and the rest of society in understanding the potential causes and consequences of the quality gaps in each area?
Determining Evidence and Measurability
It has been about 10 years since the term “evidence-based medicine” entered the medical literature (Guyatt, 1991), and since that time the term has become nearly ubiquitous in the vocabulary of clinicians, managers, and researchers. Evidence-based medicine involves the use of clinical research data to develop guidelines or best-practice standards for care. Just as evidence-based medicine can serve as a goal to guide clinical practice, the committee sought to ground its decisions, wherever possible, in quantitative data from the peer-reviewed literature and from nationally representative surveys.
While it is appealing to seek to apply the principles of evidence-based medicine to priority setting and quality improvement, a number of factors make doing so challenging for health systems (Walshe and Rundall, 2001). First, valid and widely applied measures do not exist for some exceptionally important concerns, such as functional capacity. As a result, available sources of data may not include essential gauges of the impact of some conditions in causing a loss of function or
ability to cope with everyday situations, as opposed to the resulting mortality. Although there are cross-sectional and longitudinal data on functional status available at the national1 and state levels2 these surveys provide only population-specific information and are not linked to particular interventions. Second, there is a considerably smaller evidence base for the impact of systems-based quality improvement efforts than for more traditional efforts to improve treatment efficacy for particular clinical conditions, as the former type of work is very difficult to accomplish. Third, even if more data were available, many of the most important decisions—such as how to weigh the potential for far-reaching improvements against the needs of the sickest individuals—cannot be made using quantitative data. Rather, addressing such matters requires a qualitative approach that explicitly emphasizes principles of social justice and equity.
Thus, the committee sought to incorporate the broadest possible range of evidence into its decision-making process, but to have at each step of the process a clear understanding of the sources and potential limitations of that evidence. Similarly, while it was essential for the selection of priority areas to be based on strong evidence of the existence of quality problems, it was also critical to find scientific evidence that a particular intervention in an area can result in improvement and that accurate, valid, and reliable measures to document that improvement are available and can be collected at a reasonable cost. Thus, some conditions that have a major impact on morbidity or mortality were excluded from the initial set of candidate areas because there is no strong evidence of effective interventions for them, or there are no adequate measures to indicate when improvement has occurred. Where potentially important data were lacking, this report provides recommendations for conducting further research.
Balancing Breadth and Focus
The committee sought to balance breadth and focus in defining particular priority areas. If the areas were defined broadly enough (e.g., infectious disease, chronic illnesses), it would literally have been possible to construct a list of 15 to 20 areas that included almost every condition. However, using such broad categories would likely have resulted in quality improvement efforts that were diffuse and ultimately ineffective. On the other hand, focusing on narrowly defined conditions might mean squandering an opportunity to effect more systemic change or failing to address areas across the entire age spectrum. In considering the appropriate level of specificity for the priority areas, the committee sought to identify areas that, give the available evidence, could serve as a basis for achieving meaningful quality improvement over the next 3 to 5 years.
Defining the Boundaries of the Health System
A final issue that inevitably arises in setting health care priorities is where to draw the line between the health system and the rest of society. Thus while it is known that poverty is one of the strongest and most persistent predictors of ill health, it is less clear whether or to what degree addressing this concern falls within the purview of health policy (Deaton, 2002; Marmot, 2002). In dealing with this issue, the committee considered the burden of illness and potential benefits of quality improvement from the broadest possible perspective, including benefits accruing not only to patients, but also to families, employers, and society as a whole. In keeping with the committee’s charter, however, the health system, rather than other forms of societal intervention, was considered the primary potential engine for such improvement. In addressing the problem of tobacco use, for example, tobacco counseling by health
providers, not cigarette taxes or antismoking campaigns, would be the focus for health care system-based quality improvement (Rigotti, 2002). On the other hand, the potential benefits of such counseling were considered from a broad societal perspective.
In this connection, the committee did not explicitly examine strategies for improving access to health care or eliminating disparities in care. Two other IOM reports—Care Without Coverage: Too Little, Too Late (Institute of Medicine, 2002a) and Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care (Institute of Medicine, 2002b)— focus explicitly on these problems. However, the committee did examine unwarranted variation in the delivery of quality care based on socioeconomic status, race/ethnicity, age, gender, and other demographic factors. For example, issues of the stigma associated with mental health care and of health literacy were addressed.
Criteria for Selecting the Priority Areas
The committee identified and used three interdependent criteria—impact, inclusiveness and improvability—for selecting the priority areas. Applied together, these criteria were intended to result in a set of priority areas for which quality improvement could be transformative to the U.S. health care system, resulting in wide-ranging, sustainable changes in other areas.
Impact comprises the extent of the burden—including disability, premature death, and economic costs—imposed by a condition with respect to quality and quantity of life for patients, as well as for families, employers, and society as a whole.
Improvability focuses on the degree to which gaps and unwarranted variations in the delivery of evidence-based medical care could be eliminated through changes in an area, and on opportunities to achieve major improvements in the heath care system by addressing the six national aims of quality improvement defined in the Quality Chasm report as being most important to patients and their families: safety, effectiveness, patient-centeredness, timeliness, efficiency, and equity (Institute of Medicine, 2001a). Core aspects of improvability include: (1) the existence of evidence-based standards or guidelines for effective care, (2) the availability of reliable measures that can be applied to understand and close gaps in care, (3) documented (measured, known) and unwarranted variations in care, and (4) examples of circumstances in which system changes have been applied successfully to close gaps in care.
Inclusiveness involves the applicability of the priority areas to a broad range of patients across the life span from birth to death, with particular emphasis on redressing disparities and social inequities in care and not leaving vulnerable groups behind. In addition, this criterion is defined to ensure that the full spectrum of health care services (from preventive to palliative care) would be included in an area and that a variety of health care settings, organizations, and providers would be engaged in any national health care quality improvement effort in that area.
Table 2–1 summarizes the criteria chosen by the committee for selecting priority areas for health care improvement. Each criterion is discussed in detail in the following subsections.
Recommendation 2: The committee recommends use of the following criteria for identifying priority areas:
Impact—the extent of the burden— disability, mortality, and economic costs—imposed by a condition, including effects on patients, families, communities, and societies.
TABLE 2–1 Criteria for Selecting the Priority Areas for Quality Improvement
How big is the problem?
Patient and family impact: Clinical burden of illness and quality gaps; psychosocial effect on patients and caregivers; consumer dissatisfaction with care quality
System impact: Health care system burden; health care provider burden; costs
Societal impact: Burden on society outside of the health care system
Are there significant gaps between best practice and usual care and unwarranted variations in care?
Is there evidence that existing quality gaps and variations in care can be narrowed or eliminated?
Condition improvability: Best-practice treatment or standard; valid and reliable measures for understanding and eliminating variations
System improvability: Opportunity to improve relative to six aims of the Quality Chasm report; efficacy and cost-effectiveness of efforts to close quality gaps; existing and emerging areas
Health disparity improvability: Opportunity to narrow gaps in care and disease burden for vulnerable and disadvantaged populations
Will addressing the priority area improve quality of care for a broad spectrum of patients and health care settings?
Will the priority area result in improved health and quality of life for persons who are otherwise at a disadvantage in health care?
Patient inclusiveness (breadth, reach, and equity): Age, gender, race/ethnicity, socioeconomic status, geographic location
Condition inclusiveness: Preventive care, acute care, chronic care, palliative care
Health care system inclusiveness: Intervention directed at multiple settings and constituents: inpatient and outpatient, providers, managers/policy makers
Improvability—the extent of the gap between current practice and evidence-based best practice and the likelihood that the gap can be closed and conditions improved through change in an area; and the opportunity to achieve dramatic improvements in the six national quality aims identified in theQuality Chasmreport (safety, effectiveness, patient-centeredness, timeliness, efficiency and equity).
Inclusiveness—the relevance of an area to a broad range of individuals with regard to age, gender, socioeconomic status, and ethnicity/ race (equity); the generalizability of associated quality improvement strategies to many types of conditions and illnesses across the spectrum of health care (representativeness); and the breadth of change effected through such strategies across a range of health care settings and providers (reach).
Impact: How Big Is the Problem?
The criterion of impact seeks to quantify the magnitude of the burden imposed by a particular condition at both the individual and population levels. Clinically, the burden imposed can be quantified in terms of prevalence (how many people have the condition), premature mortality (how many years of life are lost because of the condition and because of shortcomings in its treatment), and morbidity (the extent to which the condition affects function and quality of life). Summary measures—such as disability-adjusted life years (DALYs), which combines years of life lost as a result of premature death with years lived with disability (World Health Organization, 2000), and quality-adjusted life years—have been developed to combine length and quality of life into a single metric (Coffield et al., 2001; Fryback, 1998; Maciosek et al., 2001). For the health system, burden can be estimated using cost and quality data.
Impact has been the criterion used most frequently in previous priority-setting efforts. Earlier IOM reports have identified prevalence, burden of illness, and cost as key criteria for technology assessment and guideline development (Institute of Medicine, 1992; Institute of Medicine, 1995). The Medical Expenditure Panel Survey uses prevalence and cost in identifying its list of priority conditions (Cohen, 2001). Similarly, prevalence, mortality, disability, quality of life, and cost are central criteria used by the National Institutes of Health for assessing the distribution of research funding (Gross et al., 1999; Institute of Medicine, 1998a).
Why Look at Impact?
Two arguments can be made for the importance of impact as a complement to improvability in setting priorities. First, gaps in evidence on improvability can potentially be filled by data on disease impact. That is, in the absence of clear evidence on effectiveness, it can reasonably be assumed that conditions imposing the greatest burden potentially represent the greatest need for improvement. Similarly, since a small portion of the population incurs the vast majority of health expenditures in the United States (Berk and Monheit, 2001), targeting improvement toward high-cost populations/conditions could yield widespread change in the health system.
Second, even if one can show that a condition is improvable, a measure of impact is necessary to ensure a level of equity in the distribution of the resources devoted to improvement. Strict adherence to cost-effectiveness, or the “greatest good for the greatest number,” can lead to a crowding out of life-saving treatments by interventions that provide small benefit to a large portion of the population. For instance, during one priority-setting process, an initial approach focusing on cost-effectiveness analysis resulted in tooth capping being assigned a higher priority than appendectomy (Hadorn, 1991).
In contrast to this utilitarian calculus, egalitarian approaches seek to balance such considerations with attention to the least well off (Brock, 2002; Olsen, 1997). For receipt of clinical care, the least well off can be defined in at least two ways—patients who are the sickest and those who may be at particularly high risk of receiving poor-quality care. Similarly, data on cost and quality can be used to determine those potential priority areas for which the system is most in need of improvement.
To balance the above approaches, the committee examined impact using three broad types of evidence—estimates for clinical health and disease burden, estimates for system health and gaps in quality, and evidence on disparities in care.
Measuring Impact with Reference to Population Health
In 1998 an IOM panel addressed the critical issue of how best to measure health and illness across populations (Institute of Medicine, 1998b). The panel concluded that population health measures clearly need to account for both mortality and morbidity, but that more work
was needed to understand how best to combine the two into summary measures. The panel suggested that while some of the limitations of existing summary measures were methodological, others represented the fact that “all measures of population health involve choices and value judgments in both their construction and application” (Institute of Medicine, 1998b:12). Therefore, the panel encouraged “side-by-side” comparisons of different measures, informed by an understanding of the assumptions involved in each (Institute of Medicine, 1998b:20).
In keeping with those previous recommendations, the committee examined a number of sources of data for understanding the impact of disease on population health. For each candidate priority area, the committee examined data on mortality from the National Center for Health Statistics (Minino and Smith, 2001) and on prevalence and disability from the Medical Expenditure Panel Survey (Medical Expenditure Panel Survey, 1997). The committee also considered current estimates for DALYs. Rather than seeking to combine these data for a single ranking, the committee examined the assumptions underlying each measure and the similarities and differences among them. The committee also recognized some of the shortcomings of the data sources used, including inherent biases in determining the quality of a person’s life and the focus of most datasets on diseases as opposed to health status and functionality. These datasets focus on diseases because the clinical literature is organized around diseases as well as the health care system—hence the prevalence of specialists. Thus, it difficult to create an evidence framework that starts from symptoms or functioning.
How “Broken” is the Health System?
In addition to focusing on the clinical burden of disease, the committee looked at the degree to which the health system is “broken” with regard to how it cares for a particular condition. The committee used two measures— cost-of-illness data and national quality information—to identify those conditions for which system problems are the most pronounced. The committee reviewed cost data from the Medical Expenditure Panel Survey, examining costs for both specific conditions and groupings of particular relevance (e.g., patterns of costs for multiple chronic conditions). For each potential priority area, the committee looked at total population expenditures, average expenditures among affected individuals, and the gap between total expenditures and disability.
To examine quality of care across the range of candidate areas and begin to identify those areas with the greatest potential improvability, the committee examined data from the National Committee for Quality Assurance’s (NCQA) State of Managed Care Report, which rates health plans that collectively cover 90 percent of Americans enrolled in health maintenance organizations (HMOs) (National Committee for Quality Assurance, 2001). Areas in which variation across plans was largest and in which care most consistently fell short of performance goals were identified as potential candidates for quality improvement.
Improvability: Can the Problem Be Fixed?
Improvability reflects the degree to which existing gaps in the use of proven best practices can be closed for a priority area through system and policy changes that can lead to better care and better outcomes, including improved health, productivity, and quality of life. Improvability also encompasses the opportunity to achieve dramatic improvements in the six quality aims identified in the Quality Chasm report, enumerated above (Institute of Medicine, 2001a).
Evidence for Improvability
Improvability originates in the premise that an effective (and potentially cost-effective) treatment has been identified, but that this treatment is not delivered in safe or appropriate ways to all those who need it. Applying this
criterion requires evidence not only for an effective best-practice treatment, but also for the potential to eliminate existing quality gaps or unwarranted variations in clinical practice in ways that could be widely replicated throughout the health care system in both public and private settings and among a broad range of patient and provider populations.
Under ideal circumstances, evidence for improvability would include supporting data for both the efficacy and cost-effectiveness of feasible system and policy improvements. However, existing studies of health care quality improvement initiatives provide very little data on the relative efficacy or cost-effectiveness of various types of system and policy change. The committee considered the existence of valid and reliable measures for assessing the processes and outcomes of care to be a major asset in efforts to guide and monitor health care system improvement.
Cost-Effectiveness Analysis in Priority Setting
Cost-effectiveness analysis has typically been the starting point for economic models of priority setting (Weinstein et al., 1996) and was the method used in the first version of the Oregon Medicaid process. Such analysis seeks to determine how, given limited financial resources, health resources should be allocated to maximize health benefits.
While appealing from a conceptual perspective, the committee identified substantial limitations in relying excessively on cost-effectiveness estimates for its priority-setting process. First, a review of the current literature comparing cost-effectiveness across interventions (Harvard School of Public Health, 2002) showed that few studies meet criteria designed to ensure interpretability and comparability among estimates (Rigotti, 2002; Siegel et al., 1996). Second, the gaps in the literature are particularly notable for chronic conditions such as diabetes, chronic obstructive pulmonary disease, and asthma, all of which were potential priority areas for the committee. Third, even for conditions for which such analyses are available, treatments range widely in cost-effectiveness, making it difficult to summarize overall improvability for care within particular priority areas. Fourth, there are a number of special populations for whom information is scarce, such as children, the medically indigent, and those receiving home-based care. Finally, the vast majority of cost-effectiveness studies have focused on clinical treatments rather than quality improvement or system change. For this report, as noted earlier, quality improvement is defined not as a function of the development of new treatment methods or technologies, but as of the deployment of existing methods and technologies more widely and effectively through health system redesign (Chassin and Galvin, 1998).
The Agency for Healthcare Research and Quality and private organizations such as The Robert Wood Johnson Foundation are currently leading efforts to foster more rigorous studies of the effectiveness of quality improvement initiatives. As yet, however, few such studies exist (Renders et al., 2001; Schoenbaum et al., 2001; Wells et al., 2000). Generalizing findings and comparing results across interventions will likely be a complex undertaking, given how sensitive the interventions are to the context of the systems in which they are implemented. However, it will be essential to attempt to synthesize the data as they become available. For example, the U.S. Preventive Services Task Force recently published its findings from a systematic review of the literature examining the cost-effectiveness of colorectal screening. Although screening for colorectal cancer was found to be cost-effective (between $10,000 and 25,000 per life-year saved) the question of what screening method or methods are best remains unanswered (Pignone et al., 2002; United States Preventive Services Task Force, 2002).
Thus, while the committee did examine cost and cost-effectiveness data, it had to rely more on studies examining the efficacy and effectiveness of clinical treatments than on data regarding the cost-effectiveness of alternative system-based quality improvement strategies.
Because most studies typically use disease-specific outcome measures, they were more useful for ensuring that effective treatments are available than for ranking improvability across candidate priority areas. For understanding potential improvability, then, the committee supplemented quantitative research with case studies of successful efforts to improve systems of care within each of the candidate priority areas.
Inclusiveness: Breadth, Reach, and Equity
It is essential not only to examine the impact and improvability of candidate priority areas, but also to address the issue of inclusiveness: the mix of areas, the populations affected, and opportunities to address inequities in health care and health status in the United States. This criterion reflects the committee’s strong belief that the priority list needs to include areas relevant to children and adults, rich and poor, and urban and rural populations. It is also important to focus on conditions for which there are particularly large disparities in care between the general population and vulnerable racial, ethnic, economic, or geographic populations. Given the goal of broad health system change, the committee also believed it important to select priority areas that would involve interventions across different sectors and specialties within the health care delivery system. In addition, the committee wanted to ensure that the list of areas would be representative across the four broad domains of health care enumerated in Chapter 1— preventive, acute, chronic, and palliative (Institute of Medicine, 2001b). Indeed, inclusiveness considerations helped motivate the decision to include some cross-cutting areas among the priority areas to ensure that national quality improvement efforts would in some way benefit all Americans and all health care systems and providers.
The end product of the above three criteria is a set of priority areas with the potential to transform the health care system. The concept of transformative potential reflects the extent to which the priority areas, both individually and collectively, are likely to catalyze far-reaching and sustainable changes by perturbing the current system in a way that helps trigger a self-corrective course. More specifically, the concept has to do with the extent to which improvements in the delivery of care in one priority area could extend beyond this initial target to generate changes in other areas. Such influence could occur because one condition (e. g., diabetes) may be a risk factor for multiple other conditions (e.g., blindness, renal failure, artherosclerosis), or because system changes, such as creating a patient reminder system, if implemented in one area, could also improve care for other conditions. Similarly, system changes required, for instance, to integrate tobacco dependence treatment into routine care could stimulate similar improvements in other preventive services. With regard to palliative care, system changes designed to engender and document advanced care plans could be used for every fatal illness. In addition, improvements in some areas might serve as leverage points or requirements in other areas. For example, improvements in health literacy are needed to promote active patient self-management for most conditions.
Berk, M.L., and A.C.Monheit. 2001. The concentration of health care expenditures, revisited. Health Aff (Millwood)20(2):9–18.
Brock, D.2002. Priority to the worst off in health care resource prioritizations. Chapter 28 in Medicine and Social Justice.R.Rhodes, M.P. Battin, and A.Silvers, eds. New York, NY: Oxford University Press.
Centers for Disease Control and Prevention. 2002a. “Behavioral Risk Factor Surveillance System.” Online. Available at http://www.cdc.gov/brfss [accessed Oct. 15, 2002a].
——. 2002b. “National Health Interview Survey (NHIS).” Online. Available at http://www.cdc.gov/nchs/nhis.htm [accessed Oct. 15, 2002b].
Centers for Medicare and Medicaid Services. 2002. “The Medicaire Current Beneficiary Survey (MCBS).” Online. Available at http://cms.hhs.gove/mcbs/default.asp [accessed Oct. 15, 2002].
Chassin, M.R., and R.W.Galvin. 1998. The urgent need to improve health care quality. Institute of Medicine National Roundtable on Health Care Quality. JAMA280 (11): 1000–5.
Coffield, A.B., M.V.Maciosek, J.M.McGinnis, J. R.Harris, M.B.Caldwell, S.M.Teutsch, D. Atkins, J.H.Richland, and A.Haddix. 2001. Priorities among recommended clinical preventive services. Am J Prev Med21 (1):1–9.
Cohen, S.B.2001. Enhancements to the Medical Expenditure Panel Survey to improve health care expenditure and quality measurement. In Proceedings of the Statistics in Epidemiology Section, American Statistical Association.
Deaton, A.2002. Policy implications of the gradient of health and wealth. An economist asks, would redistributing income improve population health?Health Aff (Millwood)21 (2):13–30.
Fryback, D.G.1998. Methodological issues in measuring health status and health-related quality of life for population health measures: A brief overview of the “DALY” family of measures. In Summarizing Population Health: Directions for the Development and Application of Population Metrics.M.J. Field and M.R. Gold, eds. Washington, D.C.: National Academy Press.
Gross, C.P., G.F.Anderson, and N.R.Powe. 1999. The relation between funding by the National Institutes of Health and the burden of disease. N Engl J Med340 (24): 1881–7.
Guyatt, G.H.1991. Evidence-based medicine. ACP J Club114:A-16.
Hadorn, D.C.1991. Setting health care priorities in Oregon: Cost-effectiveness meets the rule of rescue. JAMA265 (17):2218–25.
Harvard School of Public Health. 2002. “The CUA Data Base: Standardizing the Methods and Practices of Cost-Effectiveness Analysis.” Online. Available at http://www.hsph.harvard.edu/organizations/hcra/cuadatabase/intro.html [accessed May 29, 2002].
Institute of Medicine. 1992. Setting Priorities for Health Technologies Assessment: A Model Process.M.S.Donaldson and H.C.Sox, Jr., eds. Washington, D.C.: National Academy Press.
——. 1995. Setting Priorities for Clinical Practice Guidelines.M.J.Field, eds. Washington, D.C.: National Academy Press.
——. 1998a. Scientific Opportunities and Public Needs: Improving Priority Setting and Public Input at the National Institutes of Health. Washington, D.C.: National Academy Press.
——. 1998b. Summarizing Population Health: Directions for the Development and Application of Population Metrics.M.J.Field and M.R. Gold, eds. Washington, D.C.: National Academy Press.
——. 2001a. Crossing the Quality Chasm: A New Health System for the 21st Century. Washington, D.C.: National Academy Press.
——. 2001b. Envisioning the National Health Care Quality Report.M.P.Hurtado, E.K. Swift, and J.M.Corrigan, eds. Washington, D. C.: National Academy Press.
——. 2002a. Care Without Coverage: Too Little, Too Late.Washington, D.C.: National Academy Press.
——. 2002b. Unequal Treatment: Confronting Racial and Ethnic Disparities in Health Care.B. S.Smedley, A.Y.Stith, and B.D.Nelson, Eds. Washington, D.C.: National Academy Press.
Maciosek, M.V., A.B.Coffield, J.M.McGinnis, J. R.Harris, M.B.Caldwell, S.M.Teutsch, D. Atkins, J.H.Richland, and A.Haddix. 2001. Methods for priority setting among clinical preventive services. Am J Prev Med21 (1):10– 9.
Marmot, M.2002. The influence of income on health: Views of an epidemiologist. Does money really matter? Or is it a marker for something else?Health Aff (Millwood)21 (2):31–46.
Mechanic, D.2002. Improving the quality of health care in the United States of America: The need for a multi-level approach. J Health Serv Res Policy7 (3 Suppl):35–9.
Medical Expenditure Panel Survey. 1997. Panel Population Characteristics and Utilization Data for 1996.AHCPR Publication 98-DP12. Rockville, MD: Agency for Health Care Policy and Research.
——. 2002. “Introduction to MEPS Data and Publications.” Online. Available at http://www.meps.ahcpr.gov/Data_Public.htm [accessed June 4, 2002].
Minino, A.M. and B.L.Smith. 2001. National Vital Statistics Reports.Centers for Disease Control and Prevention.
National Committee for Quality Assurance. 2001. The State of Managed Care Quality. Washington, D.C.: NCQA.
Olsen, J.A.1997. Theories of justice and their implications for priority setting in health care. J Health Econ16 (6):625–39.
Pignone, M., S.Saha, T.Hoerger, and J. Mandelblatt. 2002. Cost-effectiveness analyses of colorectal cancer screening: A systematic review for the U.S. Preventive Services Task Force. Ann Intern Med137 (2):96–104.
Renders, C.M., G.D.Valk, S.J.Griffin, E.H. Wagner, J.T.Eijk Van, and W.J.Assendelft. 2001. Interventions to improve the management of diabetes in primary care, outpatient, and community settings: A systematic review. Diabetes Care24 (10):1821–33.
Rigotti, N.A.2002. Clinical practice: Treatment of tobacco use and dependence. N Engl J Med 346 (7):506–12.
Schoenbaum, M., J.Unutzer, C.Sherbourne, N. Duan, L.V.Rubenstein, J.Miranda, L.S. Meredith, M.F.Carney, and K.Wells. 2001. Cost-effectiveness of practice-initiated quality improvement for depression: Results of a randomized controlled trial. JAMA286 (11):1325–30.
Siegel, J.E., M.C.Weinstein, L.B.Russell, and M. R.Gold. 1996. Recommendations for reporting cost-effectiveness analyses. Panel on Cost-Effectiveness in Health and Medicine . JAMA 276 (16):1339–41.
United States Preventive Services Task Force. 2002. “Recommendations and Rationale: Screening for Colorectal Cancer.” Online. Available at http://www.ahrq.gov/clinic/3rduspstf/colorectal/colorr.htm [accessed Nov. 20, 2002].
Walshe, K., and T.G.Rundall. 2001. Evidence-based management: From theory to practice in health care. Milbank Q79 (3):429–57, IV-V.
Weinstein, M.C., J.E.Siegel, M.R.Gold, M.S. Kamlet, and L.B.Russell. 1996. Recommendations of the Panel on Cost-effectiveness in Health and Medicine. JAMA 276 (15):1253–8.
Wells, K.B., C.Sherbourne, M.Schoenbaum, N. Duan, L.Meredith, J.Unutzer, J.Miranda, M.F. Carney, and L.V.Rubenstein. 2000. Impact of disseminating quality improvement programs for depression in managed primary care: A randomized controlled trial. JAMA283 (2):212–20.
World Health Organization. 2000. “Global Burden of Disease 2000 Version 1 Estimates.” Online. Available at http://www3.who.int/whosis/menu [accessed May 29, 2002].