1
Introduction

Abstract: This chapter describes the legislative mandate and scope of work for the current study, as well as previous Institute of Medicine reports on Clinical Practice Guideline (CPG) development and related topics. It also briefly touches on the current state of CPGs and common challenges faced by many CPG developers and users, which will be further explored in Chapters 2 and 3. An updated definition of CPG is provided along with an argument to differentiate high-quality, evidence-based CPGs from other forms of clinical guidance. The new definition reads: Clinical Practice Guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Following the definition, the committee identifies essential attributes for a CPG to be considered trustworthy. Finally, the committee’s methodology for deriving its eight standards for trustworthy CPGs, enumerated in Chapters 4 and 5, is described.

Nationally and around the world, health professionals increasingly understand that health care must be based on a combination of scientific evidence, knowledge gained from clinical experience, and patient value judgments and preferences. Clinical Practice Guidelines



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 15
1 Introduction Abstract: This chapter describes the legislative mandate and scope of work for the current study, as well as previous Institute of Medicine reports on Clinical Practice Guideline (CPG) develop- ment and related topics. It also briefly touches on the current state of CPGs and common challenges faced by many CPG developers and users, which will be further explored in Chapters 2 and 3. An updated definition of CPG is provided along with an argument to differentiate high-quality, evidence-based CPGs from other forms of clinical guidance. The new definition reads: Clinical Practice Guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Following the definition, the committee identifies essential attributes for a CPG to be considered trustworthy. Finally, the committee’s meth- odology for deriving its eight standards for trustworthy CPGs, enumerated in Chapters 4 and 5, is described. Nationally and around the world, health professionals increas- ingly understand that health care must be based on a combination of scientific evidence, knowledge gained from clinical experience, and patient value judgments and preferences. Clinical Practice Guidelines 15

OCR for page 15
16 CLINICAL PRACTICE GUIDELINES WE CAN TRUST (CPGs) ideally reflect this understanding when they are developed via a transparent process by a group of multidisciplinary experts (including patient representatives) screened for minimal potential bias and conflicts of interest, and supported by a systematic review of the evidence. Rather than dictating a one-size-fits-all approach to patient care, CPGs should aid clinician and patient decision making by clearly describing and appraising the evidence and reasoning regarding the likely benefits and harms related to specific clinical recommendations. This chapter provides background on the history of Institute of Medicine (IOM) involvement in issues related to CPG development, a discussion of the scope of work for this study, and a new definition for trustworthy CPGs. BUILDING ON PREVIOUS IOM STUDIES The IOM’s involvement in the clinical practice guideline arena began with the Omnibus Budget Reconciliation Act of 1989,1 which created the Agency for Health Care Policy and Research (AHCPR), now known as the Agency for Healthcare Research and Quality (AHRQ), to focus on outcomes and effectiveness research. The agency was created in part due to frustrations with “ceaselessly escalating healthcare costs, wide variations in medical practice pat - terns, evidence that some health services are of little or no value, and claims that various kinds of financial, educational, and orga- nizational incentives can reduce inappropriate utilization” (IOM, 1990, p. 2). A relatively small portion of the agency’s budget was dedicated to creation and update of guidelines through a public– private enterprise. The agency contracted with the IOM for expert advice on launching this function, and in 1990, the IOM’s Committee to Advise the Public Health Service on Clinical Practice Guidelines issued its report, Clinical Practice Guidelines: Directions for a New Pro- gram. In 1990, the IOM defined CPGs as “systematically developed statements to assist practitioner and patient decisions about appro - priate health care for specific clinical circumstances” (IOM, 1990, p. 8). The first IOM guideline committee recommended that CPGs include the following: • A statement about the strength of the scientific evidence and expert judgment 1 Omnibus Budget Reconciliation Act of 1989, Public Law 101-239, 101st Cong. (De- cember 19, 1989).

OCR for page 15
17 INTRODUCTION • An explanation of the rationale for any deviations from the scientific evidence • Key stakeholders or their representatives as participants in the development process, either through membership on the development panel, formal testimony, or other forms of participation • Consideration of implementation and evaluation issues from the beginning of the development process In 1990, before completion of the first report to AHCPR, the IOM formed a second expert committee to assess processes of guide- line development and use, and recommended an improved con- ceptual and practical framework for future CPGs. Its report, Guide- lines for Clinical Practice: From Development to Use, published in 1992, expanded on the 1990 report, adopted the eight guideline attributes proposed in 1990 (Box 1-1), and stressed the importance of both CPG content and the development process in guideline credibility and utility. The study also identified four problems with guideline efforts at that time: 1. Multiple organizations developing guidelines was an inef- ficient use of resources, resulting in varying methods of de- velopment and uneven quality. Also, selection of guideline topics was not coordinated. 2. The guideline development process often lacked method- ological quality controls. Guideline users and review criteria designers had no means for assessing the quality of methods and products of various developers. 3. Most guidelines failed to meet the needs of multiple stake- holders, such as quality assurance, cost control, comparison with alternative practices, and medical liability reduction. Because evidence and rationale for guidelines typically were not offered, the educational value of the guidelines was limited. 4. Finally, because only a limited evaluation of the impact of clinical practice guidelines had been conducted, their ef- fectiveness in improving quality of care was indeterminate (IOM, 1992). During this period, concerns regarding the selection of guideline topics by AHCPR caused Congress to direct AHCPR to report on methods for setting priorities for guideline topics. This resulted in the IOM report, Setting Priorities for Clinical Practice Guidelines (IOM,

OCR for page 15
18 CLINICAL PRACTICE GUIDELINES WE CAN TRUST BOX 1-1 Recommended Attributes of CPGs Validity: Practice guidelines are valid if, when followed, they lead to the health and cost outcomes projected for them, with other things being equal. A prospective assessment of validity will consider the projected health out- comes and costs of alternative courses of action, the relationship between the evidence and recommendations, the substance and quality of the sci- entific and clinical evidence cited, and the means used to evaluate the evidence. Reliability/Reproducibility: Practice guidelines are reliable and repro- ducible: (1) if—given the same evidence and methods for guidelines de- velopment—another set of experts would produce essentially the same statements; and (2) if—given the same circumstances—the guidelines are interpreted and applied consistently by practitioners or other appropriate parties. A prospective assessment of reliability may consider the results of independent external reviews and pretests of guidelines. Clinical Applicability: Practice guidelines should be as inclusive of appro- priately defined patient populations as scientific and clinical evidence and expert judgment permit, and they should explicitly state the populations to which statements apply. Clinical Flexibility: Practice guidelines should identify the specifically known or generally expected exceptions to their recommendations. Clarity: Practice guidelines should use unambiguous language, define terms precisely, and use logical, easy-to-follow modes of presentation. Multidisciplinary Process: Practice guidelines should be developed by a process that includes participation by representatives of key affected groups. Participation may include serving on panels that develop guide- lines, providing evidence and viewpoints to the panels, and reviewing draft guidelines. Scheduled Review: Practice guidelines should include statements about when they should be reviewed to determine whether revisions are war- ranted, given new clinical evidence or changing professional consensus. Documentation: The procedures followed in developing guidelines, the participants involved, the evidence used, the assumptions and rationales accepted, and the analytic methods employed should be meticulously docu- mented and described. SOURCE: IOM (1990).

OCR for page 15
19 INTRODUCTION 1995), which recommended principles and methods for prioritizing guideline topics. Although many of that report’s recommendations became moot when AHCPR discontinued its guideline development function, one conclusion of the report remains relevant: the agency should serve as a clearinghouse and disseminate guidelines devel- oped by others. In 2008, the current status of clinical practice guidelines was discussed in the IOM report, Knowing What Works in Health Care: A Roadmap for the Nation (IOM, 2008). The committee identified many of the quality problems cited by earlier committees, and in remediation of those long-standing problems, recommended a public–private “program” to create standards for guideline development and a process of documentation of adherence to standards. Development standards that the committee identified included • objectivity in both the development process and the conclu- sions, resulting from a balanced panel; • transparency of the deliberative process in all aspects, includ- ing conflicts of interest among panelists or the sponsoring organization, the methods of data gathering and evidence assessment, and evaluation of strength of recommendations; • efficiency and timeliness when considering clinicians’ need for timely advice and minimization of duplication and re- source wastage by guideline developers; • external review by outside experts of draft guidelines and independent party oversight of developer responses to re- viewer comments; • continuous monitoring of relevant literature so that guide- lines are reevaluated when important new evidence is pro- duced and their currency is ensured; and • reduction of overlaps and gaps through voluntary efforts to increase consensus among various organizations develop- ing guidelines on the same topics and to address topics that need guidelines (IOM, 2008). Furthermore, Knowing What Works in Health Care (IOM, 2008) and the 2009 IOM report, Conflict of Interest in Medical Research, Edu- cation, and Practice (IOM, 2009), address conflicts of interest specific to guideline development. That issue will be discussed thoroughly in Chapter 4.

OCR for page 15
20 CLINICAL PRACTICE GUIDELINES WE CAN TRUST CURRENT STATE OF CLINICAL PRACTICE GUIDELINES The development and application of CPGs have evolved dra- matically in the 20 years since the IOM first became involved in this area. Specialty societies, disease advocacy groups, the federally supported U.S. Preventive Services Task Force, health plans, com- mercial companies, and other organizations in the United States and across the globe have produced thousands of guidelines of widely varying quality (Coates, 2010). This current IOM study was initiated in response to concerns about the state-of-the-art of the quality of CPGs. Many current clinical practice guidelines suffer from limita- tions in the scientific evidence base and shortcomings in the guide - line development process (IOM, 2008; Shaneyfelt and Centor, 2009). First, the evidence base is limited by an absence or paucity of studies on relevant topics, as well as variable quality of individual scientific studies and systematic reviews of the efficacy and effectiveness of medical and surgical procedures, treatments, drugs, and devices. Consequently, CPG recommendations often rely on low quality evi- dence or expert opinion. For example, in a study of practice bulletins from the American College of Obstetricians and Gynecologists, just 29 percent of recommendations were supported by “good and con- sistent scientific evidence” (Chauhan et al., 2006, p. 94). Furthermore, evidence for clinical decision making regarding patients with rare conditions or subpopulations, such as patients with comorbidities and the socially and economically disadvan- taged, is often nonexistent or inaccurate. Consequently, many guidelines either do not address or apply to a significant number of patients (Boyd, 2010). Also, few CPGs address many significant clin- ical conditions and decisions; consequently, tension persists between the need for guidance in a particular clinical area and the lack of an evidence base to allow the development of high-quality guidelines (IOM, 2008). The quality of many CPGs is further diminished by the process used to develop some CPGs, including CPG development panel formation without sufficient attention to conflicts of interest (either financial or intellectual) or to balancing bias and including all rel- evant topical and methodological disciplines and stakeholders; poor coordination with systematic review groups, which does not permit tailoring of reviews to the specific CPG topic; and a lack of transpar- ency concerning the derivation of recommendations (Coates, 2010; Jacobs, 2010; Koster, 2010). GDGs may cherry pick studies that sup- port their positions, even when high quality SRs are available. Or, different reviewers of high-quality individual studies and system-

OCR for page 15
21 INTRODUCTION atic reviews on the same topic may reach inconsistent conclusions because of conflicting interests or research methodologies (Shrier et al., 2008). Ultimately, these evidence and process deficiencies have led to a plethora of often-conflicting recommendations produced by multiple organizations, and little means for potential CPG users to evaluate CPG quality and identify the most trustworthy CPGs (Coates, 2010). Knowledge of which guidelines are based on high-quality sci- entific evidence and employ desired standards of development is important for clinicians and other guideline users. Knowing What Works in Health Care recommended further focus on these issues (IOM, 2008). This study builds on that earlier report. STUDY SCOPE Study Objectives The U.S. healthcare environment has evolved dramatically since the IOM’s early guideline reports. This committee was challenged to determine how that evolution has affected and should affect the CPG development process, exploring which prior recommendations are no longer relevant, where progress has been made, and what new problems have arisen. Legislative Mandate The U.S. Congress, through the Medicare Improvements for Patients and Providers Act (MIPPA) of 2008,2 took steps to implement two rec- ommendations of IOM’s Knowing What Works in Health Care (2008). It called on the Secretary of Health and Human Services (HHS) to contract with the IOM to “develop evidence-based, methodologi- cal standards for systematic reviews” (IOM, 2008, p. 108) and to develop standards for clinical practice guidelines. Specifically, Con- gress requested the IOM to undertake two studies: one “to identify the methodological standards for conducting systematic review of clinical effectiveness research on health and health care in order to ensure that organizations conducting such reviews have information on methods that are objective, scientifically valid, and consistent” and the other to focus on “the best methods used in developing clinical practice guidelines in order to ensure that organizations 2 Medicare Improvements for Patients and Providers Act, Public Law 110-275, 110th Cong. (July 15, 2008).

OCR for page 15
22 CLINICAL PRACTICE GUIDELINES WE CAN TRUST developing such guidelines have information on approaches that are objective, scientifically valid, and consistent.”3 Request from the Department of Health and Human Services The Secretary of HHS, through AHRQ, contracted with the IOM in July 2009 to conduct these studies. The IOM formed two commit- tees with expertise across a broad range of relevant domains. The committees worked independently, but in coordination, to enhance the relevance of each study to the other and to minimize duplication. The Committee on Standards for Developing Trustworthy Clinical Practice Guidelines produced the report that follows, and the Com - mittee on Standards for Systematic Reviews of Comparative Effec- tiveness Research produced a companion report. IOM Committee Charge Given the MIPPA legislative charge to the IOM mentioned previ- ously, the IOM formed the Committee on Standards for Developing Trustworthy Clinical Practice Guidelines to concentrate on processes for the development of unbiased, scientifically valid, and trustwor- thy clinical practice guidelines. The committee examined existing standards for guideline development, assessing which might be suit- able for adoption or adaptation in the United States. It considered methods for modifying CPGs for patients with multiple conditions, ways to reduce the number of overlapping guidelines and harmo- nize CPGs on the same topic, methods to distinguish and promote adoption of quality CPGs, and procedures for identifying strong rec- ommendations that could become the basis for measures of quality of care. Throughout the study process, the committee was mindful of the great number and variety of guideline developers and users, and systematically elicited their input. Relation to Committee on Standards for Systematic Reviews of Comparative Effectiveness Research High-quality, effective guidelines depend on high-quality, com- parative effectiveness research and systematic reviews (SRs) of 3 Medicare Improvements for Patients and Providers Act, Public Law 110-275, 110th Cong. (July 15, 2008).

OCR for page 15
23 INTRODUCTION that evidence. A systematic review is a scientific investigation that focuses on a specific question and uses explicit, preplanned scientific methods to identify, select, assess, and summarize the findings of individual, relevant studies. It may or may not include a quantita- tive synthesis of the results from separate studies (meta-analysis). For SRs to support development of a CPG, the SRs need to address clinical questions critical to formulation of a recommendation about a particular clinical practice in a certain circumstance. Guideline Development Groups (GDGs) and SR teams may be highly inter- related, where a GDG might conduct its own SR, or an SR member might serve on a GDG panel. However, for the purpose of dividing the two MIPPA studies, all activities related to selecting, assessing and summarizing findings of individual studies fittingly are covered by the SR report. As a result of the substantive association of the two mandated MIPPA studies, the IOM took the unusual step of ensuring close collaboration of the two committees. In addition to two study staffs working cooperatively, the IOM created an informal Coordinating Group, consisting of the chairs, vice chairs, and another member from each committee. The Coordinating Group met in person four times during the study course and held conference calls to share insight on interrelated issues and build on the progress of both committees. In the end, however, each committee authored its own report and recommendations. Research Methods Essential to the IOM committee formation process is the recruit- ment of relevant subject–matter experts and individuals with rel - evant experience, as well as inclusion of a diversity of opinion and sociodemographics. These tenets are consistent with guidance found within the standard-setting literature (Norcini and Shea, 1997). Once formed, the committee complied with established standards by requiring review of the relevant published (including grey) litera- ture, solicitation of expert advice, and public comment through the activities described below. Literature Searches The committee conducted an initial literature search in late 2009 of the following databases: MEDLINE, Embase, PsychInfo, Global Health, Web of Science, and several grey literature sources. Comput- erized literature searches were updated routinely and the table of

OCR for page 15
24 CLINICAL PRACTICE GUIDELINES WE CAN TRUST contents alerts from more than 20 journals were monitored through- out the committee’s term. Searches focused on articles with “Clinical Practice Guideline” or “Practice Guideline” in the topic heading, and any of the following terms in the title or abstract: consensus development, decision-making, development, evaluation, imple- mentation, comorbidities, heterogeneity, policy, law, legal, implica- tions, tool, taxonomy, reimbursement, measurement, performance, performance measures, consumer, public, grading, rating, issues, concerns, methods, quality, electronic medical record, computer decisions support system. The committee reviewed abstracts and complete texts for more than 2,500 peer-reviewed, published articles. For a complete description of the committee’s literature search, see Appendix E. Gathering Data from the Web Data from the National Guideline Clearinghouse and the Guide- lines International Network, collected from the web, formed founda- tions for some committee analyses and provided considerable back- ground on the current status of guidelines. The web also contributed information on other major activities in the CPG world. Public Forum, January 11, 2010 Sixteen individuals and organizations responded to the commit- tee’s broadly disseminated invitation to present public testimony at its second meeting. Respondents were asked to offer advice regard- ing the key issues the committee should address in deriving stan- dards for clinical practice guideline development. Some speakers made statements on behalf of organizations; others made personal statements or offered expert counsel. These statements and testimo- nials, as well as speaker responses to committee queries, were valu- able contributions to committee understanding of the CPG status quo, its limitations, and improvement options. Workshop Panels The committee asked representative organizations and individ- uals to address questions of interest across a series of public panel presentations and discussions. Panelists responded to the follow- ing questions: (1) What do you believe are the biggest challenges that clinical practice guidelines developers face today? (2) What are the biggest challenges that CPG users face? (3) What topics and/

OCR for page 15
25 INTRODUCTION or processes do you think the committee should consider in deriv- ing quality standards for clinical practice guidelines? Twenty-four individuals participated and presented valuable information to the committee based on their experiences in developing and applying CPGs. Public Comments In addition to the January experience, members of the public communicated with the committee through its public website, staff, and direct contact with committee members. Those contributions enriched committee understanding. All submitted materials have been placed in the project’s Public Access File, which is accessed through the National Academy of Sciences website. Commissioned Papers Staff commissioned several papers from technical experts to complement the committee’s expertise across the following topics: • “The State-of-the-Art of CPG Development and Best Practices,” by Paul Shekelle, Steve Woolf, Martin Eccles, Jeremy Grimshaw, and Holger Schünemann • “Legal and Administrative Alternatives for Enhancing CPG Quality and Adherence,” by Ronen Avraham and William Sage • “The Implementation and Evaluation of CPGs in the Present,” by Anne Sales, David Atkins, Mori Krantz, and Leif Solberg • “The Implementation and Evaluation of CPGs in an Electronic Future,” by Walter Stewart, J. B. Jones, Jon Darer, and Dean F. Sittig These papers contributed to committee discussion and the evi- dentiary underpinnings of its report, although the perspectives and any implicit recommendations within each are solely those of the authors. PURPOSE AND UPDATED DEFINITION OF CLINICAL PRACTICE GUIDELINES Clinical Practice Guidelines are statements that include recom- mendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the ben -

OCR for page 15
26 CLINICAL PRACTICE GUIDELINES WE CAN TRUST efits and harms of alternative care options. To be trustworthy, guide- lines should • be based on a systematic review of the existing evidence; • be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups; • consider important patient subgroups and patient prefer- ences, as appropriate; • be based on an explicit and transparent process that mini- mizes distortions, biases, and conflicts of interest; • provide a clear explanation of the logical relationships be- tween alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations; and • be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations. The new definition provides clarification of the term “CPG,” which in the past was commonly used to describe clinical guidance derived from widely different development processes and statements taking various forms, including consensus statements, practice bul - letins, expert advice, quality measures, and evidence-based recom - mendations. This study emphasizes characteristics that distinguish among methodologically rigorous, transparent, evidence-based guidelines (CPGs) and other forms of clinical guidance. Although the committee recognizes that other forms of clinical guidance may have value, addressing them was beyond the scope of this report. Furthermore, the committee is aware that for many clinical domains, there is little or no high-quality evidence; however, guideline devel - opers may still produce trustworthy CPGs if they follow a rigorous and transparent process, as will be detailed. This study recommends standards for development of trustworthy guidelines in Chapters 4 and 5, and methods for identifying those guidelines in Chapter 7. Trustworthy CPGs have the potential to reduce inappropriate practice variation, enhance translation of research into practice, and improve healthcare quality and safety. Patient and public involve- ment and trust in guideline development and their engagement in CPG implementation will enhance adoption of guidelines by all stakeholders. Optimally, CPGs may • guide clinician and patient decision making based on evi- dence regarding the care outcomes that particular practices are expected to produce;

OCR for page 15
27 INTRODUCTION • provide a basis for measuring, evaluating, and improving provider performance and quality of care; • support appropriate resolution of malpractice claims by considering guideline recommendations as a standard of care; • contribute to the development of clinical decision support systems and other decision aids; • assist in educating patients, caregivers, and the media re- garding best healthcare practices; and • aid policy makers in the allocation of healthcare resources. Many of these activities are supported by CPGs, but they are not always informed by rigorous assessment of the science. Guidelines based on systematic literature review, consideration of the benefits and harms associated with particular recommendations, and unam- biguous translation of this appraisal for clinicians and patients could enhance the above activities. ORGANIZATION OF REPORT Six chapters follow this introductory chapter. Chapter 2 pres- ents background and an overview of CPGs, including a historical perspective on evidence-based medicine and the evolution of and a description of major participants in CPG development. A critical appraisal of the current status of CPGs follows in Chapter 3, including assessment of the limitations that exist in the scientific evidence base and the CPG development process. Chapters 4 and 5 discuss current best practices in the guideline development process and emphasize critical elements in the creation of trustworthy CPGs, including methods for systematic appraisal of the guideline development process. These chapters include pro - posed standards for guideline development. Chapter 6 examines strategies for enhancing the likelihood of guideline adoption, and applying health information technology to promote guideline implementation. Additionally, it considers the legal implications of guidelines. This chapter includes recommenda- tions for guideline implementation interventions. Chapter 7 discusses national policy issues related to CPGs. The committee makes a recommendation for how to identify high- quality CPGs because even with quality standards, practitioners, patients, and other potential users can have difficulties recognizing which guidelines are unbiased, scientifically valid, and trustworthy. The chapter also includes recommendations concerning guideline

OCR for page 15
28 CLINICAL PRACTICE GUIDELINES WE CAN TRUST harmonization, assessing the reliability and validity of proposed standards, and process and impact evaluation. REFERENCES Boyd, C. 2010. CPGs for people with multimorbidities. Paper presented at IOM Com- mittee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, November 11, 2010, Washington, DC. Chauhan, S. P., V. Berghella, M. Sanderson, E. F. Magann, and J. C. Morrison. 2006. American College of Obstetricians and Gynecologists practice bulletins: An overview. American Journal of Obstetrics and Gynecology 194(6):1564–1572. Coates, V. 2010. National Guidelines Clearinghouse NGC/ECRI Institute. Paper presented at the IOM Committee on Standards for Developing Trustworthy Clinical Prac- tice Guidelines meeting, January 11, 2010, Washington, DC. IOM (Institute of Medicine). 1990. Clinical practice guidelines: Directions for a new pro- gram. Edited by M. J. Field and K. N. Lohr. Washington, DC: National Academy Press. IOM. 1992. Guidelines for clinical practice: From development to use. Edited by M. J. Field and K. N. Lohr. Washington, DC: National Academy Press. IOM. 1995. Setting priorities for clinical practice guidelines. Edited by M. J. Field. Wash- ington, DC: National Academy Press. IOM. 2008. Knowing what works in health care: A roadmap for the nation. Edited by J. Eden, B. Wheatley, B. McNeil, and H. Sox. Washington, DC: The National Academies Press. IOM. 2009. Conflict of interest in medical research, education, and practice. Edited by B. Lo and M. J. Field. Washington, DC: The National Academies Press. Jacobs, A. 2010. American College of Cardiology and American Heart Association. Pre- sented at the IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, January 11, 2010, Washington, DC. Koster, M. A. 2010. Technology assessment and guidelines unit: Kaiser Permanente South- ern California. Paper presented at IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, January 11, 2010, Washington, DC. Norcini, J. J., and J. A. Shea. 1997. The credibility and comparability of standards. Applied Measurement in Education 10(1):39–59. Shaneyfelt, T. M., and R. M. Centor. 2009. Reassessment of clinical practice guidelines: Go gently into that good night. JAMA 301(8):868–869. Shrier, I., J.-F. Boivin, R. Platt, R. Steele, J. Brophy, F. Carnevale, M. Eisenberg, A. Furlan, R. Kakuma, M. Macdonald, L. Pilote, and M. Rossignol. 2008. The inter- pretation of systematic reviews with meta-analyses: An objective or subjective process? BMC Medical Informatics and Decision Making 8(1):19.