Abstract: This chapter describes the legislative mandate and scope of work for the current study, as well as previous Institute of Medicine reports on Clinical Practice Guideline (CPG) development and related topics. It also briefly touches on the current state of CPGs and common challenges faced by many CPG developers and users, which will be further explored in Chapters 2 and 3. An updated definition of CPG is provided along with an argument to differentiate high-quality, evidence-based CPGs from other forms of clinical guidance. The new definition reads: Clinical Practice Guidelines are statements that include recommendations intended to optimize patient care that are informed by a systematic review of evidence and an assessment of the benefits and harms of alternative care options. Following the definition, the committee identifies essential attributes for a CPG to be considered trustworthy. Finally, the committee’s methodology for deriving its eight standards for trustworthy CPGs, enumerated in Chapters 4 and 5, is described.
Nationally and around the world, health professionals increasingly understand that health care must be based on a combination of scientific evidence, knowledge gained from clinical experience, and patient value judgments and preferences. Clinical Practice Guidelines
(CPGs) ideally reflect this understanding when they are developed via a transparent process by a group of multidisciplinary experts (including patient representatives) screened for minimal potential bias and conflicts of interest, and supported by a systematic review of the evidence. Rather than dictating a one-size-fits-all approach to patient care, CPGs should aid clinician and patient decision making by clearly describing and appraising the evidence and reasoning regarding the likely benefits and harms related to specific clinical recommendations. This chapter provides background on the history of Institute of Medicine (IOM) involvement in issues related to CPG development, a discussion of the scope of work for this study, and a new definition for trustworthy CPGs.
BUILDING ON PREVIOUS IOM STUDIES
The IOM’s involvement in the clinical practice guideline arena began with the Omnibus Budget Reconciliation Act of 1989,1 which created the Agency for Health Care Policy and Research (AHCPR), now known as the Agency for Healthcare Research and Quality (AHRQ), to focus on outcomes and effectiveness research. The agency was created in part due to frustrations with “ceaselessly escalating healthcare costs, wide variations in medical practice patterns, evidence that some health services are of little or no value, and claims that various kinds of financial, educational, and organizational incentives can reduce inappropriate utilization” (IOM, 1990, p. 2). A relatively small portion of the agency’s budget was dedicated to creation and update of guidelines through a public–private enterprise. The agency contracted with the IOM for expert advice on launching this function, and in 1990, the IOM’s Committee to Advise the Public Health Service on Clinical Practice Guidelines issued its report, Clinical Practice Guidelines: Directions for a New Program. In 1990, the IOM defined CPGs as “systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances” (IOM, 1990, p. 8). The first IOM guideline committee recommended that CPGs include the following:
A statement about the strength of the scientific evidence and expert judgment
An explanation of the rationale for any deviations from the scientific evidence
Key stakeholders or their representatives as participants in the development process, either through membership on the development panel, formal testimony, or other forms of participation
Consideration of implementation and evaluation issues from the beginning of the development process
In 1990, before completion of the first report to AHCPR, the IOM formed a second expert committee to assess processes of guideline development and use, and recommended an improved conceptual and practical framework for future CPGs. Its report, Guidelines for Clinical Practice: From Development to Use, published in 1992, expanded on the 1990 report, adopted the eight guideline attributes proposed in 1990 (Box 1-1), and stressed the importance of both CPG content and the development process in guideline credibility and utility. The study also identified four problems with guideline efforts at that time:
Multiple organizations developing guidelines was an inefficient use of resources, resulting in varying methods of development and uneven quality. Also, selection of guideline topics was not coordinated.
The guideline development process often lacked methodological quality controls. Guideline users and review criteria designers had no means for assessing the quality of methods and products of various developers.
Most guidelines failed to meet the needs of multiple stakeholders, such as quality assurance, cost control, comparison with alternative practices, and medical liability reduction. Because evidence and rationale for guidelines typically were not offered, the educational value of the guidelines was limited.
Finally, because only a limited evaluation of the impact of clinical practice guidelines had been conducted, their effectiveness in improving quality of care was indeterminate (IOM, 1992).
During this period, concerns regarding the selection of guideline topics by AHCPR caused Congress to direct AHCPR to report on methods for setting priorities for guideline topics. This resulted in the IOM report, Setting Priorities for Clinical Practice Guidelines (IOM,
Recommended Attributes of CPGs
Validity: Practice guidelines are valid if, when followed, they lead to the health and cost outcomes projected for them, with other things being equal. A prospective assessment of validity will consider the projected health outcomes and costs of alternative courses of action, the relationship between the evidence and recommendations, the substance and quality of the scientific and clinical evidence cited, and the means used to evaluate the evidence.
Reliability/Reproducibility: Practice guidelines are reliable and reproducible: (1) if—given the same evidence and methods for guidelines development—another set of experts would produce essentially the same statements; and (2) if—given the same circumstances—the guidelines are interpreted and applied consistently by practitioners or other appropriate parties. A prospective assessment of reliability may consider the results of independent external reviews and pretests of guidelines.
Clinical Applicability: Practice guidelines should be as inclusive of appropriately defined patient populations as scientific and clinical evidence and expert judgment permit, and they should explicitly state the populations to which statements apply.
Clinical Flexibility: Practice guidelines should identify the specifically known or generally expected exceptions to their recommendations.
Clarity: Practice guidelines should use unambiguous language, define terms precisely, and use logical, easy-to-follow modes of presentation.
Multidisciplinary Process: Practice guidelines should be developed by a process that includes participation by representatives of key affected groups. Participation may include serving on panels that develop guidelines, providing evidence and viewpoints to the panels, and reviewing draft guidelines.
Scheduled Review: Practice guidelines should include statements about when they should be reviewed to determine whether revisions are warranted, given new clinical evidence or changing professional consensus.
Documentation: The procedures followed in developing guidelines, the participants involved, the evidence used, the assumptions and rationales accepted, and the analytic methods employed should be meticulously documented and described.
SOURCE: IOM (1990).
1995), which recommended principles and methods for prioritizing guideline topics. Although many of that report’s recommendations became moot when AHCPR discontinued its guideline development function, one conclusion of the report remains relevant: the agency should serve as a clearinghouse and disseminate guidelines developed by others.
In 2008, the current status of clinical practice guidelines was discussed in the IOM report, Knowing What Works in Health Care: A Roadmap for the Nation (IOM, 2008). The committee identified many of the quality problems cited by earlier committees, and in remediation of those long-standing problems, recommended a public–private “program” to create standards for guideline development and a process of documentation of adherence to standards.
Development standards that the committee identified included
objectivity in both the development process and the conclusions, resulting from a balanced panel;
transparency of the deliberative process in all aspects, including conflicts of interest among panelists or the sponsoring organization, the methods of data gathering and evidence assessment, and evaluation of strength of recommendations;
efficiency and timeliness when considering clinicians’ need for timely advice and minimization of duplication and resource wastage by guideline developers;
external review by outside experts of draft guidelines and independent party oversight of developer responses to reviewer comments;
continuous monitoring of relevant literature so that guidelines are reevaluated when important new evidence is produced and their currency is ensured; and
reduction of overlaps and gaps through voluntary efforts to increase consensus among various organizations developing guidelines on the same topics and to address topics that need guidelines (IOM, 2008).
Furthermore, Knowing What Works in Health Care (IOM, 2008) and the 2009 IOM report, Conflict of Interest in Medical Research, Education, and Practice (IOM, 2009), address conflicts of interest specific to guideline development. That issue will be discussed thoroughly in Chapter 4.
CURRENT STATE OF CLINICAL PRACTICE GUIDELINES
The development and application of CPGs have evolved dramatically in the 20 years since the IOM first became involved in this area. Specialty societies, disease advocacy groups, the federally supported U.S. Preventive Services Task Force, health plans, commercial companies, and other organizations in the United States and across the globe have produced thousands of guidelines of widely varying quality (Coates, 2010). This current IOM study was initiated in response to concerns about the state-of-the-art of the quality of CPGs.
Many current clinical practice guidelines suffer from limitations in the scientific evidence base and shortcomings in the guideline development process (IOM, 2008; Shaneyfelt and Centor, 2009). First, the evidence base is limited by an absence or paucity of studies on relevant topics, as well as variable quality of individual scientific studies and systematic reviews of the efficacy and effectiveness of medical and surgical procedures, treatments, drugs, and devices. Consequently, CPG recommendations often rely on low quality evidence or expert opinion. For example, in a study of practice bulletins from the American College of Obstetricians and Gynecologists, just 29 percent of recommendations were supported by “good and consistent scientific evidence” (Chauhan et al., 2006, p. 94).
Furthermore, evidence for clinical decision making regarding patients with rare conditions or subpopulations, such as patients with comorbidities and the socially and economically disadvantaged, is often nonexistent or inaccurate. Consequently, many guidelines either do not address or apply to a significant number of patients (Boyd, 2010). Also, few CPGs address many significant clinical conditions and decisions; consequently, tension persists between the need for guidance in a particular clinical area and the lack of an evidence base to allow the development of high-quality guidelines (IOM, 2008).
The quality of many CPGs is further diminished by the process used to develop some CPGs, including CPG development panel formation without sufficient attention to conflicts of interest (either financial or intellectual) or to balancing bias and including all relevant topical and methodological disciplines and stakeholders; poor coordination with systematic review groups, which does not permit tailoring of reviews to the specific CPG topic; and a lack of transparency concerning the derivation of recommendations (Coates, 2010; Jacobs, 2010; Koster, 2010). GDGs may cherry pick studies that support their positions, even when high quality SRs are available. Or, different reviewers of high-quality individual studies and system-
atic reviews on the same topic may reach inconsistent conclusions because of conflicting interests or research methodologies (Shrier et al., 2008). Ultimately, these evidence and process deficiencies have led to a plethora of often-conflicting recommendations produced by multiple organizations, and little means for potential CPG users to evaluate CPG quality and identify the most trustworthy CPGs (Coates, 2010).
Knowledge of which guidelines are based on high-quality scientific evidence and employ desired standards of development is important for clinicians and other guideline users. Knowing What Works in Health Care recommended further focus on these issues (IOM, 2008). This study builds on that earlier report.
The U.S. healthcare environment has evolved dramatically since the IOM’s early guideline reports. This committee was challenged to determine how that evolution has affected and should affect the CPG development process, exploring which prior recommendations are no longer relevant, where progress has been made, and what new problems have arisen.
The U.S. Congress, through the Medicare Improvements for Patients and Providers Act (MIPPA) of 2008,2 took steps to implement two recommendations of IOM’s Knowing What Works in Health Care (2008). It called on the Secretary of Health and Human Services (HHS) to contract with the IOM to “develop evidence-based, methodological standards for systematic reviews” (IOM, 2008, p. 108) and to develop standards for clinical practice guidelines. Specifically, Congress requested the IOM to undertake two studies: one “to identify the methodological standards for conducting systematic review of clinical effectiveness research on health and health care in order to ensure that organizations conducting such reviews have information on methods that are objective, scientifically valid, and consistent” and the other to focus on “the best methods used in developing clinical practice guidelines in order to ensure that organizations
developing such guidelines have information on approaches that are objective, scientifically valid, and consistent.”3
Request from the Department of Health and Human Services
The Secretary of HHS, through AHRQ, contracted with the IOM in July 2009 to conduct these studies. The IOM formed two committees with expertise across a broad range of relevant domains. The committees worked independently, but in coordination, to enhance the relevance of each study to the other and to minimize duplication. The Committee on Standards for Developing Trustworthy Clinical Practice Guidelines produced the report that follows, and the Committee on Standards for Systematic Reviews of Comparative Effectiveness Research produced a companion report.
IOM Committee Charge
Given the MIPPA legislative charge to the IOM mentioned previously, the IOM formed the Committee on Standards for Developing Trustworthy Clinical Practice Guidelines to concentrate on processes for the development of unbiased, scientifically valid, and trustworthy clinical practice guidelines. The committee examined existing standards for guideline development, assessing which might be suitable for adoption or adaptation in the United States. It considered methods for modifying CPGs for patients with multiple conditions, ways to reduce the number of overlapping guidelines and harmonize CPGs on the same topic, methods to distinguish and promote adoption of quality CPGs, and procedures for identifying strong recommendations that could become the basis for measures of quality of care. Throughout the study process, the committee was mindful of the great number and variety of guideline developers and users, and systematically elicited their input.
Relation to Committee on Standards for Systematic Reviews of Comparative Effectiveness Research
High-quality, effective guidelines depend on high-quality, comparative effectiveness research and systematic reviews (SRs) of
that evidence. A systematic review is a scientific investigation that focuses on a specific question and uses explicit, preplanned scientific methods to identify, select, assess, and summarize the findings of individual, relevant studies. It may or may not include a quantitative synthesis of the results from separate studies (meta-analysis). For SRs to support development of a CPG, the SRs need to address clinical questions critical to formulation of a recommendation about a particular clinical practice in a certain circumstance. Guideline Development Groups (GDGs) and SR teams may be highly interrelated, where a GDG might conduct its own SR, or an SR member might serve on a GDG panel. However, for the purpose of dividing the two MIPPA studies, all activities related to selecting, assessing and summarizing findings of individual studies fittingly are covered by the SR report.
As a result of the substantive association of the two mandated MIPPA studies, the IOM took the unusual step of ensuring close collaboration of the two committees. In addition to two study staffs working cooperatively, the IOM created an informal Coordinating Group, consisting of the chairs, vice chairs, and another member from each committee. The Coordinating Group met in person four times during the study course and held conference calls to share insight on interrelated issues and build on the progress of both committees. In the end, however, each committee authored its own report and recommendations.
Essential to the IOM committee formation process is the recruitment of relevant subject–matter experts and individuals with relevant experience, as well as inclusion of a diversity of opinion and sociodemographics. These tenets are consistent with guidance found within the standard-setting literature (Norcini and Shea, 1997). Once formed, the committee complied with established standards by requiring review of the relevant published (including grey) literature, solicitation of expert advice, and public comment through the activities described below.
The committee conducted an initial literature search in late 2009 of the following databases: MEDLINE, Embase, PsychInfo, Global Health, Web of Science, and several grey literature sources. Computerized literature searches were updated routinely and the table of
contents alerts from more than 20 journals were monitored throughout the committee’s term. Searches focused on articles with “Clinical Practice Guideline” or “Practice Guideline” in the topic heading, and any of the following terms in the title or abstract: consensus development, decision-making, development, evaluation, implementation, comorbidities, heterogeneity, policy, law, legal, implications, tool, taxonomy, reimbursement, measurement, performance, performance measures, consumer, public, grading, rating, issues, concerns, methods, quality, electronic medical record, computer decisions support system. The committee reviewed abstracts and complete texts for more than 2,500 peer-reviewed, published articles. For a complete description of the committee’s literature search, see Appendix E.
Gathering Data from the Web
Data from the National Guideline Clearinghouse and the Guidelines International Network, collected from the web, formed foundations for some committee analyses and provided considerable background on the current status of guidelines. The web also contributed information on other major activities in the CPG world.
Public Forum, January 11, 2010
Sixteen individuals and organizations responded to the committee’s broadly disseminated invitation to present public testimony at its second meeting. Respondents were asked to offer advice regarding the key issues the committee should address in deriving standards for clinical practice guideline development. Some speakers made statements on behalf of organizations; others made personal statements or offered expert counsel. These statements and testimonials, as well as speaker responses to committee queries, were valuable contributions to committee understanding of the CPG status quo, its limitations, and improvement options.
The committee asked representative organizations and individuals to address questions of interest across a series of public panel presentations and discussions. Panelists responded to the following questions: (1) What do you believe are the biggest challenges that clinical practice guidelines developers face today? (2) What are the biggest challenges that CPG users face? (3) What topics and/
or processes do you think the committee should consider in deriving quality standards for clinical practice guidelines? Twenty-four individuals participated and presented valuable information to the committee based on their experiences in developing and applying CPGs.
In addition to the January experience, members of the public communicated with the committee through its public website, staff, and direct contact with committee members. Those contributions enriched committee understanding. All submitted materials have been placed in the project’s Public Access File, which is accessed through the National Academy of Sciences website.
Staff commissioned several papers from technical experts to complement the committee’s expertise across the following topics:
“The State-of-the-Art of CPG Development and Best Practices,” by Paul Shekelle, Steve Woolf, Martin Eccles, Jeremy Grimshaw, and Holger Schünemann
“Legal and Administrative Alternatives for Enhancing CPG Quality and Adherence,” by Ronen Avraham and William Sage
“The Implementation and Evaluation of CPGs in the Present,” by Anne Sales, David Atkins, Mori Krantz, and Leif Solberg
“The Implementation and Evaluation of CPGs in an Electronic Future,” by Walter Stewart, J. B. Jones, Jon Darer, and Dean F. Sittig
These papers contributed to committee discussion and the evidentiary underpinnings of its report, although the perspectives and any implicit recommendations within each are solely those of the authors.
PURPOSE AND UPDATED DEFINITION OF CLINICAL PRACTICE GUIDELINES
Clinical Practice Guidelines are statements that include recommendations intended to optimize patient care. They are informed by a systematic review of evidence and an assessment of the ben-
efits and harms of alternative care options. To be trustworthy, guidelines should
be based on a systematic review of the existing evidence;
be developed by a knowledgeable, multidisciplinary panel of experts and representatives from key affected groups;
consider important patient subgroups and patient preferences, as appropriate;
be based on an explicit and transparent process that minimizes distortions, biases, and conflicts of interest;
provide a clear explanation of the logical relationships between alternative care options and health outcomes, and provide ratings of both the quality of evidence and the strength of recommendations; and
be reconsidered and revised as appropriate when important new evidence warrants modifications of recommendations.
The new definition provides clarification of the term “CPG,” which in the past was commonly used to describe clinical guidance derived from widely different development processes and statements taking various forms, including consensus statements, practice bulletins, expert advice, quality measures, and evidence-based recommendations. This study emphasizes characteristics that distinguish among methodologically rigorous, transparent, evidence-based guidelines (CPGs) and other forms of clinical guidance. Although the committee recognizes that other forms of clinical guidance may have value, addressing them was beyond the scope of this report. Furthermore, the committee is aware that for many clinical domains, there is little or no high-quality evidence; however, guideline developers may still produce trustworthy CPGs if they follow a rigorous and transparent process, as will be detailed. This study recommends standards for development of trustworthy guidelines in Chapters 4 and 5, and methods for identifying those guidelines in Chapter 7.
Trustworthy CPGs have the potential to reduce inappropriate practice variation, enhance translation of research into practice, and improve healthcare quality and safety. Patient and public involvement and trust in guideline development and their engagement in CPG implementation will enhance adoption of guidelines by all stakeholders. Optimally, CPGs may
guide clinician and patient decision making based on evidence regarding the care outcomes that particular practices are expected to produce;
provide a basis for measuring, evaluating, and improving provider performance and quality of care;
support appropriate resolution of malpractice claims by considering guideline recommendations as a standard of care;
contribute to the development of clinical decision support systems and other decision aids;
assist in educating patients, caregivers, and the media regarding best healthcare practices; and
aid policy makers in the allocation of healthcare resources.
Many of these activities are supported by CPGs, but they are not always informed by rigorous assessment of the science. Guidelines based on systematic literature review, consideration of the benefits and harms associated with particular recommendations, and unambiguous translation of this appraisal for clinicians and patients could enhance the above activities.
ORGANIZATION OF REPORT
Six chapters follow this introductory chapter. Chapter 2 presents background and an overview of CPGs, including a historical perspective on evidence-based medicine and the evolution of and a description of major participants in CPG development.
A critical appraisal of the current status of CPGs follows in Chapter 3, including assessment of the limitations that exist in the scientific evidence base and the CPG development process.
Chapters 4 and 5 discuss current best practices in the guideline development process and emphasize critical elements in the creation of trustworthy CPGs, including methods for systematic appraisal of the guideline development process. These chapters include proposed standards for guideline development.
Chapter 6 examines strategies for enhancing the likelihood of guideline adoption, and applying health information technology to promote guideline implementation. Additionally, it considers the legal implications of guidelines. This chapter includes recommendations for guideline implementation interventions.
Chapter 7 discusses national policy issues related to CPGs. The committee makes a recommendation for how to identify high-quality CPGs because even with quality standards, practitioners, patients, and other potential users can have difficulties recognizing which guidelines are unbiased, scientifically valid, and trustworthy. The chapter also includes recommendations concerning guideline
harmonization, assessing the reliability and validity of proposed standards, and process and impact evaluation.
Boyd, C. 2010. CPGs for people with multimorbidities. Paper presented at IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, November 11, 2010, Washington, DC.
Chauhan, S. P., V. Berghella, M. Sanderson, E. F. Magann, and J. C. Morrison. 2006. American College of Obstetricians and Gynecologists practice bulletins: An overview. American Journal of Obstetrics and Gynecology 194(6):1564–1572.
Coates, V. 2010. National Guidelines Clearinghouse NGC/ECRI Institute. Paper presented at the IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, January 11, 2010, Washington, DC.
IOM (Institute of Medicine). 1990. Clinical practice guidelines: Directions for a new program. Edited by M. J. Field and K. N. Lohr. Washington, DC: National Academy Press.
IOM. 1992. Guidelines for clinical practice: From development to use. Edited by M. J. Field and K. N. Lohr. Washington, DC: National Academy Press.
IOM. 1995. Setting priorities for clinical practice guidelines. Edited by M. J. Field. Washington, DC: National Academy Press.
IOM. 2008. Knowing what works in health care: A roadmap for the nation. Edited by J. Eden, B. Wheatley, B. McNeil, and H. Sox. Washington, DC: The National Academies Press.
IOM. 2009. Conflict of interest in medical research, education, and practice. Edited by B. Lo and M. J. Field. Washington, DC: The National Academies Press.
Jacobs, A. 2010. American College of Cardiology and American Heart Association. Presented at the IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, January 11, 2010, Washington, DC.
Koster, M. A. 2010. Technology assessment and guidelines unit: Kaiser Permanente Southern California. Paper presented at IOM Committee on Standards for Developing Trustworthy Clinical Practice Guidelines meeting, January 11, 2010, Washington, DC.
Norcini, J. J., and J. A. Shea. 1997. The credibility and comparability of standards. Applied Measurement in Education 10(1):39–59.
Shaneyfelt, T. M., and R. M. Centor. 2009. Reassessment of clinical practice guidelines: Go gently into that good night. JAMA 301(8):868–869.
Shrier, I., J.-F. Boivin, R. Platt, R. Steele, J. Brophy, F. Carnevale, M. Eisenberg, A. Furlan, R. Kakuma, M. Macdonald, L. Pilote, and M. Rossignol. 2008. The interpretation of systematic reviews with meta-analyses: An objective or subjective process? BMC Medical Informatics and Decision Making 8(1):19.