5
Developing Trusted Clinical Practice Guidelines

Abstract: This chapter reviews the current landscape of clinical practice guideline development in the United States, presents the committee recommendations for creating trusted clinical practice guidelines, and describes key challenges in promoting the development and adoption of high-quality guidelines under the aegis of a proposed national clinical effectiveness assessment program (“the Program”). Under the status quo, the processes underlying guideline development are often vulnerable to bias and conflict of interest. Overall, the quality of clinical practice guidelines is often poor. The committee recommends that the Program establish standards for guideline development but also promote voluntary adoption of Program standards by guideline developers. The standards must address the composition of guidelines panels to ensure that guidelines are created by a diversity and balance of competing interests with minimal bias. The standards should also promote objectivity, transparency, and efficiency in guideline development and clear, standardized reporting of clinical recommendations. Groups developing clinical practice guidelines should document their adherence to Program standards and make this documentation publicly available. Individuals and organizations that utilize guideline information would then be in a better position to assess guideline quality and utilize only those guidelines that meet the Program’s standards.

The development of clinical practice guidelines for use by practitioners, payers, patients, and others is a key strategy in promoting the use of highly effective clinical services. When they are used, rigorously developed guidelines have the potential to reduce undesirable practice variation, reduce the use of services that are of minimal or questionable value, increase the



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 121
5 Developing Trusted Clinical Practice Guidelines Abstract: This chapter reiews the current landscape of clinical practice guideline deelopment in the United States, presents the committee recom- mendations for creating trusted clinical practice guidelines, and describes key challenges in promoting the deelopment and adoption of high-quality guidelines under the aegis of a proposed national clinical effectieness assessment program (“the Program”). Under the status quo, the pro- cesses underlying guideline deelopment are often ulnerable to bias and conflict of interest. Oerall, the quality of clinical practice guidelines is often poor. The committee recommends that the Program establish stan- dards for guideline deelopment but also promote oluntary adoption of Program standards by guideline deelopers. The standards must address the composition of guidelines panels to ensure that guidelines are created by a diersity and balance of competing interests with minimal bias. The standards should also promote objectiity, transparency, and efficiency in guideline deelopment and clear, standardized reporting of clinical recom- mendations. Groups deeloping clinical practice guidelines should docu- ment their adherence to Program standards and make this documentation publicly aailable. Indiiduals and organizations that utilize guideline information would then be in a better position to assess guideline quality and utilize only those guidelines that meet the Program’s standards. The development of clinical practice guidelines for use by practitioners, payers, patients, and others is a key strategy in promoting the use of highly effective clinical services. When they are used, rigorously developed guide- lines have the potential to reduce undesirable practice variation, reduce the use of services that are of minimal or questionable value, increase the 

OCR for page 121
 KNOWING WHAT WORKS IN HEALTH CARE utilization of services that are effective but underused, and target services to those populations most likely to benefit (Grimshaw and Russell, 1993). Underlying the effort to produce evidence-based guidelines is a pressing need for trusted information on clinical effectiveness. As described earlier, in recent years there has been a substantial increase in the number of treatment alternatives available to providers and patients, as well as in the volume of studies describing the effectiveness (or ineffectiveness) of those options. This body of evidence has become complex and difficult to man- age for most providers. As a result, guidelines have become a key tool for summarizing the available literature and placing it in a format accessible to physicians (Druss and Marcus, 2005). This chapter has three principal objectives: (1) to review the current landscape of clinical practice guideline development in the United States, (2) to present the committee’s recommendations for creating trusted clini- cal practice guidelines, and (3) to highlight key challenges in promoting the development and adoption of high-quality guidelines under the aegis of a proposed national clinical effectiveness assessment program (“the Program”). BACKGROUND Clinical practice guidelines attempt to define practices that meet the needs of most patients under most circumstances. They do not attempt to supplant the independent judgment of clinicians in responding to particular clinical situations. Ideally, the specific clinical recommendations that are contained within practice guidelines have been systematically developed by panels of experts who have access to the available evidence, an understand- ing of the clinical problem and the relevant research methods, and sufficient time to absorb the information and make considered judgments (GRADE Working Group, 2004). These panels are expected to be objective and to produce recommendations that are unbiased, up-to-date, and free from conflict of interest. Groups that measure provider performance frequently use adherence to clinical practice guidelines as a basis upon which to evaluate the qual- ity of care, and many payers are now moving toward the use of pay-for- performance strategies that establish differential payments on the basis of adherence to quality measures. In addition to performance-based payment, with the increased use of health information technology and direct decision support at the point of care, guidelines are likely to become increasingly influential in clinical practice (O’Malley et al., 2007). Perhaps the earliest guidelines produced in the United States were the American Academy of Pediatrics’ Redbook of Infectious Diseases, pub-

OCR for page 121
 DEVELOPING TRUSTED GUIDELINES lished in the 1930s (American Academy of Pediatrics, 2007). The groups that were among the first to use systematic reviews to support clinical rec- ommendations were the Canadian Task Force on the Periodic Health Ex- amination and the U.S. Preventive Services Task Force (USPSTF) (Fielding and Briss, 2006). The Canadian task force was established in 1976 to make recommendations about the inclusion of preventive services in the periodic health examination; the USPSTF was established in 1984 and also provided prevention-related recommendations for health profession- als (Woolf and Atkins, 2001). The American College of Physicians began to publish explicit recommendations based on systematic reviews in 1981 (Eddy, 2005). In 1989, Congress established the Agency for Health Care Policy and Research (AHCPR) and tasked it with developing clinical practice guide- lines, among its other responsibilities. The Institute of Medicine (IOM) noted that this effort was part of a cultural shift: a move away from an unexamined reliance on professional judgment and toward more structured support and accountability for these judgments (1990). Before the move toward evidence-based practice, medical textbooks and articles were filled with thousands of statements and care recommendations that were based solely on the belief of the author or at best a consensus of experts (Eddy, 2005). Evidence-based guidelines initiatives aim to base recommendations on empirical evidence. Relationship to Systematic Reviews Clinical guidelines go beyond systematic reviews by recommending what should and should not be done in specific clinical circumstances. Although systematic reviews produce findings about clinical effectiveness, transforming that evidence into specific care recommendations is often chal- lenging. Given the gaps in information that frequently exist and the variable quality of the information that is available, a key component of guideline development is the establishment of a link between the strength of the clini- cal recommendation and the quality of the underlying evidence. Guyatt and colleagues (2006a) argue that one of the first criteria of an effective guideline development process is having two separate grading systems: one for the quality of the evidence and another for the recommen- dations themselves. The quality of evidence grade reflects the level of con- fidence that, if the recommendation is followed, the anticipated outcomes will occur. The strength of the recommendation takes into account the bal- ance of the benefits and the harms that are associated with the intervention and the guideline authors’ views about the importance of adhering to the recommendation.

OCR for page 121
 KNOWING WHAT WORKS IN HEALTH CARE Resource Requirements Guideline production requires a significant commitment by professional societies and others who perform the work, especially if they conduct high- quality systematic reviews themselves. Locating and analyzing all of the available evidence requires substantial skills, resources, and time, and pro- fessional groups often lack what is needed to do a credible job (Woolf et al., 1999). The resource demands of conducting a rigorous systematic review often leads guideline developers to revert to short-cuts or processes centered on expert opinion (Browman, 2001). Moreover, a substantial investment in evidence gathering does not guarantee a good return on evidence available to address a question (Ricci et al., 2006). In fact, guideline developers often must reckon with research that is not sufficiently rigorous, yields conflicting results, or does not exist (Cook and Giacomini, 1999). This also contributes to pressures to rely more heavily on professional opinion. Guideline Developers1 As described in Chapter 2, many groups produce clinical practice guidelines and recommendations. The National Guideline Clearinghouse (NGC) currently includes guidelines from approximately 360 organiza- tions (NGC, 2007c). Medical professional societies are the most common sponsors of guidelines. In addition, patient advocacy groups, payers, gov- ernment agencies, and others in the United States may conduct systematic reviews and develop clinical recommendations. Organizations in other countries also produce guidelines that are available in the United States, including the National Institute for Health and Clinical Excellence (NICE), the Scottish Intercollegiate Guidelines Network (SIGN), and organizations in Australia and Canada. In the United States, the NGC provides free access to guidelines pro- duced across a range of clinical areas. The NGC included approximately 650 guidelines in 1999 (O’Connor, 2005) and has grown to nearly 2,200 guidelines today (NGC, 2007b). The website now receives an estimated 1.3 million visits per month.2 For a guideline to be included on the website, guideline producers are required to demonstrate that they performed a systematic literature search and that they developed, reviewed, or revised the guideline within the last five years (NGC, 2007a). By meeting NGC standards and being admitted to the website, guideline developers are able to improve the dissemination of their products. 1 See Chapter 2 for background on organizations that develop or use clinical practice guidelines. 2 Personal communication, J. Slutsky, Agency for Healthcare Research and Quality, Septem- ber 4, 2007.

OCR for page 121
 DEVELOPING TRUSTED GUIDELINES The USPSTF, having been in existence for over 20 years, serves as a model of recommendation development in the United States, especially because of its adherence to detailed methodologies and the restrictions it places on conflicts of interest. Clinicians, health plans, and payers have come to rely on the regular reports from the task force to update their prac- tice, payment, or coverage policies regarding clinical preventive services. CURRENT LANDSCAPE Quality of Guidelines The IOM Committee on Clinical Practice Guidelines defined high- quality guidelines as having a number of attributes, including validity, reliability, reproducibility, clinical applicability and flexibility, clarity, devel- opment through a multidisciplinary process, scheduled reviews, and docu- mentation (IOM, 1992). Over time there have been noted improvements in the capacities of some clinical and professional organizations to develop robust, evidence-based guidelines (Jackson and Feder, 1998). Nevertheless, the overall quality of clinical practice guidelines is highly variable, and in fact, the quality is often very poor (Shaneyfelt et al., 1999). Shaneyfelt and colleagues (1999) assessed the quality of 279 guidelines produced over the period of 1985 to 1997 and assessed their quality against a set of 25 stan- dards. The investigators found that the mean number of quality standards satisfied over that period was 11 (43 percent). For example, less than 10 percent of the guidelines described formal methods of combining scientific evidence and expert opinion. The investigators also evaluated the guide- lines in accordance to their specification of purpose (75 percent compli- ance), definition of the patient population involved (46 percent), pertinent health outcomes (40 percent), method of external review (32 percent), and whether an expiration date or scheduled update was included (11 percent). Overall, the investigation found significant improvement over time, but each guideline still only met 50 percent of the standards, on average, in 1997 (Shaneyfelt et al., 1999). For some, this variability in guideline qual- ity called for greater transparency in guideline reporting and more rigorous peer review (Cook and Giacomini, 1999). An evaluation of 86 guidelines developed in 11 countries (which did not include the United States) concluded that the guidelines produced by government-funded agencies and established guideline development pro- grams were of higher quality than guidelines produced by specialty societies (Burgers, 2003). This finding was consistent with the conclusions of Grilli and colleagues (2000), and also with Hasenfeld and Shekelle (2003), who found that the 17 guidelines produced by the AHCPR from 1990 to 1996 were of a substantially higher quality than those subsequently produced

OCR for page 121
6 KNOWING WHAT WORKS IN HEALTH CARE by other groups. The authors postulated that the higher-quality scores of guidelines developed by government agencies reflect the fact that the production of high-quality guidelines requires substantial and sufficient resources and that government agencies have more resources available to do the work. Smaller professional organizations often lack the internal resources, in- cluding staff capacity and expertise, required to produce guidelines. This is especially true when the organization produces both the systematic reviews and the guideline recommendations, two tasks requiring different skill sets. Even larger professional organizations can face resource constraints in this area. Some have suggested that, given these resource constraints, govern- ment is in the best position to produce clinical practice guidelines (Burgers, 2003; Hasenfeld and Shekelle, 2003). Many of the criticisms directed at the U.S. system of guideline produc- tion in 1990 still apply today (IOM, 1990). These criticisms focused on conflicting clinical recommendations; failure to address certain topics; and incomplete public disclosure of the evidence surveyed, methods used, com- position of the panel, and conflicts of interest. In addition, it remains true that, aside from the role that AHRQ plays in populating the NGC website, no independent entity exists in the United States to certify guideline qual- ity or to develop national standards regulating the content or methods of guideline developers. The 1990 IOM report Clinical Practice Guidelines: Directions for a New Program sought to encourage more standardization and consistency in guideline development, and although the quality of clinical practice guidelines has generally improved since then, substantial inconsistencies in the methodologies and reporting language used still exist (Guyatt et al., 2006b; Shiffman et al., 2003). Quality of Information The translation of systematic reviews into practice recommendations is not straightforward. The same information can be interpreted in different ways by different panelists, resulting in the provision of different guidance (Burgers and van Everdingen, 2004). Often, even when there is substantial consensus about what the scientific evidence says, there are disagreements about what the evidence means for clinical practice. Conclusions about clinical effectiveness can vary widely as a result of conflicting viewpoints, such as which outcomes are the most important and which course of action is appropriate given that the evidence is imperfect (Atkins et al., 2005b). This section highlights strategies that guideline developers have used to improve the reliability and trustworthiness of the information that they provide. It also examines methodological approaches, and how groups

OCR for page 121
7 DEVELOPING TRUSTED GUIDELINES have sought to ensure objectivity in their procedures. Finally, this section examines assessments of the overall quality of the recommendations cur- rently being made. Methodological Rigor Although there have been recent efforts to standardize approaches to guideline development, it is not yet possible to say that guideline develop- ment is based on a scientifically validated process. The key challenges stem from the fact that guideline development frequently forces organizations to go beyond what is known from a scientific point of view to make practical recommendations for use in everyday practice. Two examples of such chal- lenges are the approaches to limitations in the evidence base and subjective assessments of the net benefit. Limitations of the evidence base The evidence base that supports clini- cal practice guidelines is often quite limited and guideline developers must often wrestle with what to do when “the irresistible force of the need to offer clinical advice meets with the immovable object of flawed evidence” (Ricci et al., 2006, p. 229). They must consider the best way to address the trade-off between rigor and pragmatism (Browman, 2001), and between adherence to evidence and broader clinical utility (Perlin and Kupersmith, 2007; Stewart et al., 2007). As a result, a consensus of expert opinion among clinical and methodologist panelists often fills in the gaps between areas supported by scientific evidence. In making their treatment decisions, practicing clinicians might want to place less reliance on guidelines that are based primarily on expert opin- ion rather than empirical evidence. Often, however, it is not clear which parts of guidelines are evidence-based, and which are not. Many times, when groups incorporate expert opinion, they do not do so in a standard- ized way (Thomson et al., 1998). The methods for incorporating opinion into guidelines is less well-developed than the methods for incorporating research results and often they are not made explicit. Disclosing the role of expert opinion is especially important when the data are sparse (Cook and Giacomini, 1999). When combining a review of research data with practice recommenda- tions, guidelines often do not identify an explicit search strategy used, do not have defined inclusion criteria for selecting eligible studies, and do not assess the findings against consistent methodological standards (Miller and Petrie, 2000). Guidelines such as these often reflect a subjective assessment of the consistency, clinical relevance, and external validity of the available evidence (Ricci et al., 2006).

OCR for page 121
 KNOWING WHAT WORKS IN HEALTH CARE Subjective assessments of net benefit The development of clinical recom- mendations should involve a summary of the harms and benefits of a par- ticular service or intervention. The strength of the recommendation reflects this assessment. Table 5-1 illustrates how the USPSTF addresses net benefit in its strength of recommendation categories. Although some bodies of evidence show a high degree of benefit and few harms, in many cases the benefit and harm seem to be more closely balanced and it is much more difficult to justify a strong recommendation. In situations in which the evidence is of poor quality, it may be difficult to come to an agreement about the balance between the benefits and harms (Atkins et al., 2005a). However, even when the data and the evidence are solid, value judgments come into play when making these assessments (Woolf et al., 1999). Rendering judgments about evidence and the subsequent development of appropriate recommendations are complex and the use of some sub- TABLE 5-1 USPSTF Strength of Recommendations Grade Definition Suggestions for Practice A The USPSTF recommends the service. There Offer or provide this service. is high certainty that the net benefit is substantial. B The USPSTF recommends the service. There Offer or provide this service. is high certainty that the net benefit is moderate or there is moderate certainty that the net benefit is moderate to substantial. C The USPSTF recommends against routinely Offer or provide this service only providing the service. There may be if other considerations support the considerations that support providing the offering or providing the service in service in an individual patient. There is at an individual patient. least moderate certainty that the net benefit is small. D The USPSTF recommends against the Discourage the use of this service. service. There is moderate or high certainty that the service has no net benefit or that the harms outweigh the benefits. I The USPSTF concludes that the current Read the clinical considerations evidence is insufficient to assess the balance section of USPSTF Recommenda- of benefits and harms of the service. tion Statement. If the service Evidence is lacking, of poor quality, or is offered, patients should conflicting, and the balance of benefits and understand the uncertainty about harms cannot be determined. the balance of benefits and harms. SOURCE: AHRQ (2007).

OCR for page 121
9 DEVELOPING TRUSTED GUIDELINES jectivity in the process is unavoidable (Atkins et al., 2005a). Inconsistent recommendations from different practice guideline development commit- tees often reflect differences in values and tolerances for potential harm. People may perceive the importance of a specific health outcome differently and thus may differ on the point at which the likely benefits of a treatment outweigh the likely harms (IOM, 1990). Guyatt and colleagues (2006a) have indicated that when value or preference judgments are particularly important to the recommendation, guideline development panels should describe the key values attached to these outcomes, and how they influenced the content or strength of the recommendation. Guideline development panels often do not include patients or con- sumers as members, and they may not seek patient input when weighing particular health states (Guyatt et al., 2006a). However, some patient and consumer advocacy groups are taking a more prominent role in the evidence-based health care field, and the concept of shared decision mak- ing has begun to take hold. The use of decision aids is bringing objective information about benefits and harms directly to patients so that they and their physicians can make informed and appropriate decisions (Weinstein et al., 2007). Shared decision making is often the best approach for elective procedures, for example, in deciding whether an arthritic knee hurts enough to justify the risks of knee replacement. Addressing Bias Patients, clinicians, payers, purchasers, and many others rely on having clinical recommendations that are produced in an objective manner. Groups making clinical recommendations have attempted to ensure objectivity in a variety of ways. The following sections examine in detail measures that promote the formation of panels with a balanced composition of members and freedom from conflict of interest. Panel composition To protect against a bias in perspective, it is impor- tant that guideline development panels include individuals from a range of relevant professional groups. Panels that are composed of members from a single specialty are likely to reach conclusions different from those of panels with multispecialty representation, even when both panels are presented with the same set of evidence (Shekelle et al., 1999). Kahan and colleagues (1996) examined six surgical procedures and found that between 10 and 42 percent of all cases that were deemed appropriate by specialists who performed the procedure were deemed less than appropri- ate by primary care providers. Murphy and colleagues (1998) found that members of a specialty are more likely to advocate the use of techniques that involve their specialty. Possible explanations for these systematic dif-

OCR for page 121
0 KNOWING WHAT WORKS IN HEALTH CARE ferences in judgment include superior knowledge, economic self-interest, and inadvertent cognitive bias (Kahan et al., 1996). To address some of the problems noted, many researchers and others have encouraged the use of balanced, multidisciplinary panels that include representatives from different clinical specialties as well as methodologists and patients. For example, the RAND Corporation-University of California at Los Angeles appropriateness method employs a nine-member multidis- ciplinary panel to assess the appropriateness of specific interventions for specific indications. These panels include specialists who perform the pro- cedure in question, specialists who do not perform the procedure but have practices in related areas (e.g., noninvasive cardiologists for a coronary arteriography panel), and primary care providers (e.g., internists). Shekelle and colleagues (1999) have argued that guideline development panels with multidisciplinary representation may produce more reliable re- sults because such a structure can balance the biases of the various individu- als on the panel. The IOM Committee to Advise the Public Health Service on Clinical Practice Guidelines found that multidisciplinary participation (1) increases the probability that all relevant scientific evidence will be lo- cated and critically evaluated, (2) increases the chances that the committee will address practical problems relating to application of the guidelines, and (3) helps build support among the groups for whom the guideline is intended (IOM, 1990). However, the specific make-up of guideline development panels often remains unaffected by these findings. Grilli and colleagues (2000) exam- ined the guidelines produced by specialty societies and found that only 28 percent mentioned the inclusion of a panelist of a different specialty. Others most often invited to participate included epidemiologists or meth- odologists, primary care physicians, health administrators, and patients or consumer representatives (Grilli et al., 2000). Another study found that only 26 percent of the guidelines examined provided a description of the participants included in the guideline development process along with their areas of expertise (Shaneyfelt et al., 1999). Conflicts of interest3 Actual and perceived conflicts of interest are a ma- jor source of concern for stakeholders seeking objective assessments about clinical effectiveness. These conflicts can occur when decision makers— including individual clinicians and clinicians serving on guideline develop- ment panels—have a personal stake in the outcome of the decision, such 3 A recently formed IOM Committee on Conflict of Interest in Medical Research, Education, and Practice is studying conflicts of interest in the conduct of medical research, development of practice guidelines, and patient care. A final report is expected in 2009.

OCR for page 121
 DEVELOPING TRUSTED GUIDELINES as a potential financial gain or the loss of intellectual standing (i.e., reputa- tion). In the past several years, these types of conflicts of interest among decision makers have come under increasing scrutiny. Because the interpre- tation of scientific evidence and its translation into clinical decisions often involve the use of a substantial amount of judgment, conflicts of interest add to concerns that bias may be injected into the process. One recent survey of physicians found that 94 percent have some type of relationship with the pharmaceutical industry (Campbell et al., 2007). More than one-third reported that they had received reimbursement for costs associated with professional meetings or continuing medical educa- tion; and 28 percent reported that they had received payments for consult- ing, lecturing, or enrolling patients in clinical trials. For many, there is a persistent concern that these relationships have an undue impact on treat- ment decisions, creating risks for individual patients, and undermining the integrity of the medical profession (Tonelli, 2007). For example, a recent analysis conducted by the New York Times concluded that from 1997 through 2005, Minnesota physicians who received the most money from makers of atypical antipsychotic drugs were more likely to prescribe the drugs to children (Harris et al., 2007). On average, Minnesota psychiatrists who received at least $5,000 from the makers of atypical antipsychotic drugs from 2000 to 2005 wrote three times as many atypical prescriptions for children as psychiatrists who received less or no money, according to the authors. In addition, investigators’ public positions on drug safety can be asso- ciated with their financial relationships with pharmaceutical manufactur- ers. For example, investigators who supported the use of calcium-channel antagonists were significantly more likely to have a financial relationship with manufacturers, as compared to those who took a neutral or critical position (90, 60, and 37 percent respectively) (Stelfox, 1998). Conflict of interest is a problem for guideline developers as well. In a survey of clinical practice guideline authors, 59 percent indicated that they had a relationship with companies whose products were included in the guideline that they authored (Choudhry et al., 2002). Aside from equity interest in the companies being evaluated, other types of conflicts include receipt of royalties, speakers fees, consulting fees, and research funding for unrelated products, in addition to various types of intellectual conflicts of interest. The public and Congress have become concerned about perceptions of conflict of interest at both the U.S. Food and Drug Administration (FDA) and the National Institutes of Health (NIH). Both agencies have recently promulgated new guidelines that limit the amount of money that external advisory panel members can receive from companies whose products or

OCR for page 121
 KNOWING WHAT WORKS IN HEALTH CARE sideration of the benefits anticipated if the recommendation is followed and consideration of the potential harms and costs of such adherence. Strong recommendations are made when the benefits clearly exceed the harms or when the harms clearly exceed the benefits. On the contrary, lower-level recommendations (sometimes referred to as clinical “options”) are made when the balance of the anticipated benefits compared with the anticipated harms and costs is less clear-cut or is essentially equivalent. As mentioned above, the statement of the strength of the recommenda- tion communicates an expectation regarding adherence. Whereas clinicians should be expected to follow strong recommendations unless a clear and compelling rationale for not doing so is present, patient preferences should also have a substantial role in influencing clinical decision making, and may even sometimes choose not to proceed with an intervention that has been found to be strongly beneficial. In addition, pay-for-performance measures should be built from strong recommendations and not clinical options. The quality of the evidence (based on factors related to minimizing bias such as study design, consistency, and directness of the evidence) helps determine the confidence that should be placed in the balance equa- tion. The guideline developer can confidently and strongly recommend an intervention when it is found effective and with minimal adverse effects in multiple, well-designed studies. Under such circumstances, one can be confident of the importance of adherence to the guideline. On the other hand, when high-quality evidence indicates both important benefits and important harms, the recommendation should be made accordingly. Strong recommendations should not be created when the evidence is poor. Nor should high-quality evidence on effectiveness automatically lead to a strong recommendation; potential harm must also be considered. The committee believes that a common language that describes both the quality of the underlying evidence and the strength of recommendations is an important tool for promoting greater consistency among clinical practice guideline developers. This common language will reduce the requirements placed on end users in sorting through and navigating all the various terms, symbols, and expressions that currently exist. An important task for the Program will be to facilitate the process of achieving a common language. Minimizing Bias Due to Conflicts of Interest Organizations that produce guidelines convene panels of experts to assess the available evidence and develop clinical recommendations. To produce objective, well-balanced, reliable clinical guidance, guideline de- velopers must address several basic structural issues that, if they are not managed properly, can create the perception—if not the reality—of bias.

OCR for page 121
 DEVELOPING TRUSTED GUIDELINES These can diminish the value of the clinical information provided and can undermine public confidence in the guidelines overall. Unbiased Information Bias can enter into the guideline development process in a number of ways, as illustrated in Box 5-2. These biases can occur at the individual, panel, and organizational levels. Groups and organizations that develop clinical practice guidelines should address each of these to ensure that the end users view their guidelines as credible and trustworthy. The committee identified and compared three approaches for handling conflicts of interest. The first is a permissive approach by which guideline producers are able to address these issues as they fit on a case-by-case ba- sis. The current system is relatively permissive in how it handles potential conflicts, although some measures are in place to limit influence, such as restrictions on money or other gifts received from commercial sources that are placed on external advisors by the FDA and the NIH. Investigative reporting conducted by the print media has called into question guideline producers that have received industry financing, and this has raised public awareness. In addition, financial disclosures have increasingly been em- ployed to address conflict of interest, and while this is an important step, its effectiveness may be limited. Moreover, guideline developers do not al- ways disclose potential conflicts and biases that may exist at the individual, panel, or organizational level. BOX 5-2 Potential Sources of Bias in Clinical Practice Guidelines • Panelists have material interests in the recommendations, e.g., stock owner- ship, royalties, or other returns. • Panelists have indirect financial interests, e.g., they could be paid for the health service under review or receive honoraria for discussing it. • The panel is primarily made up of individuals from one specialty with only lim- ited participation by other types of providers, patients, plans, methodologists, etc. • Panelists have intellectual biases, e.g., prior research, strongly held opinions, or professional specialty that might compromise one’s objectivity or bring it into question. • The organization producing the guideline receives funding from companies with a material interest in the recommendations. • The panel does not allow participation from members of the public. • Panels do not allow participation from members of the public.

OCR for page 121
 KNOWING WHAT WORKS IN HEALTH CARE The second approach is to promote a system that is completely free from conflict of interest. Under such a system, panelists who had received any remuneration from affected industries would be disqualified from serv- ing on guideline development panels. The restriction would also be applied to physicians who receive fee-for-service compensation for doing the proce- dures being reviewed. Presumably salaried physicians or physicians who do not provide the procedure themselves would qualify as panelists. Intellectual conflicts of interest, such as those relating to professional reputation, would also have to be addressed. At the organizational level, groups sponsored or convened by potentially affected manufacturers (including professional societies, consumer advocacy groups, and others) would not be recognized as appropriate sponsors of clinical practice guidelines. Given the extent to which these types of conflicts exist in the current environment, the second “pure” model seems largely impractical. Guideline producers require panelists who have expertise and, in today’s environment, these experts typically have conflicts of interest. Therefore, the committee identified a third, more pragmatic approach that recognizes that conflicts of interest are likely to persist for members of most guideline production panels, but that a number of steps can be taken to manage these conflicts. These steps include placing limits on the financial remuneration that panel- ists and organizations receive, balancing the composition of panels, and establishing a transparent process that includes public participation. Table 5-3 illustrates measures that might be taken under each of the TABLE 5-3 Measures to Address Conflicts of Interest Approach Panelist Panel Organization Permissive • Discretion of guideline • Discretion of • Discretion of producer guideline producer guideline producer Pure • No remuneration from • Balanced panel • Guideline-producing affected manufacturers participation organization • Individuals with conflicts (various provider/ receives no restricted to brief panel stakeholder types, payments presentations including plans, from affected patients, and others) manufacturers Pragmatic • Limited remuneration from • Balanced panel • Guideline-producing affected manufacturers participation organization • Disclosure of conflicts (various provider/ receives limited • Publication of disclosed stakeholder types, payments conflicts (transparency) including plans, from affected • Limited voting rights for patients, and others) manufacturers members with conflicts • Open (public) meetings

OCR for page 121
 DEVELOPING TRUSTED GUIDELINES three approaches (permissive, pure, and pragmatic), across the various levels of the guideline production process (individual panelist, entire panel, and sponsoring organization). The permissive and the pure categories are intended to represent the extreme ends of the spectrum. The committee maintains that the pragmatic approach is the most ap- propriate course of action, given that the current, more permissive approach provides too few safeguards against conflicts of interest and bias and that the “pure” approach, although it is theoretically desirable, is impractical and would strip too much expertise from the guideline development pro- cess. However, the Program, as detailed in Chapter 6, should develop (or endorse) strict standards to protect against bias and ensure that clinical practice guideline producers are adhering to these standards. In particular, the committee identified the following measures as a means of improving the quality of the information provided by guideline developers: Recommendation: To minimize bias due to conflicts of interest, panels should include a balance of competing interests and diverse stakehold- ers, publish conflict of interest disclosures, and prohibit voting by members with material conflicts. Indiidual Leel At the individual level, guideline developers should vet potential pan- elists for financial or intellectual biases and should have panelists disclose all financial relationships with commercial companies. They should also reveal any relevant positions that the panelists have advocated publicly. The Program should establish parameters to indicate when personal conflicts of interest are significant enough to warrant disqualification from panel participation or voting. Panel Leel At the panel level, groups should be multidisciplinary and should in- clude topic experts, generalists, consumers, payers (e.g., health plans), and others. For example, recommendations on care for children with Attention Deficit Hyperactivity Disorder should be developed with representation from pediatrics, family practice nursing, psychiatry and behavioral medi- cine, educators, parent organizations, and payers. Organizations should seek to build a broad consensus about the treatment alternatives that fall within the scope of the review. Purchasers and health plan representatives should be included on guideline development panels to moderate any clini- cal or manufacturer bias in favor of greater service utilization. In general,

OCR for page 121
6 KNOWING WHAT WORKS IN HEALTH CARE the panel should represent a balance of competing interests to the greatest extent possible. Organizational Leel At the organizational level, guideline developers should disclose the monies they have received from affected manufacturers, either related to the subject of guideline development or any general contributions. The standards generated by the Program should establish the levels of commer- cial involvement or support at which organizations should be considered insufficiently protected from commercial bias. Adherence to Standards The end users of clinical guidelines and recommendations—physicians, performance measurement groups, health plans, purchasers, patients, policy makers, and others—would benefit from a rigorous set of development standards and a common reporting language that would improve the qual- ity and usability of the guidelines. However, achieving this objective will be difficult. Various groups have developed distinct ways to speak about and assess evidence, and to grade the strength of their recommendations. More- over, the rigors of their processes are highly variable; and many guideline developers do not have the resources or the ability to meet a set of structure, process, and product standards that are externally imposed. Nevertheless, ensuring the quality and the usability of the information provided in clinical practice guidelines is vital to the performance of the health system and there is a need to promote compliance with these new guideline standards. Given the impracticality of centralizing the guideline development process in the U.S. government, the committee believes that building on the current pluralistic system is the most appropriate course of action. The committee proposes that the users of guidelines serve as the primary arbiters of guideline quality, with guideline developers volun- tarily providing documentation that will allow end users to make informed judgments. Recommendation: Providers, public and private payers, purchasers, accrediting organizations, performance measurement groups, patients, consumers, and others should preferentially use clinical recommenda- tions developed according to the Program standards. The committee envisions that the Program will develop a common reporting mechanism that will enable guideline developers to describe the features of their process. Through a standardized survey instrument, guide-

OCR for page 121
7 DEVELOPING TRUSTED GUIDELINES line developers will be asked to report on their methodologies and their adherence to the common standards. Guideline-producing organizations, spurred by end users, will want to include this documentation as part of the guideline itself. In addition, the information will be uploaded to a public web page to enable end users to view the extent to which organizations producing guidelines (and the guidelines themselves) adhere to the common standards. If guideline users preferentially adopt guidelines that are developed according to Program standards, guideline producers will be motivated to adhere to those stan- dards and to provide documentation about their processes. The availability of this information will enable the end users of the guideline material to become more informed about the quality of the infor- mation they receive. Although the documentation that guideline develop- ers provide may not be complete, gaps in that information may serve as a red flag for the end users. In addition, through increased transparency and openness in the guideline development process, the accuracy of the informa- tion reported will be more easily verified. The Program may want to consider instituting a certification or accredi- tation process to assure that specific guidelines or organizations developing them adhere to specific standards. Such an accreditation or certification process would allow for continued decentralized guideline production. The end users of guideline information then become the group that holds the guideline developers accountable for their work products. The vi- sion of the committee is that performance measurement groups will primar- ily rely on the highest-quality information—as indicated by the standards reporting document—and establish measures that will encourage physicians to comply with these high-quality recommendations. Health plans and purchasers should also be selective in choosing only guidelines that adhere to standards and base performance-based programs only on these types of guidelines. Accreditation groups (e.g., the National Committee for Quality Assurance, the Utilization Review Accreditation Commission, and the Joint Commission) should assess the extent to which the groups that they moni- tor are relying on the highest-quality clinical practice guideline information. Through this mechanism, the committee believes that improvements in guideline quality and clinical effectiveness information can be achieved. REFERENCES The AGREE Collaboration. 2001. The Appraisal of Guidelines for Research and Ealuation (AGREE) instrument. London, UK: The AGREE Research Trust http://www.agreetrust. org/docs/AGREE_Instrument_English.pdf (accessed September 2007). AHRQ (Agency for Healthcare Research and Quality). 2007. U.S. Preentie Serices Task Force ratings http://www.ahrq.gov/clinic/uspstf07/ratingsv2.htm (accessed September 14, 2007).

OCR for page 121
 KNOWING WHAT WORKS IN HEALTH CARE American Academy of Pediatrics. 2007. History of the Red Book® http://aapredbook. aappublications.org/about/#hist (accessed October 23, 2007). Atkins, D., P. A. Briss, M. Eccles, S. Flottorp, G. Guyatt, R. T. Harbour, S. Hill, R. Jaeschke, A. Liberati, N. Magrini, J. Mason, D. O’Connell, A. D. Oxman, B. Phillips, H. Schünemann, T. T. Edejer, G. E. Vist, R. D. Williams, and the GRADE Working Group. 2005a. Systems for grading the quality of evidence and the strength of recommendations II: Pilot study of a new system. BMC Health Serices Research 5(1). Atkins, D., J. Siegel, and J. Slutsky. 2005b. Making policy when the evidence is in dispute. Health Affairs 24(1):102-113. Ayanian, J. Z., M. B. Landrum, S.-L. T. Normand, E. Guadagnoli, and B. J. McNeil. 1998. Rating the appropriateness of coronary angiography—Do practicing physicians agree with an expert panel and with each other? New England Journal of Medicine 338(26): 1896-1904. Beghi, E. 2004. Efficacy and tolerability of the new antiepileptic drugs: Comparison of two recent guidelines. Lancet Neurology 3(10):618-621. Boyd, C. M., J. Darer, C. Boult, L. P. Fried, L. Boult, and A. W. Wu. 2005. Clinical practice guidelines and quality of care for older patients with multiple comorbid diseases: Implica- tions for pay for performance. JAMA 294(6):716-724. Boyd, E. A., and L. A. Bero. 2006. Improving the use of research evidence in guideline develop- ment: 4. Managing conflicts of interests. Health Research Policy and Systems 4(16). Browman, G. P. 2001. Development and aftercare of clinical guidelines: The balance between rigor and pragmatism. JAMA 286(12):1509-1511. Burgers, J. 2003. Characteristics of high-quality guidelines. International Journal of Technol- ogy Assessment in Health Care 19(1):148-157. Burgers, J. S., and J. J. van Everdingen. 2004. Beyond the evidence in clinical guidelines. The Lancet 364(9432):392-393. Cabana, M. D., C. S. Rand, N. R. Powe, A. W. Wu, M. H. Wilson, P.-A. C. Abboud, and H. R. Rubin. 1999. Why don’t physicians follow clinical practice guidelines?: A frame- work for improvement. JAMA 282(15):1458-1465. Campbell, E. G., R. L. Gruen, J. Mountford, L. G. Miller, P. D. Cleary, and D. Blumenthal. 2007. A national survey of physician-industry relationships. New England Journal of Medicine 356(17):1742-1750. Choudhry, N. K., H. T. Stelfox, and A. S. Detsky. 2002. Relationships between authors of clinical practice guidelines and the pharmaceutical industry. JAMA 287(5):612-617. Cook, D., and M. Giacomini. 1999. The trials and tribulations of clinical practice guidelines. JAMA 281(20):1950-1951. Druss, B. G., and S. C. Marcus. 2005. Growth and decentralization of the medical literature: Implications for evidence-based medicine. Journal of the Medical Library Association 93(4):499-501. Ebell, M., J. Siwek, B. D. Weiss, S. H. Woolf, J. Susman, B. Ewingman, and M. Bowman. 2004. Strength of Recommendation Taxonomy (SORT): A patient-centered approach to grading evidence in medical literature. American Family Physician 69(3):548-556. Eddy, D. M. 2005. Evidence-based medicine: A unified approach. Health Affairs 24(1):9-17. Fielding, J. E., and P. A. Briss. 2006. Promoting evidence-based public health policy: Can we have better evidence and more action? Health Affairs 25(4):969-978. Fried, M., M. Farthing, J. Krabshuis, and E. Quigley. 2006. Global guidelines: Is gastroenterol- ogy leading the way? Lancet 368(9552):2041-2042. GRADE Working Group. 2004. Grading quality of evidence and strength of recommenda- tions. BMJ 328(7454):1490. Gray, B. H. 1992. The legislative battle over health services research. Health Affairs 11(4): 38-66.

OCR for page 121
9 DEVELOPING TRUSTED GUIDELINES Gray, B. H., M. K. Gusmano, and S. Collins. 2003. AHCPR and the changing politics of health services research. Health Affairs w3.283. Grilli, R., N. Magrini, A. Penna, G. Mura, and A. Liberati. 2000. Practice guidelines developed by specialty societies: The need for critical appraisal. Lancet 355:103-106. Grimshaw, J. M., and I. T. Russell. 1993. Effect of clinical guidelines on medical practice: A systematic review of rigorous evaluations. Lancet 342:1317-1322. Guirguis-Blake, J., N. Calonge, T. Miller, A. Siu, S. Teutsch, and E. Whitlock for the U.S. Preventive Services Task Force 2007. Current processes of the U.S. Preventive Services Task Force: Refining evidence-based recommendation development. Annals of Internal Medicine 0000605-200707170-200700170. Guyatt, G., D. Gutterman, M. H. Baumann, D. Addrizzo-Harris, E. M. Hylek, B. Phillips, G. Raskob, S. Z. Lewis, and H. Schünemann. 2006a. Grading strength recommendations and quality of evidence in clinical guidelines: Report from an American College of Chest Physicians task force. Chest 129(1):174-181. Guyatt, G., G. Vist, Y. Falck-Ytter, R. Kunz, N. Magrini, H. Schünemann, and R. Elena. 2006b. An emerging consensus on grading recommendations? ACP Journal Club A8-A9. Harris, G., B. Carey, and J. Roberts. 2007. Psychiatrists, children and drug industry’s role. The New York Times, “Health” http://www.nytimes.com/2007/05/10/health/10psyche. html?ei=5070&en= a90e19408d5df0cd&ex=1188964800&adxnnl=1&adxnnlx=11888 45745-KhpszR5rlQnw7Dxp8FR0Pw (accessed May 10, 2007). Hasenfeld, R., and P. G. Shekelle. 2003. Is the methodological quality of guidelines declin- ing in the U.S.? Comparison of the quality of U.S. Agency for Health Care Policy and Research (AHCPR) guidelines with those published subsequently. Quality and Safety in Health Care 12(6):428-434. Hunt, S. A., D. W. Baker, M. H. Chin, M. P. Cinquegrani, A. M. Feldmanmd, G. S. Francis, T. G. Ganiats, S. Goldstein, G. Gregoratos, M. L. Jessup, R. J. Noble, M. Packer, M. A. Silver, L. W. Stevenson, R. J. Gibbons, E. M. Antman, J. S. Alpert, D. P. Faxon, V. Fuster, G. Gregoratos, A. K. Jacobs, L. F. Hiratzka, R. O. Russell, and S. C. Smith, Jr. 2001. ACC/AHA guidelines for the evaluation and management of chronic heart failure in the adult: Executive summary a report of the American College of Cardiology/ American Heart Association Task Force on Practice Guidelines (Committee to Revise the 1995 Guidelines for the Evaluation and Management of Heart Failure). Circulation 104(24):2996-3007. IOM (Institute of Medicine). 1990. Clinical practice guidelines: Directions for a new program. Edited by Field, M. J., and K. N. Lohr. Washington, DC: National Academy Press. ———. 1992. Guidelines for clinical practice: From deelopment to use. Edited by Field, M. J., and K. N. Lohr. Washington, DC: National Academy Press. Jackson, R., and G. Feder. 1998. Guidelines for clinical guidelines. BMJ 317(7156):427-428. Kahan, J. P., R. E. Park, L. L. Leape, S. J. Bernstein, L. H. Hilborne, L. Parker, C. J. Kamberg, D. J. Ballard, and R. H. Brook. 1996. Variations by specialty in physician ratings of the ap- propriateness and necessity of indications for procedures. Medical Care 34(6):512-523. Kassirer, J. P. 2007. Chapter 7: Medicine’s obsession with disclosure of financial conflicts: Fix- ing the wrong problem. In Science and the media: Delgado’s brae bulls and the ethics of scientific disclosure. Edited by Snyder, P. J., L. Mayes, and D. Spencer. San Diego, CA: Academic Press, an imprint of Elsevier, Inc. Miller, J., and J. Petrie. 2000. Development of practice guidelines. Lancet 355(9198):82-83. Murphy, M. K., N. A. Black, D. L. Lamping, C. M. McKee, C. F. B. Sanderson, J. Askham, and T. Marteau. 1998. Consensus development methods, and their use in clinical guideline development. Health Technology Assessment 2(3):i-88. NGC (National Guideline Clearinghouse). 2007a. About inclusion criteria http://www. guideline.gov/about/inclusion.aspx (accessed August 17, 2007).

OCR for page 121
0 KNOWING WHAT WORKS IN HEALTH CARE ———. 2007b. Guideline index http://www.guideline.gov/browse/guideline_index.aspx (ac- cessed September 14, 2007). ———. 2007c. NGC browse—organizations http://www.guideline.gov/browse/browseorgsbyLtr. aspx?Letter=* (accessed June 2, 2007). ———. 2007d. Search for cardiology http://www.guideline.gov/search/searchresults.aspx? Type=3&txtSearch=cardiology&num=500 (accessed July 11, 2007). ———. 2007e. Search for hypertension http://www.guideline.gov/search/searchresults.aspx? Type=3&txtSearch=hypertension&num=500 (accessed August 12, 2007). ———. 2007f. Search for stroke http://www.guideline.gov/search/searchresults.aspx?Type=3 &txtSearch=stroke&num=500 (accessed August 12, 2007). O’Connor, P. J. 2005. Adding value to evidence-based clinical guidelines. JAMA 294(6): 741-743. O’Malley, A. S., H. H. Pham, and J. D. Reschovsky. 2007. Predictors of the growing influence of clinical practice guidelines. Journal of General Internal Medicine 22(6):742. Perlin, J. B., and J. Kupersmith. 2007. Information technology and the inferential gap. Health Affairs 26(2):w192-w194. Reinertsen, J. L. 2003. Zen and the art of physician autonomy maintenance. Annals of Internal Medicine 138(12):992-995. Ricci, S., M. G. Celani, and E. Righetti. 2006. Development of clinical guidelines: Method- ological and practical issues. Neurological Sciences 27(3):S228-S230. Schünemann, H. J., D. Best, G. Vist, A. D. Oxman, and the GRADE Working Group. 2003. Letters, numbers, symbols and words: How to communicate grades of evidence and recommendations. Canadian Medical Association Journal 169(7):677-680. Schünemann, H. J., A. Fretheim, and A. D. Oxman. 2006. Improving the use of research evidence in guideline development: 9. Grading evidence and recommendations. Health Research Policy and Systems 4(21). Schwartz, J. S. 1984. The role of professional medical societies in reducing practice variations. Health Affairs 3(2):90-101. Shaneyfelt, T. M., M. F. Mayo-Smith, and J. Rothwangl. 1999. Are guidelines following guide- lines?: The methodological quality of clinical practice guidelines in the peer-reviewed medical literature. JAMA 281(20):1900-1905. Shekelle, P. G., S. H. Woolf, M. Eccles, and J. Grimshaw. 1999. Clinical guidelines: Developing guidelines. BMJ 318(7183):593-596. Shekelle, P. G., E. Ortiz, S. Rhodes, S. C. Morton, M. P. Eccles, J. M. Grimshaw, and S. H. Woolf. 2001. Validity of the Agency for Healthcare Research and Quality clinical practice guide- lines: How quickly do guidelines become outdated? JAMA 286(12):1461-1467. Shiffman, R. N., P. Shekelle, M. Overhage, J. Slutsky, J. Grimshaw, and A. M. Deshpande. 2003. Standardized reporting of clinical practice guidelines: A proposal from the Confer- ence on Guideline Standardization. Annals of Internal Medicine 139(6):493-500. Stelfox, H. T. 1998. Conflict of interest in the debate over calcium-channel antagonists. New England Journal of Medicine 338(2):101-106. Stewart, W. F., N. R. Shah, M. J. Selna, R. A. Paulus, and J. M. Walker. 2007. Bridging the inferential gap: The electronic health record and clinical evidence. Health Affairs 26(2): w181-w191. Thomson, R., H. McElroy, and M. Sudlow. 1998. Guidelines on anticoagulant treatment in atrial fibrillation in Great Britain: Variation in content and implications for treatment. BMJ 316(7130):509-513. Tierney, W. M. 2001. Improving clinical decisions and outcomes with information: A review. International Journal of Medical Informatics 62:1-9. Tonelli, M. R. 2007. Conflict of interest in clinical practice. Chest 132(2):664-670.

OCR for page 121
 DEVELOPING TRUSTED GUIDELINES Tunis, S. R., R. S. A. Hayward, M. C. Wilson, H. R. Rubin, E. B. Bass, M. Johnston, and E. P. Steinberg. 1994. Internists’ attitudes about clinical practice guidelines. Annals of Internal Medicine 120(11):956-963. Weinstein, J. N., K. Clay, and T. S. Morgan. 2007. Informed patient choice: Patient-centered valuing of surgical risks and benefits. Health Affairs 26(3):726-730. Wennberg, J. E. 2004. Perspective: Practice variations and health care reform: Connecting the dots. Health Affairs var.140. Woolf, S. H., and D. Atkins. 2001. The evolving role of prevention in health care: Contribu- tions of the U.S. Preventive Services Task Force. American Journal of Preentie Medicine 20(3, S1):13-20. Woolf, S. H., R. Grol, A. Hutchinson, M. Eccles, and J. Grimshaw. 1999. Clinical guidelines: Po- tential benefits, limitations, and harms of clinical guidelines. BMJ 318(7182):527-530.

OCR for page 121