8
A Framework for the Future

It might be helpful to try viewing our world with both pride and alarm, both tempered by historical sense.

Herbert Muller, 1952

What, then, are we to make of the world of clinical practice guidelines? Gratification, perhaps, over the increasing commitment to develop systematic guidelines and move them into the everyday world of patient care. Concern for the future if too much is expected too soon. And dismay—but not surprise—that little empirical evidence exists about what makes guidelines useful and effective. Nevertheless, some findings, some conclusions, some lessons, some predictions, and some recommendations can be drawn from this study.

In this chapter, the committee responds to the second and third of its charges. It identifies some general strengths and limitations of current structures and processes for developing and applying clinical practice guidelines and then presents a framework for the future development, use, evaluation, and improvement of guidelines. That framework includes, first, a recapitulation of the committee's more specific findings about existing processes for developing and implementing guidelines; second, its recommendations for improvements; and, third, a proposed method and structure for assessing the soundness of guidelines developed by various organizations. The chapter concludes with some recommendations for a research agenda and a few comments on the role of guidelines in health care reform.

WORKING ASSUMPTIONS AND PRINCIPLES

The strongest of the committee's working assumptions was that practice guidelines can help to improve the quality of health care, reduce expen-



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use 8 A Framework for the Future It might be helpful to try viewing our world with both pride and alarm, both tempered by historical sense. Herbert Muller, 1952 What, then, are we to make of the world of clinical practice guidelines? Gratification, perhaps, over the increasing commitment to develop systematic guidelines and move them into the everyday world of patient care. Concern for the future if too much is expected too soon. And dismay—but not surprise—that little empirical evidence exists about what makes guidelines useful and effective. Nevertheless, some findings, some conclusions, some lessons, some predictions, and some recommendations can be drawn from this study. In this chapter, the committee responds to the second and third of its charges. It identifies some general strengths and limitations of current structures and processes for developing and applying clinical practice guidelines and then presents a framework for the future development, use, evaluation, and improvement of guidelines. That framework includes, first, a recapitulation of the committee's more specific findings about existing processes for developing and implementing guidelines; second, its recommendations for improvements; and, third, a proposed method and structure for assessing the soundness of guidelines developed by various organizations. The chapter concludes with some recommendations for a research agenda and a few comments on the role of guidelines in health care reform. WORKING ASSUMPTIONS AND PRINCIPLES The strongest of the committee's working assumptions was that practice guidelines can help to improve the quality of health care, reduce expen-

OCR for page 196
Guidelines for Clinical Practice: From Development to Use ditures for inappropriate and unnecessary services, and increase the value received for this country's spending on personal health care. Individual and organizational behavior are amenable to change; progress is possible. Nothing in the course of this project fundamentally undermined this assumption. Progress in the development and use of practice guidelines will be accompanied by continuing methodological, technical, ethical, political, attitudinal, and behavioral challenges. Perfect rationality, unswerving ethical behavior, and consistently thoughtful compliance with guidelines lie in the realm of ideals. Scientific knowledge will grow but will always be an incomplete foundation for guidelines. Concerns about costs will always be present. Practitioners and policy makers will inevitably face ethical strains as they try to balance the interests of the individual and of society. Fortunately, the long history of professional commitment by physicians, nurses, and other clinicians to the good of their patients and to the advancement of clinical knowledge is a critical resource for both the development and implementation of practice guidelines. Public commitment to the support of research and training is another essential component of progress. As a working principle, the committee accepted that its recommendations must be sensitive to the country's strong tradition of pluralism, despite some negative aspects of that tradition (to be noted shortly). Strong political and cultural traditions favor shared roles and responsibilities for the private and public sectors, although the general inclination is to favor private over public initiative. GENERAL STRENGTHS AND WEAKNESSES OF CURRENT PRACTICE GUIDELINES ACTIVITIES Efforts to develop practice guidelines can be characterized, somewhat simplistically, as long-standing or embryonic. Many professional and provider organizations have for many years been creating and applying what they construe to be practice guidelines; for them, guidelines development and implementation is not new. Others focus primarily on "modern" efforts to develop guidelines, which they see as dating back only a decade or so. In either formulation, strong and weak points stand out; they must be understood and either built on or overcome as efforts to develop and apply guidelines push forward. One caution: it is usually easier to spot problems than successes. The committee notes the limitations and problems in a spirit of identifying opportunities for progress. It hopes that this attitude will help to encourage those interested in better development, use, and evaluation of guidelines. Ultimately, this committee is confident that the history of clinical practice guidelines will be a positive one.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use What Are the Strong Points of Current Efforts? A first strength of current efforts is their pluralism. The commitment of both public- and private-sector resources helps to protect guidelines efforts from real or perceived "capture" by narrow interests. The lack of a dominant model and the existence of multiple, diverse sponsors have encouraged innovation in methods and flexibility to accommodate different potential users. For example, some initiatives have concentrated on new technologies that are of great interest to particular specialties or payers; others have focused on high-volume, low-unit-cost services. Likewise, some efforts aim to guide practitioners through complex clinical problems, whereas others seek practical tools for quality review and payment entities to assess the appropriateness of clinical decisions. By fostering a wider range of development and implementation activities than would be prompted by less diverse sponsorship, pluralism may also facilitate broader understanding and acceptance of guidelines. A second strength of the guidelines enterprise is simple enthusiasm. Policy makers have endorsed the undertaking, funding is increasing, and how-to-do-it conferences and similar products have been multiplying. Professional and specialty societies are clearly involved to a degree far beyond that observed 2 to 5 years ago. Processes for guidelines development are even seen as mechanisms for defining health insurance and benefit packages in ways that were rarely thought possible just a short time ago. Third, guidelines are gaining credibility. Expectations about the rigor needed to develop sound guidelines are increasing, and processes for guideline development are beginning to be reshaped. Also growing is professional consensus on two scores: the outcomes of patient care must be more broadly defined and carefully appraised, and the appropriateness of both new and old services must be subjected to more objective, critical scrutiny. A fourth strength is that researchers, clinicians, educators, and managers are being stimulated to consider how guidelines and other efforts to improve the quality and efficiency of health care can support and complement each other. These efforts include outcomes and effectiveness research, methods for strengthening informed patient decision making, and both traditional and newer techniques for quality assessment and quality improvement. The above strengths have not emerged from an overarching, deliberate plan. Rather, they are the result of a combination of deliberate strategies (for example, the creation of a guidelines function in the Agency for Health Care Policy and Research [AHCPR]) and the unorchestrated accumulation of many separate organizational initiatives. Part of the message of this report is the dual need to understand and capitalize on these processes and to channel them to better match health care needs.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use What Are the Limitations of Current Efforts? Another part of the message of this report concerns the current limitations and weaknesses of efforts to develop and apply practice guidelines. Some of these drawbacks are the "downsides" of factors mentioned above. Others relate to more general problems inherent in the nation's health care system. First, pluralism—the involvement of diverse groups in guidelines development—has negative as well as positive consequences. The limited resources for guideline development, use, evaluation, and improvement are inefficiently deployed. Development efforts are fragmented across groups with greatly varying goals, methods, and capacities, and cooperative efforts to develop guidelines that affect multiple specialties and practitioner types are still too atypical. Even if formal priorities have been established, the actual selection of topics for guidelines development seems to be haphazard within organizations and thus across the entire system. Second, the lack of quality control over methods and procedures is a particularly serious drawback of both national and local processes for developing guidelines. Many national organizations involved actively in developing guidelines and review criteria are moving to improve their programs, but weak procedures and products are common. Methods and procedures for local adaptation of national guidelines and for translation of guidelines into medical review criteria have not been thoroughly documented, but they certainly appear to be subject to equal or greater weaknesses. Potential users of guidelines and review criteria have no ready means to judge the soundness of materials produced by different groups with different approaches. Third, most guidelines fail in fundamental ways to anticipate the needs of clinicians, patients, and programs to assure quality, control costs, and reduce medical liability. Few guidelines provide any explicit treatment of patient preferences or estimates of the cost implications of their recommendations, certainly not in comparison with alternative practices. In addition, the educational opportunities implicit in guidelines cannot be fully exploited because the evidence and rationale for the guidelines are not presented. A fourth weakness is that efforts to evaluate the impact of practice guidelines have been limited. Despite widespread interest in guidelines as a tool for improving the quality and cost-effectiveness of care, almost nothing is known about whether they can or do contribute to these goals. IMPROVING THE DEVELOPMENT OF GUIDELINES Although this committee intended to focus attention almost exclusively on the implementation of guidelines rather than on their development, it discovered that the application of guidelines was sufficiently dependent on

OCR for page 196
Guidelines for Clinical Practice: From Development to Use certain characteristics of the development process that revisiting this subject became imperative. In doing so, the committee has stressed several points. First, guidelines developers must do better in anticipating the needs and concerns of potential users. Second, for developers to do this, procedures and methods need improvement. Third, more attention should be paid to the identification and analysis of inconsistencies among guidelines and to the rationales and results of local processes to develop or adapt guidelines. The committee also asked whether guidelines developers should be expected to take on even more demanding tasks by factoring cost-effectiveness into all their recommendations and by defining the minimum level and types of care that should be provided and ensured for all individuals. Building a Compelling Case for Recommendations Projections of health outcomes and the costs of achieving those outcomes are absent from most guidelines. Most also lack explicit assessments of the strength of the evidence, the relative importance of the projected benefits and risks, and how compelling is the case for particular interventions. Many of the future directions endorsed by this committee and summarized later in this chapter depend on better performance in these areas. For potential users to accept the case being made for the recommendations offered in a set of guidelines, other factors are also important and are included among the attributes for guidelines and review criteria identified in Chapters 1 and 5. In varying degrees, practitioners, payers, risk managers, and those involved in quality assurance and improvement perceive that many guidelines fall short in their applicability to real-world circumstances and in their clarity and precision. The committee recognizes the considerable gaps in empirical information about the natural history of many diseases and conditions, about health outcomes for many diagnostic or therapeutic interventions, and about the costs of providing those (or alternative) interventions. It also recognizes that the development strategy recommended here is highly demanding and that some, perhaps most or all, guidelines will never fully achieve the ideal. Nonetheless, if developers of practice guidelines make serious, persistent efforts in that direction, their products should become substantially more valuable and credible. Procedural and Methodological Issues Needing Particular Attention Given its emphasis on evidence, outcomes, and patient preferences and its concerns about the impact of guidelines on the quality and costs of health care, the committee focused on six methodological issues where further research and development is especially desirable:

OCR for page 196
Guidelines for Clinical Practice: From Development to Use means for setting priorities among topics for guidelines development procedures for securing thoughtful and useful statements of expert judgments methods for analyzing and rating scientific evidence techniques for improving knowledge of health outcomes and giving due importance to patient preferences methods for identifying and projecting the costs of alternative courses of care and comparing their cost-effectiveness mechanisms for identifying and evaluating inconsistent or conflicting guidelines. Each of the above areas now suffers a variety of minor and major weaknesses in either general procedures or technical methods. The fourth area is (and probably should be) receiving the most attention from the health services research community, but the other topics also warrant more study. In addition, the committee examined these persistent questions: Who should participate in the process and at what point, in what manner, and for what purpose? Selecting members of guidelines panels and selecting experts to review draft guidelines were underscored as especially critical steps in anticipating the challenges of implementation and in devising guidelines that are credible and useful to those involved in implementation. A further significant issue is the updating of guidelines, an essential activity, given advances in clinical research and the potential for confusion if guidelines were to include "sunset provisions" (i.e., to be authoritative one day and not the next). Yet updating places an extra burden on existing guidelines panels and organizations. Orderly processes for revising guidelines have yet to be implemented and may require some degree of assistance and coordination from AHCPR and such broad-based professional organizations as the American Medical Association. At the Interface Between Development and Use The committee also considered three subjects that arise at the interface between guidelines development and guidelines implementation: local adaptation of guidelines, inconsistent guidelines, and formatting and dissemination. In this context, the term local is used broadly to include multihospital systems, nationwide networks of HMOs, or other similar groups that may develop their own guidelines and modify those developed by others. Local Adaptation of Guidelines Some local adaptation of national guidelines is probably inevitable and may be useful, because even well-developed guidelines may have gaps and

OCR for page 196
Guidelines for Clinical Practice: From Development to Use may not foresee significant local objectives or constraints. The process of adapting guidelines can also educate practitioners and serve as a ratifying mechanism that helps win acceptance. Moreover, within a framework such as is offered by continuous quality improvement, empirical and incremental testing and modification of guidelines may well be appropriate (indeed, even necessary). This kind of testing may not conform to the highest standards of experimental research design, but it can provide a systematic, practical, and direct means of identifying where guidelines—as well as clinical practice—may need revision. Ideally, this kind of local but systematic information will become part of the broader evolutionary framework for guidelines development, revision, and improvement. To this end, the committee urges organizations that adapt guidelines to notify the originating group and explain the circumstances that led to their modifications. Adaptation may also serve less benign purposes—for example, protecting professional habits and local customs for their own sake or guarding economic self-interest by endorsing unnecessary care or care that others could provide as well or more economically. Casual, "back-of-the-envelope" approaches to adaptation offer particular temptations and opportunities for such unacceptable behavior. Where carefully developed and documented "national" guidelines exist, local adaptation processes should provide explicit rationales that relate to specific, well-defined local conditions or objectives and that take notice of the strength of the case for the original guidelines. Inconsistent Guidelines Inconsistent guidelines appear to be unavoidable, even for groups looking at the same scientific evidence and using defensible expert-judgment procedures. As suggested above, inconsistent guidelines provide an opportunity as well as a problem. The opportunity resides in the process of identifying inconsistencies and determining whether they should be tolerated, rejected, or reconciled. A form of disciplined accommodation is suggested to (1) strenuously seek areas of agreement, (2) make rationales for differences explicit and susceptible to comparison with available evidence, (3) reject recommendations or options that conflict with available evidence, and (4) allow options to remain where a case can be made that evidence is inconclusive, professional consensus is split, and variation is unlikely to harm quality of care. In any event, the areas of disagreement point strongly to topics warranting further clinical research. Formatting and Disseminating Guidelines The committee had concrete views about steps beyond development. First is effective formatting, which the committee sees as the presentation

OCR for page 196
Guidelines for Clinical Practice: From Development to Use of guidelines in physical arrangements or media that can be readily understood and applied by practitioners, patients, or other intended groups. Second is effective dissemination—delivering guidelines to the intended audiences in ways that promote the reception, understanding, acceptance, application, and positive impact of the guidelines. Effective dissemination presupposes effective formatting. The issues relating to dissemination are many, and the committee did not explore them in depth. Certainly, dissemination alone will neither induce use of the information being disseminated nor change behavior, and excessive distribution of information simply contributes to information overload. The committee concluded that a recognition of these complexities and appropriate planning for dissemination are important components of what guideline developers should do in the future. Going Further? Defining Cost-Effective and Minimum Levels of Care This report recommends that every set of clinical practice guidelines include information on the health and cost implications of alternative preventive, diagnostic, and management strategies for the clinical situation in question. The rationale is that this information can help potential users, who must take financial and other resources into account, to evaluate better the potential consequences of different practices. Should guidelines developers go further? Specifically, should every set of guidelines include cost-effectiveness as an explicit criterion for judging or recommending what constitutes appropriate care? Should guideline developers necessarily distinguish minimum, essential, or required levels of care in their products? After much debate, the committee concluded that every set of guidelines need not be based on formal judgments of cost-effectiveness; sound guidelines for clinical practice can stand on rigorous assessments of clinical evidence and carefully derived expert judgment. In addition, the committee declined—with some dissent—to recommend that guidelines must include statements of what constitutes minimum or required care for particular clinical problems. Both these responsibilities may be too expansive for individual guidelines panels or for organizations that face major challenges in following the path for guidelines development set forth in this report. Further, and perhaps more important, committee members could not agree that guidelines developers were, from a policy perspective, the right source of judgments about cost-effectiveness and minimum care; indeed, several members feared that such judgments would complicate the decisions of managers, payers, and policy makers who are actually in a position to make decisions about resources. The committee recognizes, however, that some developers of

OCR for page 196
Guidelines for Clinical Practice: From Development to Use guidelines may be technically, ethically, and politically positioned to make judgments about cost-effectiveness, particularly when those who are developing guidelines are also the intended users. This report is not meant to forestall such judgments. Doing More? Guidelines for Informed Patient Decision making With respect to informed patient decision making, the committee concluded that guideline developers should do more. Good medical care requires shared decision making by practitioners and patients. A commitment to shared decision making, however, does not in itself define what information should be provided to patients under different circumstances. Various organizational and public policies exist that, on the one hand, encourage or dictate the provision of certain information and, on the other hand, discourage or even preclude the provision of other information. Similarly, respect for patient preferences does not in itself answer the technical and policy questions about how to incorporate such preferences into the development or use of practice guidelines. Two separate paths are suggested here to deal with the difficult practical and ethical questions related to patient decision making and informed consent. One path is the development of treatment- and condition-specific practice guidelines that identify the strength of the evidence and of professional agreement behind statements about appropriate care and that estimate and assess outcomes in terms that patients can perceive as relevant. Strong evidence and consensus regarding a care option should imply strong duties to provide information to patients about that option. Weak evidence and consensus would permit more leeway for other factors to be weighed against these duties. A second path for improving the conditions for informed patient decision making is the development of a set of general guidelines for patient information and consent. These guidelines would supplement condition-or treatment-specific guidelines, on the one hand, and legally oriented patient consent forms, on the other. Such ''patient information guidelines" should be developed by a systematic process similar but not identical to that described for clinical practice guidelines. Once formulated, these guidelines would apply, unless specifically modified by condition-specific guidelines, to broad categories of patient care. The objective in creating such patient information guidelines would be to anticipate and specifically address how information and consent guidelines should cover (a) different kinds of care for (b) different kinds of patients in (c) different delivery systems and settings, given (d) different levels of certainty about the benefits, risks, and costs of care.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use ENSURING THE USE OF GOOD GUIDELINES Even when specific, well-founded guidelines exist, their effective use by patients and practitioners will require a broad range of supportive conditions and organizations. As those involved in programs to manage quality, costs, and liability begin to rely on guidelines, these common uses will provide powerful support for their consistent application in actual clinical practice. In particular, the force of peer influence should not be underestimated. Practitioner knowledge of guidelines and acceptance of their validity are key conditions for their successful application, but acceptance is not equivalent to behavioral change. Thus, as a practical matter, it may be better strategically or tactically to focus less on knowledge and acceptance and more on what changes behavior in desired directions. More than simple acceptance that a guideline is correct may be required to overcome countervailing forces, in particular, information overload, habitual practice patterns, malpractice fears, and economic disincentives. Quality Assurance and Improvement Well-developed, scientifically based practice guidelines have an important role to play in assessing, assuring, and improving the quality of health services provided in this country. Clear, specific guidelines and associated review criteria should help prevent or, alternatively, identify and remedy problems of overuse of care, underuse of care, and poor technical and interpersonal provision of care. Guidelines that have been accepted by those responsible for providing care, those responsible for financing it, and those responsible for monitoring care in the public interest are one means of bridging the chasm between internal and external quality assurance strategies. With respect to models of continuous quality improvement, the committee urges that their focus on systems problems, improvement of average performance, and reduction of variation be more systematically and explicitly joined with an effort to apply and improve sound guidelines for clinical practice. Specifically: Guidelines, medical review criteria, and other evaluative tools should be used both to improve average performance and—as is still important— identify substandard performance; Inquiries into how individual practice patterns differ from average patterns should go beyond statistical analysis to consider relevant practice guidelines as benchmarks for performance; Both the statistical information from such analyses and the pertinent guidelines should be part of educational feedback on practice patterns;

OCR for page 196
Guidelines for Clinical Practice: From Development to Use Evaluations of performance and outcome data should seek to determine the sources of poor outcomes and deviations from guidelines so that systems problems can be corrected and, if necessary, impaired individuals dealt with through training, counseling, limiting of privileges, or other appropriate mechanisms;1 Evaluations of performance and outcomes data should be used to indicate or determine whether practice guidelines ought to be updated or revised; Developers of guidelines and health care institutions should convene educational conferences to acquaint practitioners with specific guidelines and provide an opportunity for them to discuss and plan setting-specific applications; Institutional activities to develop or adapt guidelines or review criteria should aspire to incorporate the attributes for guidelines and for review criteria described elsewhere in this report. Cost Management On both philosophical and strategic grounds, this committee believes that thoughtfully designed and applied programs to encourage cost-effective use of health care have an important role to play in supporting the wider application of guidelines for clinical practice. Such programs need guidelines and related materials that provide information on the cost-effectiveness of alternative ways of managing particular clinical problems. They also need to be supplemented by explicit programs that employ guidelines and review criteria to monitor the quality and appropriateness of care. Those who develop review criteria should be guided by the attributes discussed earlier in this report. This recommendation applies to organizations that develop retrospective quality-of-care criteria, that generate prospective preprocedure or preadmission criteria, and that engage in all manner of "review" between these two extremes. Review organizations of all sorts, if they follow these attributes, will perforce do certain things. They will make their review activities as manageable and nonintrusive as possible for both patients and practitioners. They will make their review criteria available to practitioners and others. They will provide an explicit process for appealing negative decisions that is free from unreasonable complexity, delay, or other barriers. In addition, if such an organization identifies quality-of-care problems, it will have procedures for, at a minimum, discussing these problems with the involved practitioner or provider 1   The committee explicitly recognizes the need for protection of privacy and confidentiality as those concepts are understood in usual quality assurance terms (e.g., in actions of Medicare peer review organizations, state medical licensure boards, hospital quality assurance committees, and the like).

OCR for page 196
Guidelines for Clinical Practice: From Development to Use necessary for informed patient decision making, and (3) stronger assurance for practitioners that compliance with guidelines will reduce their exposure to medical liability. The design and application of any assessment process will depend on many factors—ethical, political, economic, and organizational. Furthermore, such a process requires at least two basic program components. One is a practical and valid assessment instrument; the other is a feasible administrative structure and process for applying the instrument. Assessment Instrument How might one construct an assessment instrument? The approach taken by the committee is presented in Appendix B, which contains the full text of the instrument along with an introduction that allows the appendix to be used as a freestanding document. The instrument is presented as provisional because more practical experience with it is essential. Volunteers from a large number of specialty societies and other experts reviewed a draft document; some also applied it to existing or draft guidelines. The resulting comments and suggestions, which were both extensive and candid, reinforced the committee's initial assumption that the instrument required more practical testing of its utility.2 The assessment instrument covers both the process used to develop a specific guidelines document and the substantive content of the document and its recommendations. (The committee did not want an assessment instrument that could allow a set of scientifically invalid or questionable guidelines to receive a "good" rating based on process criteria alone.) In essence, it attempts to operationalize both the substance and process attributes presented in Chapter 1. The committee believes the instrument will be useful as an educational tool for those beginning to develop guidelines; a self-assessment tool that developers of guidelines can use to check their work; and a tool for external groups to use in judging whether a set of guidelines should or should not be recommended or adopted. Successful application of the assessment instrument for this last purpose depends on several conditions. First, those assessing a set of guidelines must have access to the primary and secondary documentation of the evidence, rationales, and process involved. A summary statement of the 2   In accordance with the IOM'S contract with AHCPR, the provisional instrument was delivered in August 1991 after it had successfully completed the process of report review required by the National Research Council.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use recommendations is not sufficient. Second, those undertaking an assessment must individually or collectively possess the methodological and clinical knowledge required to apply the instrument to a particular set of guidelines. Relatively junior or inexperienced individuals can assemble relevant materials and check simple items, but experienced clinicians and methodologists must play key roles. Third, those using the instrument must be prepared to complete a task that will be both complex and tedious. Assessment Organization Given a reliable, valid assessment instrument, how might one apply it in a broader evaluative program? The above discussion of the instrument implies little about the characteristics of a specific institutional arrangement for assessing guidelines except that it would require more than trivial resources and stature. In considering recommendations about an assessment organization, the committee raised several questions. Is such an entity needed? What are the minimum conditions for its successful operation? Is there a reasonable probability that these conditions can be achieved? Is an Assessment Organization Needed? The committee believes the answer to this question is clearly yes. Existing guidelines vary enormously in quality, and the commitment and ability of developers to improve methods and content will also vary. Similarly, potential users of guidelines differ in their ability to identify the strengths and weaknesses of different guidelines. As the uses of guidelines become more clearly defined and significant, some way to distinguish good, bad, and mediocre guidelines becomes more important. The committee heard repeatedly from practitioners, providers, and others that they wanted guidance about the credibility and soundness of guidelines developed by different organizations. Existing private and public organizations may try to give such help, but acceptance of these efforts may not extend beyond limited constituencies of the involved groups. An assessment entity could stimulate more demand for credible guidelines; that demand, in turn, could stimulate the commitment of more resources to develop such guidelines. What currently exists is more like a chaotic "market" operating in an environment that has no effective means of distinguishing good products from bad. This was essentially the committee's rationale for stating (see Chapter 5) that it is premature to recommend that federal or state legislatures grant malpractice immunity to practitioners who act in conformance with practice guidelines. An effectively functioning assessment entity would go far to ameliorate the committee's concerns in this area. Such an organizational structure could also help realize the

OCR for page 196
Guidelines for Clinical Practice: From Development to Use recommendation that utilization management firms and similar organizations improve and make public their review criteria. The incentive provided by the assessment entity for such a move would be that favorable assessments might offer a competitive advantage. Is an Assessment Organization Feasible? That a need exists does not mean that a feasible means of meeting it can be devised. The conditions that must prevail for an assessment entity to be practical and viable are numerous. Four such conditions present particular challenges Effective demand for the product (i.e., the assessments of guidelines). The product has a clear potential market: providers, public and private payers, government generally, malpractice insurers, and consumer groups. That this potential demand would translate into actual financial and political support, however, is not inevitable. The survival rate of somewhat analogous organizations, such as the Council on Health Care Technology and the National Center for Health Care Technology, 3 is not encouraging, although the entity envisioned by the committee would not undertake the highly controversial task of advising public or private payers on coverage matters. Integrity of the process, participants, and assessments. Both the actual integrity of the entity and a widespread perception of its integrity must be created and maintained. To this end, considerable attention and sensitivity must be directed toward the choice of sponsors (financial as well as political), selection of managers (members of any board of directors, the executive director and other officers, consultants, and members of assessment panels or teams), and choice of topics and actual guidelines to be assessed. As to the last, explicit criteria and an open priority-setting process would be important. 3   The Council on Health Care Technology was established at the Institute of Medicine through the Health Promotion and Disease Prevention Amendments of 1984, partly in response to a recommendation in the 1983 IOM report, A Consortium for Assessing Medical Technologies; it was reauthorized in 1987. A complex system of public and private funding, which essentially called for private-sector financing to be acquired and spent before certain public matching monies (through the National Center for Health Services Research, or NCHSR) became available, proved to be an insufficient base of sustained support. The Council was disbanded at the same time that AHCPR was created (from NCHSR) in the Omnibus Budget Reconciliation Act of 1989. Earlier (1978), Congress created the National Center for Health Care Technology, which withered from lack of political support in the Executive Branch (Rettig, 1991b). A new effort to channel private funds to increase federal technology assessment activities is running into opposition from affected industries (Kent, 1991).

OCR for page 196
Guidelines for Clinical Practice: From Development to Use Sufficiency of effort. The necessary participants, methodologies, and other resources must be secured to undertake a reasonable volume of credible, usable assessments on a timely basis. This might be done, for example, by direct employment, subcontracting, creation of resource centers, or other mechanisms. Those involved in the assessments should have (collectively) an excellent understanding of actual clinical practice as well as a thorough grounding in clinical research, research methodologies, outcomes measurement, and cost-effectiveness analysis. Stability of effort. There is little reason to embark on an enterprise of this sort if one cannot project a reasonable period of existence to attract qualified participants and to support dissemination of results. For such stability, a sufficient, predictable, and visible commitment of political, financial, and other resources is essential. Is There a Reasonable Probability That These Conditions Will Prevail? The committee believes that such a reasonable probability exists. Therefore, it recommends the creation of an assessment organization. The following discussion describes an approach that the committee considers plausible and realistic. It highlights four key aspects of such an approach: governance, products, funding, and credibility. • Governance. Entities sponsored, governed, and funded by interested parties may enjoy a higher level and greater predictability of financial and other resources, but such sponsorship or funding can create threats to the real or perceived integrity of the undertaking. The perceived threats can turn on a distinction between public and private sectors per se, or on a distinction between interested and (presumably) not-so-interested parties, regardless of their private- or public-sector status. Even those generally favorable to the concept of an assessment entity—within the committee and outside—have directly conflicting views on the credibility of a public versus a private assessment entity. AHCPR already has legislative authority to perform the assessment function. Private funds could be added to appropriations for the work of the Forum for Quality and Effectiveness in Health Care to support the assessment function envisioned here.4 The agency might be viewed, however, as having a conflict of interest in assessing guidelines sponsored by its Forum or guidelines developed by outside contractors on its behalf. Furthermore, 4   In a related context, some private groups, in particular, major insurers, suggested in late 1991 that they would be interested in contributing significant funds to supplement government monies that have been appropriated to AHCPR's Office of Health Technology Assessment for assessing new technologies.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use its work would be subject to the usual vagaries of the federal appropriations process, and it would be politically vulnerable to hostile actions from organizations whose guidelines, once assessed, do not pass muster. Other groups, such as the American Medical Association (AMA), have also offered themselves in the role of "certifier" of guidelines. The AMA, in fact, has drafted its own assessment instrument and developed a process for applying it. Nevertheless, a powerful medical organization could have its own problems of credibility in this role, no matter how well it performs. Clearly, assessment would be a sensitive undertaking when guidelines from one group in the larger association clearly conflicted with or were "prejudicial" to those of another group. Weighing the various pluses and minuses, the committee finally concluded that an assessment entity would best be organized as a private, not for-profit organization and that it should have a governing board drawn from a wide range of interested parties, both public and private. The entity must be apart from, but able to work with, the parties that have a stake in guideline development. To forestall criticisms about objectivity and integrity, the board of any such organization would develop clear procedures regarding bias, conflicts of interest, and other issues of accountability.5 • Products and focus. The proposed assessment entity would have one primary product: periodic publication of assessments of the guidelines issued by public and private organizations. Overseeing publication of the assessments and other dissemination activities would be the responsibility of the governing board. In terms of publications, the committee believes that a journal is an attractive option. Its articles or reports should combine the academic rigor of top professional journals with the user-oriented style of a publication like Consumer Reports. The latter journal has several attractive features.6 It compares products with a similar purpose rather than reporting on products in isolation. It uses graphics and other devices to great advantage to provide easy-to-assimilate information on the strengths, weaknesses, and characteristics of products. Further, it explicitly recognizes that consumers have different preferences and circumstances that may lead them to different choices based on individual weighing of this information. An annual review issue might provide summary compilations of the 5   Such procedures would include peer review of the assessments. In addition, the entity might publish both its assessment methodology and its assessments in draft form for public review and comment. The latter would be analogous to the process by which federal regulations are published in the Federal Register for comment before final promulgation occurs. AHCPR's Patient Outcomes Research Teams, the committees of the National Academy of Sciences, and similar organizations offer other model procedures. 6   The committee is not recommending, however, that the assessment entity be modeled directly on Consumers Union.

OCR for page 196
Guidelines for Clinical Practice: From Development to Use assessments. Both the quarterly issues and the annual collection would, ideally, be indexed by the National Library of Medicine in a manner similar to its plans for AHCPR guidelines; short synopses of the assessments might even be available through NLM on-line services. Teleconferences or workshops organized around particularly controversial or important assessments might also be considered as dissemination strategies. The assessment methodology or methodologies could be made available to interested parties as a freestanding publication, which might be similar to the provisional instrument presented in Appendix B. The assessment organization might eventually move into other dissemination, educational, and clearinghouse activities if its initial efforts proved to be successful. For example, it might provide training sessions for organizations interested in learning how guidelines would be assessed. It might also produce assessments or commentaries on related items such as medical review criteria, commercial software products, software standards such as those promulgated by the American Society for Testing and Materials, and new graphic display techniques. Another information dissemination effort might be the production of occasional special publications or the sponsorship of conferences (with published proceedings) or workshops for the exchange of ideas and new developments. The organization might even publish—but not develop—guidelines that met its assessment standards or, in some cases, publish economic analyses for guidelines that did not include them. The committee emphasizes, however, that it sees the organization's central mission as assessing guidelines. In keeping with the proposed user orientation, the assessment entity should seek out guidelines to assess as well as accept submissions. For example, it could focus on clinical topics of particular interest for which multiple sets of guidelines exist. This appears, from anecdotal evidence, to be a critical need—one the AHCPR Forum is being asked to address by developing definitive guidelines in certain areas. An organization that can sort out good from not-so-good guidelines might help to conserve the funds of guideline developers for work on clinical problems that are not so widely endowed with conflicting (or any) contemporary guidelines. A proactive search strategy is likely to uncover second-rate guidelines that the sponsors would not submit on their own. Other kinds of user-oriented assessments of such guidelines would seek to be constructive rather than merely critical and to provide encouragement to developers of guidelines to improve their processes and products. One measure of the assessment entity's success will be the extent to which, over time, developers of guidelines seek its assessments. • Funding. Funding from both public and private sources is desirable, and it could be in the form of start-up monies, long-term core support,

OCR for page 196
Guidelines for Clinical Practice: From Development to Use special project grants, and purchase of products and services. Of these, long-term core financing is the most important, and uncertainty about public or private willingness to move from general expressions of interest to actual financial support constitutes the major short-term challenge to the creation of an assessment entity. Once core funding is secured, additional financing could be obtained in several ways. One would be to charge a substantial subscription for the products of the organization. The subscription response would provide an early test of market appeal and feasibility. Another source of revenue could come from training or other activities, for which fees would be at least sufficient to recoup costs. • Credibility. All the features described above are intended to provide the assessment organization with initial and continuing credibility. To further the entity's credibility, a key objective should be the creation of a virtual ''fail-safe" mechanism to prevent clinically flawed guidelines from receiving a generally favorable assessment. Achieving this objective may require a pretesting process. An important first step is for AHCPR to test the IOM's provisional assessment instrument and to compare the results with pretests of its guidelines. Another key to credibility is for the procedures used by the assessment organization, as well as the assessment instrument and other tools, to be open and in the public domain. The assessment organization should have a user orientation that extends to a range of interested parties. Still, its assessments should be particularly attuned to everyday clinical practice and sensitive to the reliance of practitioners on their professional societies for guidance and support. For long-lasting credibility, establishing a constructive relationship with these professional societies must be a priority. The committee was acutely interested in the potential for legislation to grant legal immunity to practitioners who practiced in accord with guidelines that received positive assessments. It concluded, however, that a specific recommendation must await the specific policy decisions and operating strategies that will emerge only as the assessment entity shifts from concept to reality. As that happens, the future legal use of assessments should be an explicit interest. RESEARCH AGENDA In addition to developing its proposal regarding the assessment entity described earlier, the committee drew some conclusions about research to help answer certain questions about practice guidelines. The points noted below are narrowly aimed at guidelines development, implementation, and evaluation. Most generally, the committee urges continued investment in research on effectiveness and outcomes of health care and in programs for

OCR for page 196
Guidelines for Clinical Practice: From Development to Use technology assessment. These activities are a necessary part of the scientific and analytic support for clinical decision making and guidelines development. They also support management and policy decision making about how to allocate limited resources among alternative uses. Yet another priority is the testing and improvement of methods for guideline and criteria development, dissemination, application, and evaluation. Chapters 6 and 7 noted a number of outstanding technical and methodologic issues relating to cost-effectiveness analysis, weighing and combining evidence, consensus development and expert judgment. Clarifying the statistical aspects of, say, meta-analysis or effect sizes as they relate to amassing and interpreting scientific evidence is another. Other questions warranting attention include the effects of different ways of phrasing recommendations on the understanding or behavior (or both) of clinicians and patients and, perhaps, whether translation into languages other than English changes the meaning or import of those recommendations in unanticipated ways. One early, specific research activity should be the testing and perhaps refining of the provisional assessment instrument. Here, the questions include reliability, in its application by different users; validity, in the sense that it will discriminate well between good and not-so-good guidelines (or development processes); and practicality. Formal investigation of the feasibility and utility of the instrument for the entire guidelines enterprise—and its revamping as necessary—strikes the committee as a likely prerequisite for the success of any assessment entity. A middle step in understanding the impact of guidelines is an understanding of adoption and diffusion patterns. In this instance, one important question involves the role of opinion leaders and so-called pacesetter physicians; related issues concern graduate medical education (or graduate training in all the health professions now engaged in guidelines development). Of particular interest may be the extent to which physicians in residency training are an appropriate target audience; both guidelines and materials about the importance of being involved in guidelines development and implementation throughout their careers would be appropriate matters to emphasize. A clear priority is research on the actual impact of clinical practice guidelines on what clinicians and patients do and on the health status of patients and populations. This research focus is of a piece with studying how to change practitioner and patient behavior—a significant question for the quality assurance field in particular. Although impact can be addressed in part through proxy measures (e.g., measures relating to dissemination), an appreciation of whether given guidelines really make a difference in practice patterns, lifestyles, and the like will require complex, longitudinal

OCR for page 196
Guidelines for Clinical Practice: From Development to Use studies. Ideally, these might be planned as randomized clinical trials; short of that, various types of demonstration projects and efforts similar to those of the AHCPR Patient Outcomes Research Team (PORT) program might be considered. A related objective should be to determine how traditional clinical trials might be redesigned and extended so that the dissemination, application, and impact of practice guidelines could be studied in actual health care delivery settings. In this formulation, the findings from such studies, for instance, in groups of academic medical centers together with participating practices in the community, might later be used to improve and update the guidelines under study. Finally, in much the same manner that "no-difference" results in clinical trials and health services research can be illuminating, understanding why seemingly good guidelines have had no impact (or why seemingly poor ones did have an effect) may be a fruitful avenue of investigation. Such research could identify facilitating or complicating factors, either intrinsic to the guideline topic or extrinsic to it but prevalent in the broader health care environment, that might be taken into account as the guidelines enterprise moves into higher gear during the 1990s. Work along these lines (e.g., as case studies) might be a profitable interim step while more complex longitudinal investigations are being planned and conducted. FINAL NOTE: GUIDELINES AND HEALTH CARE REFORM During its deliberations, the committee was quite conscious of the intense debate occurring about broad health care reform in this country and about the contributions that practice guidelines might make to workable reform. In the committee's view, reform concerns two issues: access and cost. Politically, expansion of access is contingent on some sense that the rate of escalation in health care costs can be reduced. The committee's discussions of cost-effectiveness and minimum care have already touched on these concerns. As noted earlier, the committee has expressed reservations about assigning guideline developers the task of recommending what care is worth paying for. Consequently, it recommended that guideline developers concentrate instead on providing the clinical information, judgments, and rationales on which policy makers, payers, managers, and others might base such decisions. Some proposals for reform include provisions for clinical practice guidelines that would seem generally consistent with the committee's views on who should make judgments about what is worth covering under public or private health benefits plans. For example, one proposal would create a Health Standards Board to design a set of "uniform effective health benefits" that

OCR for page 196
Guidelines for Clinical Practice: From Development to Use would be "responsive to public values about the appropriateness of various treatments,7 incorporate scientifically based information on treatment effectiveness, and be sensitive to considerations of cost in terms of the net benefits of services" (Ellwood, 1991, p. 1). A similar proposal (Brook, 1991) would establish a public governing body to oversee work by interdisciplinary teams based at academic medical centers to develop necessary care guidelines;8 federal legislation would require all health plans to cover this care. Chapter 6 raised several questions about the technical and ethical challenges involved in making such judgments, whether the decision makers be developers of guidelines, insurers, or public officials. A contrasting approach relies not on guidelines that define uniform coverage policies but on guidelines that provide for varying standards of care (Havighurst, 1990a). Health plans would adopt different guidelines (perhaps those certified as meeting a minimum standard of acceptability), and consumers could choose what level of care they were willing and able to purchase. In theory, both health plans and consumers would make judgments about what care is worth covering. Chapter 5 raised some questions about this proposal, but the discussion of inconsistent guidelines in Chapter 7 made clear that inconsistencies were not necessarily unacceptable, for example, when the guidelines reflected different value judgments about what was an important benefit (and documented this as part of the rationale for their recommendations). These and other proposals for health care reform raise many questions that are beyond the scope of this committee's charge. Some of the health care reform proposals that are described above and that are being widely discussed in the health policy literature and lay media envision sweeping changes in the nation's health care delivery and financing systems. These changes would certainly place guidelines in a framework of incentives for cost containment that is different from what currently exists. The specifics vary, but the basic ideas are that the reforms would override state benefit 7   In Ellwood's terms (1991), these values would relate to "the value of a service to society (e.g., public health impact, social costs, community compassion), the value of a service to the individual at risk, and the extent to which a service is considered an essential component of a basic level of health care below which no person should fall. Public values can be periodically assessed through surveys, focus groups, public hearings, and other community meeting formats" (p. 1). This is roughly similar to the process used in Oregon (see Chapter 6). Presumably, the Health Standards Board would limit the extent to which public distaste for some kinds of health problems (e.g., sexually transmitted diseases, smoking-related illness) could dictate coverage. 8   Necessary care "(1) is appropriate (medical benefit exceeds medical risk); (2) produces important benefits to the patients receiving it; and (3) would be considered improper for physicians not to recommend to the patient. It would disturb both primary care physicians and specialists if it were not provided" (Brook, 1991, p. 3000).

OCR for page 196
Guidelines for Clinical Practice: From Development to Use mandates, circumvent court-ordered coverage in individual cases, rewrite malpractice laws, reduce administrative costs through single-payer arrangements, and limit the coverage eligible for tax deductibility. Some reforms—for example, those that envision practice guidelines as the basis for defining a basic benefits package for all health insurance plans— would put a premium on the kinds of credible, accountable processes for developing and applying guidelines that are described in this report. The danger in such proposals is that the potential contributions guidelines have to make in improving the quality of health care and health outcomes may be lost in a perception that guidelines serve only cost-containment purposes. The committee sees, therefore, both unprecedented opportunities for the clinical practice guidelines movement as well as exceptional challenges in the years ahead.