Guidelines for Clinical Practice: From Development to Use (1992)

Guidelines for the provision of clinical care have been linked in recent years to almost every major problem and proposed solution on the American health policy agenda. Practice guidelines have been tied in some way, by some individual or organization, to costs, quality, access, patient empowerment, professional autonomy, medical liability, rationing, competition, benefit design, utilization variation, bureaucratic micromanagement of health care, and more. The concept has acted as a magnet for the hopes and frustrations of practitioners, patients, payers, researchers, and policy makers.

This recent surge of interest notwithstanding, guidelines are not new. Professional organizations have been developing guidelines for at least half a century, and recommendations about appropriate care can be found in ancient writings. What is new is the emphasis on systematic, evidence-based guidelines and the interest in processes, structures, and incentives that support the effective use and evaluation of such guidelines.

Given this history, the Institute of Medicine (IOM) study reported here had two objectives: first, to encourage constructive expectations for guidelines and, second, to promote the kind of care and rigor in their development, application, evaluation, and revision that would help such expectations to be realized. A committee of experts in clinical practice, health care policy and administration, health services research, and related fields met five times between June 1990 and September 1991 to develop this report.¹

The committee used as a starting point the 1990 IOM report, Clinical Practice Guidelines: Directions for a New Program. That report advised the Agency for Health Care Policy and Research (AHCPR) and its Forum for Quality and Effectiveness in Health Care on their responsibilities for guidelines.

Guidelines for clinical practice cannot realistically be viewed as the solution to the country's health care problems, in particular, the problem of escalating costs. Nevertheless, systematically developed, science-based guidelines can become part of the fabric of health care in this country, and they can serve as useful tools for many desirable changes. Their potential reach extends from improving the quality of clinical care (and its measurement) to helping reduce the financial costs of inappropriate, unnecessary, or dangerous care. Practice guidelines are among the building blocks for informed patient decision making and rational social judgments about what care should be covered by public and private health benefit plans.

As tools and building blocks for positive change, guidelines need to be understood and encouraged in context. That context includes powerful economic interests; changing and sometimes conflicting attitudes about professional and patient autonomy; policy making and implementing institutions that are intensely stressed and sometimes incapacitated; and scientific research that simultaneously expands both knowledge and uncertainty. Above all, the context includes the complex, intimate relationship between individual patients and practitioners who are trying to protect health, manage illness, and preserve dignity under conditions that range from routine to desperate.

As defined in the IOM's 1990 report, practice guidelines are ''systematically developed statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances." Medical review criteria, which are also discussed in this report, are "systematically developed statements that can be used to assess the appropriateness of specific health care decisions, services, and outcomes."

Practice guidelines focus, in the first instance, on assisting patients and practitioners in making decisions, but this defining characteristic does not and should not preclude their use for other purposes including quality improvement and payment policy making. Conversely, medical review criteria and related tools emphasize the evaluation of health care decisions, actions, and outcomes, but they should and do build on guidelines and may in some cases be virtually identical.

Practice guidelines are not synonymous with the reimbursement or coverage policies of Medicare and other health insurance plans, which traditionally have excluded some items from coverage (for example, immuniza-

tions and blood products) for reasons unrelated to the appropriateness of the service. Policies for reimbursement and coverage certainly may be informed by practice guidelines, but the two should not be confused.

The potential users of clinical practice guidelines are diverse, and any single user of guidelines may employ them in various ways. Five major purposes for guidelines, which are not mutually exclusive, are (1) assisting clinical decision making by patients and practitioners; (2) educating individuals or groups; (3) assessing and assuring the quality of care; (4) guiding allocation of resources for health care; and (5) reducing the risk of legal liability for negligent care. This report focuses on the primary users of guidelines: practitioners, patients and their families, and health care institutions. Other users include those payers, health benefit plans, and public policy makers and regulators who may use guidelines in making specific decisions about what health care to reimburse, cover, or encourage and in evaluating the decisions, actions, or performance of the primary users of guidelines.

Some would explain the interest in practice guidelines shown by legislators, regulators, and purchasers of health care (as contrasted with practitioners and patients) with a single phrase: out-of-control health care costs. Although the importance of costs as a stimulus for guidelines should not be understated, concerns about quality of care, risk management, and improved patient outcomes also figure prominently in the call for more and better practice guidelines.

Much of the interest in guidelines has been prompted by perceptions that higher health care expenditures have brought only marginal health benefits and that guidelines can help remedy this problem of "value." Virtually every major discussion of guidelines begins with a similar list of reasons for these perceptions: (1) wide variations in physician practice patterns and use of health services; (2) research indicating inappropriate use of many services; and (3) uncertainty about the health outcomes achieved by the use or nonuse of various services and procedures.

Whether the issue is unexplained variation, inappropriate care, or uncertain outcomes, many analysts come to similar conclusions. More research on outcomes and effectiveness of health care services is needed; more effort should be invested in using such research to formulate specific guidelines for clinical practice; more use of the resulting guidelines will help limit spending for health services.

How are guidelines to limit health care costs? How are they to increase the perceived value of health care spending? Implicitly or explicitly, the basic argument or hypothesis runs along these lines:

At least six formidable—and often unrealistic—assumptions lie behind this model. First, a sufficient quantity and quality of scientific evidence exists to serve as a foundation for guidelines. Second, programs to develop guidelines will be organized, funded, and effectively managed to produce a considerable volume of valid, usable statements about appropriate care for clinically and financially significant health conditions or technologies. Third, substantial numbers of clinicians, patients, and others will have the opportunity, the support, and the incentives to read, understand, accept, and use these statements in ways that change patterns of clinical practice, health behavior, or payment for health care services in desired directions. Fourth, such changes will be broad and intense enough to improve health outcomes. Fifth, on balance, the entire body of guidelines as actually developed and used will lead to more cost-controlling than cost-increasing behavior on the part of providers and patients. Sixth, the body of guidelines will continually expand to cover new areas so that net rates of increase in health care costs and absolute levels of expenditures will be lower than they would otherwise be.

Again, these are formidable expectations. They outstrip current capacities with respect to the base of scientific knowledge, the translation of that science into usable practice guidelines, and the incentives and structures to encourage application of such guidelines. Even if the first four expectations stated above were to be fulfilled, the fifth and sixth expectations about the cost consequences of change are questionable. Some guidelines undoubtedly will save money by reducing the use of inappropriate or unnecessary services; some will increase expenditures by encouraging more use of underutilized services; and some will shift costs from one type of service to another or from one payer to another. The result should be better value, but the net impact on the rate of increase in total health care spending cannot be predicted with confidence, even if future priorities for guidelines development stress clinical conditions for which costly overuse of services is suspected.

In sum, guidelines for clinical practice are a promising but not a quick or sure strategy for improving and rationalizing the overall use of health services. The attention and resources now invested in guidelines could dissolve in the face of a collision between unrealistic hopes and limited immediate results. Persistent commitment over the long term is required from both policy makers and health care professionals.

It is usually easier to spot problems than successes. The committee notes the limitations of current efforts to develop and use guidelines in a spirit of identifying opportunities for progress. It hopes that this attitude will help to encourage those interested in better development, use, and evaluation of guidelines. Ultimately, the committee is confident that the history of clinical practice guidelines will be a positive one.

A first strength of current efforts is their pluralism. The commitment of both public- and private-sector resources helps to protect guidelines efforts from real or perceived "capture" by narrow interests. The lack of a dominant model and the existence of multiple, diverse sponsors have encouraged innovation in methods and flexibility to accommodate different potential users. By fostering a wider range of development and implementation activities than would be prompted by less diverse sponsorship, pluralism may also facilitate wider understanding and acceptance of guidelines.

A second strength of the guidelines enterprise is simple enthusiasm. Policy makers have endorsed the undertaking, funding is increasing, and how-to-do-it conferences and similar products have been multiplying. Professional and specialty societies are clearly involved to a degree far beyond that observed two to five years ago. Processes for guidelines development are even seen as mechanisms for defining health insurance and benefit packages in ways that were rarely thought possible just a short time ago.

Third, guidelines are gaining credibility. Expectations about the rigor needed to develop sound guidelines are increasing, and processes for guideline development are beginning to be reshaped. Also growing is professional consensus on two scores: the outcomes of patient care must be more broadly defined and carefully appraised, and the appropriateness of both new and old services must be subjected to more objective, critical scrutiny.

A fourth strength is that researchers, clinicians, educators, and managers are being stimulated to consider how guidelines and other efforts to improve the quality and efficiency of health care can support and complement each other. These efforts include outcomes and effectiveness research, methods for strengthening informed patient decision making, and both traditional and newer techniques for quality assessment and quality improvement.

The above strengths have not emerged from an overarching, deliberate plan. Rather, they are the result of a combination of deliberate steps (for

example, the creation of a guidelines function in AHCPR) and the unorchestrated accumulation of many separate organizational initiatives. Part of the message of this report is the dual need to understand and capitalize on these processes and to channel them to better match health care needs.

Another part of the message of this report concerns the current limitations and weaknesses of efforts to develop and apply practice guidelines. Some of these drawbacks are the "downsides" of factors mentioned above. Others relate to more general problems inherent in the nation's health care system.

First, pluralism—the involvement of diverse groups in guidelines development—has negative as well as positive consequences. The limited resources for guideline development, use, evaluation, and improvement are inefficiently deployed. Development efforts are fragmented across groups with greatly varying goals, methods, and capacities, and cooperative efforts to develop guidelines that affect multiple specialties and practitioner types are still too atypical. Even when formal priorities have been established, actual selection of topics for guidelines development seems to be haphazard within organizations and thus across the entire system.

Second, the lack of quality control over methods and procedures is a particularly serious drawback of both national and local processes for developing guidelines. Many national organizations involved actively in developing guidelines and review criteria are moving to improve their programs, but weak procedures and products are common. Methods and procedures for local adaptation of national guidelines and for translation of guidelines into medical review criteria have not been thoroughly documented, but they certainly appear to be subject to equal or greater weaknesses. Potential users of guidelines and review criteria have no ready means to judge the soundness of materials produced by different groups with different approaches.

A third weakness is that efforts to evaluate the impact of practice guidelines have been limited. Despite widespread interest in guidelines as a tool for improving the quality and cost-effectiveness of care, virtually nothing is known about whether they can or do contribute to these goals.

Although this committee intended to focus almost exclusively on the implementation of guidelines rather than on their development, it discovered that the application of guidelines was sufficiently dependent on certain characteristics of the development process that revisiting this subject became imperative. In doing so, the committee stressed several points. First,

guidelines developers must do better in anticipating the needs and concerns of potential users and in building a case for their recommendations that users will find compelling. Second, for developers to do this, procedures and methods need improvement. Third, more attention should be paid to the identification and analysis of inconsistencies among guidelines and to the rationales and results of local processes to develop or adapt guidelines. The committee also questioned whether guidelines developers should be expected to take on even more demanding tasks by factoring cost-effectiveness into all their recommendations or by defining the minimum level and types of care that should be provided for all individuals.

Most guidelines fail to build a compelling case for the relevance, importance, and soundness of their recommendations. To varying degrees, practitioners, payers, risk managers, and those involved in quality assurance and improvement perceive that many guidelines fall short in their applicability to real-world circumstances and in their clarity and precision. In fundamental ways, they do not anticipate the needs of clinicians, patients, and programs to assure quality, control costs, and reduce medical liability.

One theme of this report is that the way clinical practice guidelines are developed can strongly affect their potential for effective use by practitioners, patients, and others. Those who devise guidelines must anticipate what will make the guidelines practical and credible. Thus, planning for successful implementation should begin with development and continue through cycles of revision and updating.

The IOM has specified desirable attributes of clinical practice guidelines and medical review criteria (Tables 1 and 2). Each attribute affects the likelihood that guidelines will be perceived as trustworthy and usable or the probability that they will, if used, help achieve desired health outcomes. For practice guidelines, four of the eight attributes relate to substantive content: validity, reliability, clinical applicability, and clinical flexibility. Four others relate to the process of guideline development or the presentation of guidelines: clarity, multidisciplinary process, scheduled review, and documentation. For review criteria, additional attributes are sensitivity, specificity, patient responsiveness, readability, minimum obtrusiveness, feasibility, computer compatibility, and appeals criteria.

This report emphasizes that every set of guidelines should be accompanied by (1) a statement of the strength of the evidence and the expert judgment behind the guidelines and (2) projections of the relevant health and cost outcomes of alternative courses of care. Assessments of relevant health outcomes will consider patient perceptions and preferences.

To the extent that guidelines move toward the characteristics outlined

above, they will identify how compelling is the case for particular services or courses of care under particular clinical circumstances. They will distinguish care for which there is good scientific evidence, care for which there is good consensus but limited or no evidence, and care for which there is neither evidence nor consensus. Conclusions backed by scientific evidence are more compelling than those based on subjective judgments. To the degree that review criteria build on such guidelines and follow the attributes

listed in Table 2, they will help those reviewing care to identify appropriate care more accurately.

In addition to following a careful analytic strategy in creating guidelines, developers should present their work to clinicians in ways that reflect the rigor of this approach and its emphasis on reasoning and critical analysis. Thus, the product of the process should not be perceived solely as information but more generally as an explication of the thinking processes that should be used in evaluating and applying that information. If guidelines are perceived only as information, they may very well be used (or rejected) as the "cookbooks" that many physicians decry. They will also not achieve their potential as educational tools.

In fact, few guidelines today provide any formal projections of health benefits and harms, any explicit treatment of patient preferences, or any estimates of the cost implications of their recommendations, certainly not in comparison with alternative practices. Most also lack explicit assessments of the strength of the evidence behind their recommendations. In addition, the educational opportunities implicit in guidelines cannot be fully exploited because the evidence and rationale for the guidelines are often not presented. Many of the future directions endorsed by this committee depend on better performance in these areas.

The committee recognizes that the development strategy recommended here is highly demanding and that some, perhaps most or all, guidelines will never fully achieve the ideal. It also recognizes the considerable gaps in empirical information about the natural history of many diseases and conditions, about health outcomes for many diagnostic or therapeutic interventions, and about the costs of providing those (or alternative) interventions. Nonetheless, if developers of practice guidelines make serious, persistent efforts in the directions recommended here, their products should become substantially more valuable and credible.

Although developers of guidelines have considerable room for improvement in the use of existing techniques, the methods for guidelines development are themselves in need of refinement. Given its emphasis on evidence, outcomes, and patient preferences and its concerns about the impact of guidelines on the quality and costs of health care, the committee focused on six questions of methodology:

1. means for setting priorities among topics for guidelines development;

2. procedures for securing thoughtful and useful statements of expert judgments;

3. methods for analyzing and rating scientific evidence;

4. techniques for improving knowledge of health outcomes and giving due importance to patient preferences;

5. methods for identifying and projecting the costs of alternative courses of care and comparing their cost-effectiveness; and

6. mechanisms for identifying and evaluating inconsistent or conflicting guidelines.

The committee considered three subjects that arise at the interface between guidelines development and guidelines implementation: local adaptation of guidelines, inconsistent guidelines, and formatting and dissemination. In this context, the term local is used broadly to include multihospital systems, nationwide networks of HMOs, or other similar groups that may develop their own guidelines and modify those developed by others.

Some local adaptation of national guidelines is probably inevitable and may be useful, because even well-developed guidelines may have gaps and may not foresee significant local objectives or constraints. The process of adapting guidelines can also educate practitioners and serve as a ratifying mechanism that helps win acceptance.

Moreover, within a framework such as is offered by continuous quality improvement, empirical and incremental testing and modification of guidelines may well be appropriate (indeed, even necessary). Such testing may not conform to the highest standards of experimental research design, but it can provide a systematic, practical, and direct means of identifying where guidelines—as well as clinical practice—may need revision. Ideally, this kind of local but systematic information will become part of the broader evolutionary framework for guidelines development, revision, and improvement nationwide. To this end, the committee urges organizations that adapt guidelines to notify the originating group and explain the circumstances that led to their modifications.

Adaptation may also serve less benign purposes—for example, protecting professional habits and local customs for their own sake or guarding economic self-interest by endorsing unnecessary care or care that others could provide as well or more economically. Casual, "back-of-the-envelope" approaches to adaptation offer particular temptations and opportunities for such unacceptable behavior. Where carefully developed and documented "national" guidelines exist, local adaptation processes should provide explicit rationales that relate to specific, well-defined local conditions or objectives and that take notice of the strength of the case for the original guidelines.

Inconsistent guidelines appear to be unavoidable, even for groups looking at the same scientific evidence and using defensible expert-judgment procedures. Inconsistencies may worry or annoy clinicians, deter the use of sound guidelines, and undermine the credibility of guidelines generally. In some respects, however, inconsistent guidelines provide an opportunity as well as a problem. The opportunity resides in the process for identifying inconsistencies and for determining whether they should be tolerated, rejected, or reconciled. A disciplined approach to inconsistent guidelines would (1) strenuously seek areas of agreement, (2) make rationales for different recommendations explicit and susceptible to comparison with available evidence, (3) reject recommendations or options that conflict with available evidence, and (4) allow options to remain where a case can be made that evidence is inconclusive, professional consensus is split, and variation is unlikely to harm quality of care. In any event, the areas of disagreement point strongly to topics warranting further clinical research.

Finally, the work of guidelines developers typically extends to some activities beyond the formulation of statements about appropriate care, activities that shade into the work involved in guidelines implementation. One step is effective formatting—presenting guidelines in physical arrangements or media that can be readily understood and applied by practitioners, patients, or other intended groups. Another step is effective dissemination—delivering guidelines to the intended audiences in ways that promote the reception, understanding, acceptance, application, and positive impact of the guidelines. Effective dissemination presupposes effective formatting.

The issues relating to dissemination are many, and the committee did not explore them in depth. Certainly, dissemination alone will neither induce the use of the information being disseminated nor change behavior, and excessive distribution of information exacerbates information overload. The committee concluded that a recognition of these complexities and appropriate planning for dissemination are important components of what guideline developers should do in the future.

This report recommends that every set of practice guidelines include information on the health and cost implications of alternative preventive, diagnostic, and management strategies for the clinical situation in question.

The rationale is that this information can help potential users, who must take financial and other resources into account, to better evaluate the potential consequences of different practices. Should guidelines developers go further? Specifically, should every set of guidelines include cost-effectiveness as an explicit criterion for judging or recommending what does or does not constitute appropriate care? Should guideline developers distinguish minimum, essential, or required levels of care in their products?

After much debate, the committee concluded that every set of guidelines need not be based on formal judgments of cost-effectiveness; sound guidelines for clinical practice can stand on rigorous assessments of clinical evidence and carefully derived expert judgment. In addition, the committee declined—with some dissent—to recommend that guidelines must include statements of what constitutes minimum or required care for particular clinical problems.

The committee did not say that conclusions and decisions based on cost-effectiveness should be or can be avoided. Governments, health benefit plans, health care providers, and others must make such judgments, although they may not always do so explicitly or rationally. However, committee members could not agree that guidelines developers were, in general and from a policy perspective, the right source of judgments about cost-effectiveness and minimum care. Indeed, several members feared that such judgments would complicate the resource decisions of managers, payers, and policy makers. Moreover, responsibilities for judging cost-effectiveness may be too expansive for individual guidelines panels or for organizations that face major challenges in following the path for guidelines development set forth in Chapters 1, 2, and 7 of the report.

The committee did recognize that some developers of guidelines may be technically, ethically, and politically positioned to make judgments about cost-effectiveness, particularly for some kinds of problems or when those who are developing guidelines are also the intended users. This report is not intended to forestall such recommendations.

In considering the issue of minimum, necessary, or required care, the committee noted troublesome inconsistency and confusion in terminology. For example, terms such as medical necessity and basic benefits are used in very different and even inconsistent ways that complicate and distort debates over important social issues. The committee's discussion underscored the substantial technical, administrative, ethical, and policy challenges involved in any effort to define what care is required, beneficial, unindicated, inappropriate, or harmful.

With respect to informed patient decision making, the committee concluded that guideline developers should do more. Good medical care re-

quires shared decision making by practitioners and patients. However, a commitment to shared decision making does not in itself define what information should be provided to patients under different circumstances. Similarly, respect for patient preferences does not in itself answer all technical and policy questions about how to incorporate such preferences into the development or use of practice guidelines.

Two separate paths are suggested here to deal with some of the difficult practical and ethical questions related to patient decision making and informed consent. The first path is the development of treatment-and condition-specific practice guidelines that identify the strength of the evidence behind statements about appropriate care and that estimate and assess outcomes in terms that patients perceive as relevant.

A second path for improving the conditions for informed patient decision making is the development of general guidelines for patient information and consent. These guidelines would supplement condition- or treatment-specific practice guidelines, on the one hand, and legally oriented patient consent forms, on the other. Such ''patient information guidelines" should be developed by a systematic process similar to that described for clinical practice guidelines. Once formulated, these guidelines would apply, unless specifically modified by condition-specific guidelines, to broad categories of patient care. Patient information guidelines would need to anticipate and specifically cover responsibilities and procedures appropriate for (a) different kinds of care for (b) different kinds of patients in (c) different delivery systems and settings, given (d) different levels of certainty about the benefits, risks, and costs of care.

Even when specific, well-founded guidelines exist, their effective use by patients and practitioners will require a wide range of supportive conditions and organizations. As those involved in programs to manage quality, costs, and liability begin to rely on guidelines, these common uses will provide powerful support for their consistent application in actual clinical practice. In particular, the force of peer influence should not be underestimated.

Practitioner knowledge of guidelines and acceptance of their validity are key conditions for their successful application, but acceptance is not equivalent to behavioral change. As a practical matter, it may be better strategically or tactically to focus less on knowledge and acceptance and more on what changes behavior in desired directions. More than simple acceptance of a guideline's correctness may be required to overcome countervailing forces, in particular, information overload, habitual practice patterns, malpractice fears, and economic disincentives.

Well-developed, scientifically based practice guidelines have an important role to play in assessing, assuring, and improving the quality of health care services provided in this country. Clear, specific guidelines and associated review criteria should help prevent or, alternatively, identify and remedy problems of overuse of care, underuse of care, and poor technical and interpersonal provision of care. Guidelines that have been accepted by those responsible for providing care, financing care, and monitoring care in the public interest are one means of bridging the chasm between internal and external quality assurance strategies.

With respect to models of continuous quality improvement, the committee urges that their focus on systems problems, strong customer-supplier relationships, improvement of average performance, and reduction of variation be more systematically and explicitly joined with an effort to apply and improve sound guidelines for clinical practice. Guidelines can support these and related quality assurance models in several ways.

First, to the extent that guidelines become more sensitive to patient preferences and participation in decision making, they should improve patients' informed consent, their participation in decision making and, ultimately, their satisfaction with both the processes and outcomes of care. Guidelines can also help identify important patient outcomes to incorporate in patient satisfaction surveys and other instruments designed to assess or improve performance. Second, guidelines and review criteria can play a role in identifying possible quality problems arising from underuse, overuse, or incompetent provision of care. They may be particularly useful in instances in which short-term health outcomes (those that are most readily employed) may not be good indicators of long-term results. Third, to the extent that guidelines identify how compelling is the evidence for certain clinical practices, they can help to determine priorities for improving or standardizing specific patterns of clinical care and to sort out competing claims for funding for biomedical and outcomes or effectiveness research. Fourth, participation by clinicians in the review, critique, and improvement of practice guidelines can help bring the science of medicine more forcefully into the planning, action, evaluation, and adjustment cycles required for quality assurance and improvement.

More specifically, the committee recommends the following:

• Guidelines, medical review criteria, and other evaluative tools should be used both to improve average performance and—as is still important—identify substandard performance;

• Inquiries into how individual practice patterns differ from average patterns should go beyond statistical analysis to consider relevant practice guidelines as benchmarks for performance;

• Both the statistical information from such analyses and the pertinent guidelines should be part of educational feedback on practice patterns;

• Evaluations of performance and outcome data should seek to determine the sources of poor outcomes and deviations from guidelines so that systems problems can be corrected and, if necessary, impaired individuals dealt with through training, counseling, limiting of privileges, or other appropriate mechanisms;²

• Evaluations of performance and outcomes data should be used to indicate or determine whether practice guidelines ought to be updated or revised;

• Developers of guidelines and health care institutions should convene educational conferences to acquaint practitioners with specific guidelines and provide an opportunity for them to discuss and plan setting-specific applications; and

• Institutional activities to develop or adapt guidelines or review criteria should aspire to meet the attributes for guidelines and for review criteria described in Tables 1 and 2.

On both philosophical and strategic grounds, this committee believes that thoughtfully designed and applied programs to encourage cost-effective use of health care have an important place in supporting the wider application of guidelines for clinical practice. Such programs are a clear market for guidelines and related materials that provide information on the cost-effectiveness of alternative ways of managing particular clinical problems.

Those who pay for health care services and their agents can use guidelines in various ways: (1) to help determine health insurance coverage and avoid payment for unnecessary or inappropriate care, (2) to aid in selecting or credentialing practitioners for participation in various health plans or institutions, and (3) to tailor other economic incentives to affect practitioner or patient behavior. Such approaches usually do not depend on a specific organized practice setting; that is, they can affect practitioners and patients in solo or group practice settings as well as those in larger organizational or institutional settings. Some approaches may be more or less confined to third-party payers whereas others may be shared by health care institutions, quality review programs, and others.

Programs that develop or employ review criteria to assess the appropriateness of care for specific patients should be guided by the attributes noted in Table 2 and should be supplemented by explicit efforts to monitor the quality and appropriateness of care. This recommendation applies to organizations that formulate or use criteria for retrospective review of care, that generate prospective preprocedure or preadmission criteria, and that engage in all manner of "review" between these two extremes. Review organizations of all sorts, if they follow these attributes, will perforce do certain things that should make their programs more effective, more palatable to practitioners, and less vulnerable to legal liability. They will make their review activities as manageable and as nonintrusive as possible for both patients and practitioners. They will make their review criteria available to practitioners and others. They will provide an explicit process for appealing negative decisions that is free from unreasonable complexity, delay, or other barriers.

It is the committee's hope that economic incentives and quality review mechanisms will, in the future, reduce the need for so-called micromanagement of professional and institutional behavior. External utilization review still may have a role in monitoring practice and targeting problem practices, but many payers will admit that they would prefer to rely more on effective self-regulation by practitioners and providers. Consistent with quality improvement principles, they can then stress education and feedback to physicians aimed at improving practice rather than punishing errors.

Guidelines that are based on available scientific evidence and that are clear, specific, and developed by a reputable process should carry greater weight in malpractice decision making than vague, nonspecific guidelines that lack documentation and careful reasoning. Guidelines that underscore their recommendations with reference to a strong foundation of scientific evidence should be particularly helpful.

Specific statutory recognition of guidelines, which is intended to provide legal protection to conforming clinicians, is desirable but premature. Acceptable legislation that provides immunity from liability would need to specify operational criteria for the organizations developing guidelines or particular criteria for guidelines themselves, or both. The criticisms directed at the variability and weaknesses of review criteria developed or adopted by Medicare PROs and carriers (and the fact that the criteria of the latter groups are often secret as well) made the committee reluctant to accept organizational imprimatur alone as a sufficient basis for a grant of immunity. Absent explicit procedures and standards for assessing the soundness of

practice guidelines (as recommended in this report), the committee believes that giving formal legal stature to any guideline at this early stage may create more problems than it solves.

No existing information infrastructure can support the kind of effective, unobtrusive, easy application of guidelines envisioned by quality improvement models, future-oriented utilization management and cost-containment systems, and patient-centered care proposals. Clearly, however, information technologies are being developed that will make the application of guidelines much easier, particularly if other conditions support their use. For clinicians, creating user-friendly decision aids that relate information about specific patients to guidelines about classes of similar patients deserves greater emphasis and more effort.

The work of the National Library of Medicine and others to establish some capability of responding to user inquiries and dissemination needs related to guidelines should be encouraged. The committee also supports efforts of the library to expand its capacity to assist in guideline development through expansion of its Office for Health Services Research Information. In addition, the committee favors the translation and movement of guidelines into computerized decision aids of various sorts. It recommends, however, that those efforts conform to emerging computer industry standards to enable guidelines (however transformed) to be used on different types of computer-based equipment and systems.

This committee has strongly urged that processes for developing and revising guidelines be soundly based on science and expert clinical judgment and that guidelines should anticipate the needs of practitioners, patients, and many other interested parties. How can these groups determine whether different guidelines measure up to these expectations? The committee believes that one answer would be a mechanism for independent assessment of practice guidelines—a stance backed by many of the individuals and organizations consulted during this study.

The design and application of an assessment process will depend on many factors—ethical, political, economic, and organizational. Furthermore, such a process requires at least two basic program components. One is a practical and valid assessment instrument; the other is a feasible structure and process for applying that instrument.

A major task of this committee was the development of a provisional instrument for assessing practice guidelines. The instrument (presented in full in Appendix B) is provisional because it requires more practical testing of its utility. It covers both the process used to develop a specific guidelines document and the substantive content of that document and its recommendations. In essence, the assessment instrument attempts to operationalize the attributes defined in Table 1. A major goal was an instrument that would not allow a set of scientifically invalid or questionable guidelines to receive a "good" rating based on process criteria alone. The committee believes the instrument will be useful as (1) an educational tool for those beginning to develop guidelines; (2) a self-assessment tool that developers of guidelines can use to check their work; and (3) a tool for external groups to use in judging whether a set of guidelines should or should not be recommended or adopted.

In considering recommendations about an assessment organization, the committee posed several questions. Is such an entity needed? What are the minimum conditions for its successful operation? Is there a reasonable probability that these conditions can be achieved?

The committee concluded that such an assessment organization could lead practitioners, patients, and others to the better guidelines from among the mixed lot now available and could generate several specific benefits. These benefits include: (1) firmer judgments about what care should be paid for under public and private health benefit programs; (2) better decisions about what information is necessary for informed patient decision making; and (3) improved use of scientific evidence and expert judgment in malpractice cases and stronger assurance for practitioners that compliance with sound guidelines would reduce their exposure to medical liability.

The minimum conditions for the successful operation of an assessment organization were relatively easy to identify: effective demand for the product (that is, the assessments); integrity of process, participants, and assessments; and sufficiency and stability of effort and resources. The probability of achieving these conditions is less easy to determine. Nonetheless, the committee concluded that it could propose an approach to creating an assessment organization that was feasible. The four key elements of this approach are the following:

• Governance. Weighing the various pluses and minuses of governance options, the committee finally concluded that an assessment entity

would best be organized as a private not-for-profit organization and that it should have a governing board drawn from a wide range of interested parties, both public and private. The entity must be apart from, but able to work with, all parties that have a stake in clinical practice guidelines. To forestall criticisms about objectivity and integrity, its board would develop clear procedures regarding bias, conflicts of interest, and other issues of accountability.

• Products and focus. The proposed assessment entity would have one primary product: periodic publication of assessments of guidelines issued by public and private organizations. A regular journal is an attractive option. Its articles or reports should combine the academic rigor of top professional journals with the user-oriented style of a publication like Consumer Reports. The latter journal has several attractive features. It compares similar-purpose products rather than reporting on products in isolation. It also uses graphics and other devices to great advantage to provide easy-to-assimilate information on the strengths, weaknesses, and characteristics of products. Finally, it explicitly recognizes that consumers have different preferences and situations that may lead them to different choices based on their own individual weighing of this information.

• Funding. Both public and private funding are desirable. This could be in the form of start-up monies, long-term core support, and special project grants. Of these, the long-term core financing is the most important. Additional financing could be obtained in several ways. One is to charge a substantial subscription for the products of the entity. The subscription response would provide an early test of market appeal and feasibility.

• Credibility. All of the features described above are intended to provide the assessment entity with initial and continuing credibility. One key objective should be the creation of a virtual "fail-safe" mechanism to prevent clinically flawed guidelines from receiving a generally favorable assessment. This may require a pretesting process. An important first step is for AHCPR to test the IOM's provisional assessment instrument and to compare the results with pretests of its guidelines. Another key to credibility is that the procedures used by the entity, the assessment instrument, and other tools be open and in the public domain.

The assessment organization should be oriented toward a broad set of potential users of guidelines. Nonetheless, its assessments should be particularly attuned to everyday clinical practice and sensitive to practitioners' reliance on their professional societies for guidance and support. Establishing a constructive relationship with these professional societies must be a priority for the organization.

In addition to proposing an assessment entity and identifying guidelines methodology issues in need of attention, the committee suggests a set of research activities aimed at guidelines development, implementation, and evaluation. They include three key areas: (1) testing and perhaps refining the provisional assessment instrument; (2) expanding research on the adoption and diffusion of medical innovations and information to consider whether, how, and why guidelines are adopted; and (3) evaluating the actual impact of clinical practice guidelines on what clinicians and patients do and on patients' health status. With respect to this last topic, "no-difference" results should be viewed as opportunities for further investigation rather than as dead ends; it is important to understand why particular guidelines have no impact.

More generally, the committee urges that recent investments in outcomes research and technology assessment be continued as a necessary part of the scientific support for clinical decision making and guideline development. Such investment will also support better policy and management decision making about how to allocate limited resources among alternative uses.

Finally, the committee notes that the clinical and health services research communities have an important role to play in smoothing the path from clinical research to better clinical practice and improved health outcomes. If more attention is paid to testing the effectiveness of procedures and patient management strategies in real-life settings rather than only assessing efficacy in highly controlled clinical trials, then developers of guidelines will be more likely to have a knowledge base with greater practical relevance. In turn, the more practitioners and institutions adopt the tools of outcomes management, the more information there will be to evaluate and revise guidelines.

During its deliberations, the committee was quite conscious of the intense debates about broad health care reform and about the contributions that practice guidelines might make to workable reform. In the committee's view, reform is about two issues: access and cost. Politically, expansion of access is contingent on some sense that the rate of escalation in health care costs can be reduced. As noted already, the net impact of guidelines on costs cannot be predicted, and expectations in this area should not be too high. The committee also expressed reservations about cost containment strategies that would assign guideline developers the task of judging what

care is worth paying for. It recommended instead that developers concentrate on providing the clinical information, judgments, and rationales on which policy makers, payers, managers, and others might base such decisions.

Some proposals for reform include provisions for cost containment that incorporate roles for clinical practice guidelines in defining or administering basic benefit packages, strengthening health plan competition and consumer choice, or restructuring malpractice decision making. The specifics vary, but the basic ideas are that the reforms would do one or more of the following: override state benefit mandates, circumvent court-ordered coverage in individual cases, rewrite malpractice laws, reduce administrative costs through national or regional administrative and regulatory structures, and limit the coverage eligible for tax deductibility. These and other proposals for health care reform raise many questions that are beyond the scope of this committee's charge. Some reform proponents envision sweeping changes in the nation's health care delivery and financing systems that would certainly place guidelines in a framework of incentives for cost containment that is much different from what currently exists.

Some reforms—for example, those that foresee practice guidelines as the basis for defining a basic benefits package for all health insurance plans— would put a premium on the kinds of credible, accountable processes for developing and applying guidelines described in this report. The danger in such proposals is that the potential contributions guidelines have to make in improving the quality of health care and health outcomes may be lost in a perception that guidelines are to serve only cost-containment ends. The committee sees, therefore, both unprecedented opportunities for the clinical practice guidelines movement and exceptional challenges as well in the years ahead.