National Academies Press: OpenBook

Guidelines for Clinical Practice: From Development to Use (1992)

Chapter: 7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES

« Previous: 6 THE INESCAPABLE COMPLEXITY OF DECISIONMAKING: ETHICS, COSTS . . .
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

7
Evolution in Procedures and Methods for Developing Practice Guidelines

You didn't tell me I'd spend all my time plowing up snakes.

Chair of an AHCPR guidelines development panel, 1990

Involvement in the development of clinical practice guidelines is a learning experience that has both positive and negative features--—as suggested by the above comment from one participant in the process. Involvement in implementation likewise provides lessons that are relevant to the process of developing guidelines. In examining the practical, technical, and policy questions about guidelines implementation and health care reform raised in the preceding chapters, the committee concluded that it needed to underscore the point made in Chapter 2: Planning for successful implementation begins with the development of guidelines. In the future, guidelines developers should give more and earlier attention to what will make guidelines practical and credible. This kind of early consideration will require both improvements in technical methods and greater sensitivity to how guidelines may be appropriately integrated into information systems, quality assurance programs, liability decision making, and cost-management efforts.

Fortunately, accelerating professional, governmental, and other involvement in the guidelines enterprise is reflected in two phenomena: the sheer amount of effort now seen and the increased focus on improving the development process and its products. This expansion of the field shows itself in several ways. Among them are the following:

  • maturation and specification of formal procedures and structures for guideline development;

  • growing appreciation of the complexity and importance of involving the appropriate kinds of individuals in guideline development; and

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
  • increased concern about competing and conflicting guidelines, locally adapted guidelines, and transformed versions of guidelines (such as medical review criteria).

More generally, experience with guidelines development is highlighting two rather different (but not mutually exclusive) emphases in or orientations to the process of guidelines development. One approach stresses the significance of the science base for guidelines and the use of quantitative modeling in systematically estimating and comparing outcomes. The other approach stresses professional judgment in areas in which the science base is weak or nonexistent.

This duality need not and should not be seen as an unbridgeable dichotomy. Professional judgment must be applied to the science base, and science must inform professional judgment. When the science base is strong, however, it should not be disregarded in favor of consensus based on customary practice. When consensus is not consistent with the evidence, the case for consensus should be explicitly and persuasively argued.

This chapter begins with a brief discussion of how certain key players in the guidelines arena have evaluated and refined their organizational structures and procedures over the years. Following are several sections that examine persistent issues about methods for developing guidelines, approaches that selected groups have taken in dealing with these issues, and problems that warrant continued attention. A final section discusses the interface of development and implementation as it involves, first, conflicting "national" guidelines; second, local adaptation of existing or emerging "national" guidelines; and, third, formatting and dissemination of guidelines.

The discussion of attributes for review criteria in Chapter 5 and the discussion of cost analysis in Chapter 6 also relate to the theme of this chapter. Although the focus here is on practice guidelines, much of this chapter is also relevant to development of medical review criteria.

GENERAL STRUCTURES AND PROCEDURES

As organizations recognize the demands of developing guidelines in a credible and accountable manner, those entities that plan an ongoing involvement tend to initiate commonly used organizational processes. They create supporting committees, staff positions, procedures, record-keeping systems, budget justification mechanisms, communications links, and eventually, with more difficulty, mechanisms for evaluating performance and results. Certainly, organizational resources constrain what can be established, but if resources are too limited to create and maintain such organizational structures, they may also be too limited to support the development of products consistent with the attributes set forth in Chapter 1.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

The following examples indicate some key ways in which guidelines development is evolving. The first considers the early learning experience of the Agency for Health Care Policy and Research (AHCPR); the second and third examples involve one private and one public organization's efforts to evaluate their work and make their products more credible and useful to practitioners; the fourth example focuses on interorganizational cooperation as a way of building both relevance and credibility.

Learning Lessons: The Agency for Health Care Policy and Research

Not surprisingly, given its short existence and its pattern of rather substantial staff turnover (since September 1990, three directors and three contractors), the AHCPR Forum for Quality and Effectiveness in Health Care has found its structures and procedures to be somewhat in flux. Still, the richness and value of the first year's experience with the AHCPR panels should not be underestimated, and the Forum has made a serious effort to evaluate and build on their experience. When the Forum was barely into its second year, the staff organized a retreat to consider what panel participants and staff had learned from its first few guidelines panels. "Lessons learned" about these complex activities included the points below.1

First, the work of the guidelines panel chairpersons has proved vastly more demanding than had been originally envisioned. Current panel chairs and AHCPR staff believe that a commitment of at least 25 percent time is needed to handle these activities adequately.

Second, the literature reviews have been more time-consuming and in some senses more costly than expected. The literature searches, reviews, and analyses took as much as nine months from start to finish and cost anywhere from $22,000 to $235,000, evidently depending chiefly on the strategy used for the literature review and the size of the body of work that needed to be included. In some cases, they were also less rewarding than anticipated, owing in part to the difficulty of identifying (only) appropriate journal articles and similar materials through current National Library of Medicine indexing and coding systems.

Third, costs could probably be brought down somewhat if time constraints on the panels could be loosened and if more specific instructions for methodology were available. Trying to meet tight deadlines tends to be expensive. For example, to complete their work on time, some panels employed unnecessarily highly qualified individuals to carry out the literature review; most made extensive use of Federal Express and overnight mail

1  

The points cited are based on unpublished materials ("Summary of Responses to a Questionnaire on Guideline Panel Activities and Views") prepared for the AHCPR Office of the Forum by Health Systems Research, Inc., January 24, 1991.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

instead of regular mail. Participants in some panels argued for more centralized guidance about, for example, summary tables, schemes for rating evidence, and other details, which might help to minimize costs related to unnecessary ''experimenting" in these areas.2

Fourth, in general, the procedures for the first round of AHCPR panels were not uniformly helpful to the panels and their chairpersons. Participants in this first staff retreat, however, could not agree on what specifically they might leave unchanged and what they might modify. One area in which consensus did materialize was that a skilled methodologist should assist the chairperson in organizing the literature search, review, and analysis throughout the process.

The variability in views of the panel chairs and others engaged in the agency's early efforts is itself instructive. Certainly, much of the seeming inefficiency of the initial panels can be ascribed to the fact that the Forum was a new unit in a new agency performing a new function with little time for adequate advance planning—under those circumstances, the endeavors may have gone as smoothly as might have been expected. A second retreat may also be scheduled.

As described in Chapter 2, the agency has elected to sponsor some guidelines panels (on otitis media, rehabilitation following stroke, and congestive heart failure) through a contracting mechanism. The contractors are required to recommend chairpersons and approximately 15 members of the panels, according to an explicit set of criteria specified by AHCPR. Among those criteria are relevant training and clinical experience, interest in quality assurance and research on the clinical condition in question, capacity to lead a health care team and to respond to consumer concerns, a broad public health view, and a commitment to and prior experience in the development of clinical practice guidelines.

Building a Formal Program: The Clinical Efficacy Assessment Program

The work of the American College of Physicians (ACP) exemplifies the ongoing formalization of professional society efforts to develop guidelines (Morris, 1987; Ball, 1990; White and Ball, 1990). The ACP began its work on guidelines in 1976 in response to a request from what is now the Blue Cross and Blue Shield Association for assistance in assessing medical techniques; the initial effort was consensus based and relatively informal. In 1981, with a grant from the John A. Hartford Foundation, the ACP initiated

2  

A consultant to AHCPR prepared an "Interim Manual" as a protocol for expert panels convened by the Forum: although dated October 1990, it was available in draft form earlier in that year (Woolf, 1990a).

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

a demonstration project, the Clinical Efficacy Assessment Project (CEAP). In 1984, the ACP established CEAP as a permanent program and in 1986 published its procedures for guidelines development (ACP, 1986).

During 1990, the college evaluated the CEAP effort on the principle that "any good policy making process must be both self-critical and, when called for, self-correcting" (White and Ball, 1990, p. 51). In one innovative step, the ACP convened focus groups to learn more about the utility and significance of its CEAP efforts. The review made clear that the ACP's decade of experience with CEAP laid the groundwork for experimenting with new models for guidelines development and evaluation.

In the 1990s, the college plans to strengthen its program. Among plans for the future are (1) using new methods for assessing data, including patient preferences; (2) revising formats for guidelines; (3) making draft guidelines available on line for a network of members who will pretest the guidelines and then measure patient outcomes when the guidelines are used according to specific protocols; (4) starting a formal convening activity to involve multidisciplinary groups in the development of guidelines; and (5) developing a systematic and perhaps new way of updating guidelines (Linda White, ACP, personal communication, August 1991). The ACP is also working with researchers at Johns Hopkins Medical Institutions to survey ACP members about their knowledge, perceptions, and use of guidelines. Finally, the ACP has announced plans for a new center to link guidelines development and outcomes research and to try to determine more reliably the use of guidelines by physicians and their utility for these practitioners. In sum, the focus is very much on improving guidelines development and evaluation so that the products of these processes can be more readily and effectively adopted.

Improving Consensus Development: The National Institutes of Health

Over time, government agencies involved directly or indirectly with guideline development have--—like professional societies—refined their procedures and methods. One example is the National Institutes of Health (NIH) Consensus Development Conference program, which is administered by the Office of Medical Applications of Research (OMAR, 1988). In the 1980s, OMAR undertook several assessments of the program. Some work was done internally—for example, trials of different mechanisms for running conferences and for disseminating consensus statements (Jacoby, 1983, 1985; Perry, 1987, 1988). Other evaluations were performed by outside parties (Wortman and Vinokur, 1982; Wortman et al., 1988), culminating in a lengthy and rigorous evaluation conducted by RAND Corporation researchers of the content of consensus statements and their impact in terms

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

of different behaviors on the part of physicians (Kosecoff et al., 1987; Kahan et al., 1988; Kanouse et al., 1989).

More recently, an IOM study committee examined OMAR's program and made several recommendations about its structure and functions (IOM, 1990d).3 The committee called for, among other things, greater emphasis on the concerns of users of consensus statements, with an acknowledgment that the program's fundamental purpose should be "to change behavior toward appropriate use of health practices and technologies" (p. 1). Other issues that the committee addressed were topic selection; better collection, analysis, and use of scientific data before a given conference; attention to dissemination strategies; continued experimentation and self- (or outside) evaluation; and appointment of an external advisory council to assist OMAR in setting its agenda. Again, the objective of these recommendations was to make consensus statements more usable and useful.

Interorganizational Cooperation: Medical Societies and Others

Moving beyond the internal use of multidisciplinary processes, several organizations are looking for opportunities to join formally with other groups in cooperative efforts. To date, collaborations appear to involve mainly physician organizations, although a few involve nonphysician professional groups, research organizations, and payers.

One notable effort aimed at training guidelines developers rather than developing guidelines per se has been sponsored by the John A. Hartford Foundation and the Council of Medical Specialty Societies. Activities have included a training course in which specialty society participants developed guidelines and an introductory manual for developing guidelines (Eddy, 1991c). Reflecting the challenges faced by those who work across specialty lines, workshops on resolving interspeciality conflicts have been another feature of this initiative.

Organizational cooperation, whether within or across professional boundaries, can serve several aims. They include the following:

  • greater efficiency through pooling of resources and expertise for methods development, training, and problem solving;

3  

In addition, in 1989 the IOM organized a workshop on international consensus development programs in conjunction with an annual meeting of the International Society for Technology Assessment in Health Care (IOM, 1990g). A group of workshop participants developed a lengthy set of recommendations about strengthening such programs. Procedures and methods figured prominently in those recommendations and presaged many of the points made by the IOM practice guidelines study committees. For example, these recommendations concern documentation, use of the best available scientific evidence (including meta-analysis where possible), monitoring and review to determine if recommendations need to be reassessed, and attention to information dissemination and evaluation at the outset of development.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
  • learning from shared experience;

  • better anticipation of user circumstances and concerns; and

  • development of commitment and support for implementation from individuals with varied professional and institutional affiliations.

One complex effort involving the American Medical Association (AMA), the RAND Corporation, and a consortium of academic medical centers has already been cited in Chapter 2. This effort has been multiorganizational less in its individual components—indicator construction, guidelines development, and guidelines testing—than in its attempt to create planned links among these activities. The complexity of coupling organizations (not just individuals from different disciplines) has made this project quite difficult to negotiate, execute, and maintain.

Two other multiorganizational efforts led by the AMA were also noted in Chapter 2: the Specialty Society Partnership, involving the AMA and 14 national medical specialty societies, and the Practice Parameters Forum, comprising national medical specialty and state medical societies. Two major objectives of the AMA and the groups working with it have been to devise criteria for judging the soundness of the process for developing practice parameters and then to establish a process for judging specific parameters according to these criteria and perhaps endorsing those that pass (AMA, 1990a). The first criterion is that guidelines should be developed by physician organizations. Reflecting the weight that most professional organizations place on individual professional judgment, the AMA's assessment effort concentrates on process and documentation; it assumes that expert health professionals will have ensured that guidelines correspond to scientific knowledge.

The long-standing collaboration between the Blue Cross and Blue Shield Association and the ACP has produced two seminal handbooks: Common Diagnostic Tests (Sox, 1987, 1990) and Common Screening Tests (Eddy, 1991a). These handbooks include papers that systematically analyze research, project outcomes, estimate cost-effectiveness, and recommend practices based on the strength of the evidence concerning each test. Each handbook concludes with summary recommendations (guidelines) intended to aid health benefit plans in making coverage decisions. The first handbook prompted considerable furor: it was hailed by the health services, technology assessment, and quality assurance communities and decried by at least some members of the practice community. Criticism soon gave way to acknowledgment of its major contribution to more effective and appropriate clinical decision making, and the major charge that has been made against the second edition is that not all the groups that wanted to be involved in its development were included.

Several other collaborative efforts can be cited. For example, the ACP, the American Academy of Ophthalmology, and the American Diabetes

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Association are cooperating on guidelines for management of diabetic retinopathy. The American College of Cardiology and the American Heart Association have had an ongoing collaboration for the past decade. This arrangement has produced nearly a dozen guidelines, which have been published in the Journal of the American College of Cardiology and in Circulation. The most recent have been for coronary artery bypass graft surgery and for implantation of cardiac pacemakers and antiarrhythmia devices; future guidelines are planned for cardiac catheterization and cardiac catheterization laboratories, electrocardiography, chest pain management in the emergency room, and cardiac radionuclide imaging (a revision of a guideline released in 1986).

PERSISTENT QUESTIONS FOR THE DEVELOPMENT PROCESS

Participation in the Process

As the guidelines development process evolves, more attention is being paid to who takes part in the process, when and how they participate, and what such participation should achieve. This interest reflects both the increasing sophistication of sponsors and the increasing visibility of the process and its products. Several persistent debates about participation can be identified, although the general trend seems to be to expand the scope of involvement. This subsection briefly reviews participative patterns to date; the next subsection addresses points about the development process from a more methodologic stance.

Creating Guidelines Panels and Selecting Panel Members

One issue related to panel selection is whether organizations assemble panels for each guideline or create standing groups. OMAR (for the NIH Consensus Development Conferences) and AHCPR establish independent panels for each guideline. RAND similarly creates a new panel for each technology, procedure, or condition for which it develops appropriateness indicators. By contrast, the Canadian Task Force on the Periodic Health Examination and the U.S. Preventive Services Task Force (USPSTF) have stable, standing panels and expert consultants and engage in a continuous process of revising previous recommendations and addressing new topics. The ACP process occupies the middle ground, with a standing oversight committee but selected experts who are engaged to develop guidelines on specific topics. Although no evidence exists on the subject, it is likely that each strategy is appropriate for different circumstances.

Principles for selecting members of guidelines panels differ along two major dimensions: (1) the generalist-specialist dimension and (2) the physi-

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

cian-nonphysician dimension. The former dimension has been a major concern of physician groups and involves three subdimensions: primary care versus specialist physician, community-based versus academic physicians, and specialists in related fields (for example, the role of a cardiologist in guidelines developed by thoracic surgeons). Questions of expertise and turf are not the only limiting factors: any group tends to find it easier to organize, communicate with, and rely on its own members than on nonmembers.

The second dimension of selection—physician-nonphysician—is debated by most groups that engage in guideline development. It tends to dissolve into two questions: Should other clinicians and health professionals, such as nurses, therapists, health educators, and nutritionists, be involved, and if so how and how much? Similarly, what should be the role of patients and consumers, payers, administrators, and public officials?

Beyond these groups, any number of other types of interested parties and experts may wish or need to be involved in the development of specific guidelines. Among those who might be considered, in the former case, are representatives of voluntary patient and disease groups or representatives of affected provider associations (e.g., hospital or home health agency associations). Involvement in the latter case might comprise expert clinical consultants, expert consultants in other disciplines (economics, law, outcomes measurement), and other methodologists (e.g., those skilled in meta-analysis).

The great majority of specialty organizations apparently rely on expert panels composed entirely of physicians in that specialty. Exceptions to this rule include AHCPR panels, which include primary and specialty physicians, nurses, selected allied health disciplines as appropriate, and consumers. NIH Consensus Development Conference panels may include Ph.D. researchers in addition to physicians and other types of clinicians. Similarly, RAND panels for developing appropriateness criteria and some ACP panels have gone beyond the specialty-specific approach. Site visits for this study suggested that institutional providers (e.g., hospitals, HMOs) that develop guidelines for internal use also are more likely to include different types of clinicians and health professionals.

Selecting Reviewers of Draft Guidelines

Identification of reviewers for sets of draft guidelines is another area in which groups may differ substantially in how they select participants—assuming that they have a process for reviewing draft documents at all. Debates about physician and nonphysician involvement tend to reappear at this stage. Nonetheless, whatever the position of an organization with respect to composition of guidelines panels, it tends at this stage to broaden its range of participants.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

The USPSTF, for example, sent its recommendations and draft background papers for review by more than 300 medical, public health, and "other" experts, including individuals in government health agencies, the U.S. Public Health Service, academic medical centers, and medical organizations. The recommendations were revised if a reviewer

identified relevant studies not examined in the report, misinterpretations of findings, or other issues deserving revision within the constraints of the Task Force methodology. The format of this [the Task Force's] report was designed in consultation with representatives of medical specialty organizations, including the American Medical Association, the American College of Physicians, the American Academy of Family Physicians, the American Academy of Pediatrics, the American College of Obstetricians and Gynecologists, the American College of Preventive Medicine, the American Dental Association, and the American Osteopathic Association (USPSTF, 1989, p. xxxvii).

The AHCPR guidelines are a special case because they are the products of nongovernmental panels supported with federal funds. The government's internal review of the guidelines examines only the process by which they were developed; an elaborate external review and pilot-testing process is being implemented to consider the soundness of the guidelines themselves. For the guidelines to be developed through the contracting mechanism, four drafts of guidelines are required. The third draft will be reviewed by an outside group of "peer reviewers," and the fourth (that is, the version produced after the peer review process) will be subjected to pilot-testing. Based on comments from the pilot-testers, a fifth and final version of the guideline is to be submitted to AHCPR.

Increasing concern about the practical needs of professionals is reflected in the recent activities of the American Society of Internal Medicine (ASIM) and its Internal Medicine Center to Advance Research and Education (IMCARE, 1990; Simmons, 1990). The center has created an innovative Guidelines Network that will not develop guidelines but instead organize network internists to review, upon request, the guidelines of other organizations. The intent is to provide greater insight into how well a guideline may work in clinical practice. For guidelines developed by a subspecialty but intended for use by all internists, network members offer broad-based feedback beyond the subspecialty.

More than 400 internists nationwide, including physicians in general and subspecialty internal medicine, have contacted the network about being volunteer reviewers. Furthermore, to broaden participation, internal medicine-related organizations are being asked to suggest additional volunteers. Although network members are primarily ASIM members, ASIM membership is not a requirement. IMCARE plans to establish an advisory panel on

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

an annual basis; 1991 appointments were announced in March (IMCARE, 1991).

The IMCARE guidelines network is informing AHCPR and other organizations of its availability to aid their guideline development or evaluation efforts. One early activity of IMCARE involved this IOM project. Specifically, the center staff organized an evaluation of the IOM's draft guidelines assessment instrument, which appears in revised form in Appendix B. That review produced 65 useful responses (and an overall summary) in a relatively short turnaround time.

Another review strategy is typified by the ACP's practice of publishing background papers and policy statements in the Annals of Internal Medicine. This opens the analyses and guidelines to very broad professional and scientific scrutiny. In general, the ACP has instituted a sort of "due process" by seeking the opinions of any agency, group, or individual with a potential vested interest.

Updating Existing Guidelines

"Scheduled review," one desirable attribute for practice guidelines, asserts that guideline documents should state when a guideline ought to be revisited and what information would trigger a detailed review and possible change in or withdrawal of the guideline. Generally, such statements put users on notice that the developer group may not or will not stand behind the guideline in its current form once the deadline has arrived. Given the acute sensitivity of professional organizations to advancing medical knowledge, the need for such a review process appears to be well understood, although in practice it may be implemented to differing degrees.

The General Accounting Office's (GAO, 1991b) survey of medical specialty societies found that most of the groups had discussed a process of periodic review and updating of guidelines but that not all had begun (or had even begun planning) such a process. Of those societies with plans or programs, seven planned annual reviews and one planned a 10-year review with earlier revisiting of the guideline if the need was clear. One society invokes a "sunset" provision by stating that guidelines will expire after 3 years and must be rewritten (unless they have been revised in the interim). As organizations continue to formalize their guidelines development activities, a typical goal is to establish a formal review process to determine if and when guidelines need updating or other action.

Formal updating activities can involve specifying a target review date when a guideline is first proposed and reinstating a former guidelines panel or appointing a new one. Alternatively, a periodic or rolling review process can be established that routinely covers all guidelines. The Canadian Task

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Force, for instance, reconvened in 1982, 1984, 1986, and 1988 to revise recommendations and evaluate new topics (USPSTF, 1989, p. xvi). A much lower level of activism is pursued by many organizations, which simply include in their guidelines a statement to the effect that the guidelines should be, or may need to be, modified as more information becomes available.

At a presentation to the committee in early 1991, J. Jarrett Clinton, the AHCPR administrator, noted that the AHCPR Forum expects that updating of at least some of its guidelines may be needed as often as every year. The Forum does not plan to conduct "continuous" review and updating but rather to institute a "regular" procedure. Overall, it aims to produce dozens of guidelines, resulting in a large family of guidelines; at some point, however, it will not be able to grow and simultaneously continue to revisit the entire set of guidelines, so some priority setting will have to be done and choices about updating will have to be made. In the meantime, the Forum expects to implement a form of scheduled review either by asking panels to state when their products should be revisited or by bringing the panel back to see how much has changed and to decide and recommend what to do. In some cases the difference between what would be needed for a new guideline and what exists in the current one might be sufficient to prompt the Forum to create a new panel.

These updating procedures can ameliorate but not resolve the dilemmas that arise if a guideline has not been revisited or revised, particularly if such a guideline has been accorded any special legal or regulatory status. Some believe that the idea of dating or withdrawing guidelines or otherwise implementing scheduled review procedures puts physicians and other clinicians in an impossible situation—using "definitive" guidelines one day and "expired" ones the next. Others contend that the dilemma of what authoritative information is available and should be used is precisely the situation health care providers find themselves in today. Arguably, definitive guidelines, even if they expire at some point, would be an improvement on today's environment.

AREAS FOR METHODS RESEARCH AND DEVELOPMENT

Several issues in particular appear to the committee to be worth further investment in methodological research and development. These issues involve the selection of topics for guideline development, processes for securing expert judgment and consensus, ways of assigning weights to the scientific evidence, outcomes measurement, techniques for determining and incorporating patient preferences, and means for identifying and evaluating inconsistencies and conflicts among guidelines on the same topic. The suggestions in the sections that follow reenforce the committee's general point that the methods and processes used in guidelines development can

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

significantly affect the credibility and utility of guidelines and the extent to which guidelines can achieve the objectives discussed in Chapter 1. Chapter 6 has already noted the need for further technical work to advance the field of cost-effectiveness analysis.

Topic Selection

The demands of guideline development on sponsor resources and the huge array of interesting topics make the identification of priorities and the selection of topics an inevitable and important issue. Any effort to establish a systematic process for these tasks faces certain questions of methodology. In particular: (1) Should the focus be chiefly on clinical conditions (broad or narrow diagnoses, illnesses, symptoms, or complaints) or on specific technologies (broadly defined to mean invasive or noninvasive procedures, medications, devices, and specific medical, nursing, or other health care practices)? (2) Should the priority-setting strategy rely more on expert opinion or quantitative data?

The Omnibus Budget Reconciliation Act of 1989 (OBRA 89) contained a fairly typical statement of broad criteria for selecting topics for guidelines development, at least for AHCPR assignments. These criteria seem to favor clinical conditions over specific technologies but take no explicit stand on expert opinion versus data. Per OBRA, AHCPR, in selecting topics, should consider practice variations, the potential for improving outcomes, and Medicare expenditures and program needs.

Many organizations list two criteria among their own priorities. One is the frequency of use of an intervention or the prevalence of the condition, and the other is the cost (to the nation or to a specific organization) of caring for that condition (or, possibly, using that technology). These criteria seem to stress quantitative data, perhaps assuming the use of expert opinion.

Even when explicit criteria are publicly stated, the process of choosing topics often remains relatively casual and uninformed by systematically collected, quantitative data. Choices may be posed and decisions made by program staff or oversight groups based on subjective judgments of what would be useful and what could actually be done, given the organization's resources.

Sometimes, topic selection is complicated by outside influences or interests. A case in point is the NIH Consensus Development Conferences. Topic selection in this case is made more difficult by OMAR's need to interest, or be responsive to, individual institutes within the NIH complex. Those institutes may well have their own criteria for topic selection, and they vary in their preferences for a focus on clinical conditions and a focus on technologies.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Paralleling the processes for developing guidelines are two ''competing" ways to set priorities for guideline development: consensus methods and quantitative (or modeling) approaches. These methods are marked by quite different techniques that are likely to produce different results (IOM, 1990j).

The consensus approach to priority setting synthesizes the judgments of a group of experts who have been asked to identify topics and rank them. In its ideal form the group would be well balanced in terms of the professional disciplines and characteristics of its members (as recommended for other aspects of the guideline development process), although this form of topic selection probably achieves the ideal only rarely, if ever. Consensus processes vary considerably in the extent to which they employ explicit criteria to guide judgments.

Phelps and Parente (1990) and Eddy (1989) have both described formal modeling approaches that rely heavily on quantitative data to inform priority setting. These approaches, too, have their drawbacks: they are necessarily limited by the availability and quality of data, especially data on important but difficult-to-quantify criteria, and by the degree to which assumptions are explicit and reasonable.

In general, a good—and attainable—priority-setting process would at tempt to establish explicit criteria for rating potential topics for guidelines or related assessments and to apply the criteria using available data; it would also include the systematic use of consensus methods when data were unobtainable. Further research, however, is needed.

The IOM is presently engaged in a follow-on study to its earlier pilot project (IOM, 1990j) of setting priorities for technology assessment and reassessment. The study is based on directives in OBRA 89 and funded by AHCPR; its charge is to advise the agency's Office of Health Technology Assessment (OHTA) on processes and criteria for priority setting. Among the criteria to be considered are the potential for an assessment to change health outcomes, the extent of variations in practice styles, the prevalence of a condition (or rate of use of a technology), the burden of the illness (for example, in terms of quality-adjusted life expectancy), costs (including total annual outlays for the condition or the technology), and various ethical, legal, and social issues. A report on IOM's proposed model and its relevance for both OHTA and other organizations that perform technology assessment was published in early 1992 (IOM, 1992).

Expert Panel Processes

A second major area for empirical investigation involves mechanisms for identifying and convening expert panels and similar groups and for deriving statements of expert judgment or consensus (Fink et al., 1984;

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

IOM, 1990g; Lomas, 1991). Group judgment techniques (e.g., pure and modified Delphi approaches, nominal group techniques) and related group-oriented activities (e.g., focus groups) have many uses beyond guideline development; some of these uses include technology assessment, consensus-building in areas outside health, sales and marketing, and the like.

Despite the long history of these approaches (some date back decades), many questions still remain about their reliability and effectiveness. For example:

  • What is the optimal size of a group? Depending on its procedures and objectives, an expert panel might have as few as 8 members (a small panel) or as many as 18 (a large one). It would be a mistake to organize a large panel and expect it to function well according to small-panel procedures. Group size affects group dynamics; the potential effect of group size on judgments and on group commitment to those judgments needs to be considered.

  • How should members be identified and appointed? Is it better to rely on personal networks and acquaintances to identify respected clinician leaders in a particular area, or to use nominations from professional associations and societies? Additionally, is it important always to have geographic and other demographic or professional diversity (e.g., practitioners from all regions of the country; rural, urban, and inner-city workers; representatives from fee-for-service and prepaid systems; experts from academic centers and from private practice)? How is the threat of bias or conflict of interest to be balanced against the need for experience and expertise?

  • Who should lead the panel? Among experts in group judgment techniques, debate still arises as to whether physician groups should be led by a physician or by a nonphysician facilitator. (The same issue arises for guidelines related to conditions or technologies involving nonphysician health professionals.) For any given guidelines development effort, the physicians who are candidates for the role of group leader may or may not be good facilitators and may or may not have biases that, whether spoken or not, would unduly shape group processes. On the other hand, a skilled nonphysician group leader may or may not be able to command respect from physician panel members and may or may not be sufficiently well versed in the clinical issues to recognize when problems are emerging or when clinical preconceptions are unduly influencing the discussion.

  • How should the deliberations and decision making of expert panels be structured? Various evocatively named approaches have been tried—for example, the "town meeting," "science court," and "science conference"—and both mail and face-to-face processes for defining questions and evaluating alternatives have been used (Lomas, 1991). Little research is available on the impact of different procedures on panel decisions. Even the question

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

of how to define consensus (e.g., through informal means or formal votes, using percentages or scaling techniques) has not been subject to much, if any, empirical testing. Similarly, the consequences of different strategies (e.g., explicit versus implicit) for defining and reaching consensus are not well documented.

Evaluating Scientific Evidence

The third set of methods issues involves techniques for assessing, rating, and combining scientific evidence. Guidelines developers normally must weigh the soundness and relevance of both direct and indirect evidence, evidence generated by processes of varying degrees of rigor, and studies that differ in design details and findings (Eddy, 1991b).

Problems tend to be most explicit for meta-analysis and related information-synthesis activities, for which investigators must formally combine information from (often completely) dissimilar articles and publications4 Mulrow (1987) and Thacker (1988) both provide useful introductions to meta-analysis and ways to combine evidence of differing types and quality. Thacker cautions that although meta-analysis is more explicit than traditional methods of narrative review of the literature and has other advantages, it may impart an "unwarranted sense of scientific validity" (p. 1688) of which consumers of such reviews (particularly those unfamiliar with the limitations of the statistical methods) need to be wary.

Most guideline developers do not elect to employ such complex techniques for weighting and synthesizing evidence, and many do not use any explicit approach. One simple but explicit rating scheme was pioneered by the Canadian Task Force and slightly adapted by the USPSTF (1989). This approach gives the greatest weight to information from randomized controlled trials (RCTs) and the least weight to simple case reports. It can also be criticized, however, for effectively placing more emphasis on poorly conducted RCTs than on well-done case-control or other quasi-experimental studies; in addition, the weighting scheme itself is an arbitrary one. More flexible but less arbitrary weighting schemes have been proposed and compared with more conventional approaches (Eddy, 1990k); one problem with

4  

Meta-analysis, a quantitative method to combine data, has several advantages over traditional narrative analyses of the literature on a clinical topic. In particular, it is a means for estimating the magnitude of effect of a given clinical intervention in terms of a statistically significant effect size or an odds (probability) ratio. More qualitatively, it forces those doing the literature analysis to think systematically about several aspects of the studies being reviewed and the data being pooled. The literature about meta-analysis (both substantive and statistical) has grown considerably in the 15 years since the term was first coined (Glass, 1976). Authoritative works include Hedges and Olkin, 1985; L'Abbe et al., 1987; Sacks et al., 1987; Oxman and Guyatt, 1988; and Berlin et al., 1989.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

these schemes, however, is that they are technically demanding. Notwithstanding that systematic clinical evidence is relatively limited and typically flawed in some respects, it is nonetheless important to find feasible ways to make the best use of that evidence.

Patient Outcomes

A fourth area of methodologic concern focuses on means of improving knowledge of patient outcomes, interest in which accelerated at about the same time but somewhat independently of the growing interest in guidelines development.5 IOM committees concerned with practice guidelines and with related issues (quality of care, effectiveness research, disability) have been particularly strong proponents of incorporating patient outcome information into clinical practice guidelines. Their recommendations are based on a broad conceptualization of outcomes that involves several dimensions of health status and health-related quality of life, especially those of direct importance to, and reported by, individual patients or their family members.

In this conceptualization, the dimensions of health status include survival and life expectancy, symptom states, physiologic states, physical function states, emotional and cognitive states, perceptions about present and future health, and satisfaction with health care. The last six of these seven domains of health status are particularly significant for guidelines aimed at conditions that are not life-threatening.

The field of health status assessment is becoming well established in the health services and health policy research arenas. (For a sampling of this literature, see Bergner, 1985; Lohr and Ware, 1987; McDowell and Newell, 1987; Lohr, 1988; Patrick and Erickson, 1988; Lohr, 1989; Mosteller and Falotico-Taylor, 1989; Tarlov et al., 1989.) In OBRA 89, Congress gave AHCPR the legislative mandate to conduct effectiveness research, chiefly through the Patient Outcomes Research Teams (PORTs) program described in Chapter 2. Acceptance and application of health status assess

5  

The Health Care Financing Administration (HCFA), with the guidance of former administrator William Roper, is generally credited with giving outcomes and effectiveness research a major boost through its proposed effectiveness initiative (Roper et al., 1988; IOM, 1990e). Paul Ellwood's Shattuck Lecture (1988) is also recognized as a major influence. (The InterStudy Outcomes Management System, a private initiative stimulated by Ellwood, was described in Chapter 4.) Also widely cited is the work by John Wennberg and his colleagues on practice variation (Wennberg and Gittelsohn, 1973, 1982; McPherson et al., 1982; Wennberg, 1984) and on outcomes of open and transurethral surgery for benign prostatic hypertrophy (BPH) versus "watchful waiting" (Wennberg et al., 1987, 1988; Barry et al., 1988; Fowler et al., 1988: Greenfield, 1989). HCFA's controversial efforts to produce hospital-specific mortality rate data on Medicare patients were an early attempt to provide the public as well as health care institutions with information on patient outcomes (Brinkley, 1986; HCFA, 1987-1991; Jencks et al., 1988a,b; Chassin et al., 1989).

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ment methods are also increasing in biomedical and clinical investigation (Wenger et al., 1984; Mor, 1987; Luce et al., 1989; Spilker, 1990). This work is not very well known to the practice community, however, and information it has assembled has not yet permeated the guideline development process.6 The GAO survey of medical specialty societies (1991b) found that some but not all guideline developers believed that outcome information should be incorporated in guidelines.7 How to do so remains a significant challenge to guidelines organizations, as reflected in a recent essay by Mulley (1991).

Several knotty problems can be cited. First, the significance, value, and validity of health measures based on patient self-reports have been questioned. Second, little empirical evidence exists because broad health status measures have not been widely used in clinical trials or technology assessments until recently. Third, amassing information on outcomes of all reasonable alternative courses of care is difficult, although information may be available on a specific intervention. Fourth, the unrepresentative populations used in most clinical trials limit the generalizability of findings to other populations such as women and minorities. This committee hopes that its strong stand on the need to incorporate estimates of patient outcomes will lead to more use of outcomes measures and further work to overcome their current limitations.

Patient Preferences

Chapter 5 discussed patient preferences in the context of concerns about ethics, costs, and informed decision making and noted the difficult conceptual and policy challenges this topic presents to developers and users of guidelines. A particular problem involves the weight to be given to patient preferences. Judgments may vary depending on the nature of the expected benefits and harms, their various probabilities of occurring, the degree to which patients truly understand the information presented to them, and the locus of responsibility for payment.

In addition to conceptual and policy challenges related to patient pref-

6  

In September 1991, the IOM convened a conference on advances in health status assessment that focused heavily on the use of health-related quality-of-life measures in clinical practice and clinical settings. The papers and discussions were aimed more at the practice community than at the research or policy making community; the proceedings will be published in mid-1992 in Medical Care (Lohr, forthcoming). A landmark set of papers concerning quality-of-life measurement in surgery has also appeared in the Journal of Theoretical Surgery (Neugebauer et al., 1991).

7  

The most ambitious project cited in the survey involved a society that proposed to estimate outcomes and to conduct the work necessary to assess actual outcomes over a three- to five-year period.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

erences, various practical difficulties arise for those seeking to apply guidelines in everyday practice. One approach is to incorporate patient preference "decision nodes" into algorithms and protocols that are devised from guidelines documents. These points would highlight for physicians and other caregivers points at which options for diagnosis and management should be discussed with patients or families and their preferences for particular options elicited.

An approach that focuses on how rather than when to elicit patient preferences involves the interactive videodisc technology developed at Dartmouth Medical School (Wennberg, 1990). Progress has been greatest on a videodisc for BPH; others on breast cancer and low back pain are under way. A patient sees, from the videodisc, a narrative about the main treatment options (a form of "guideline") and possible outcomes and complications (benefits and risks of those options). Because the information is presented in an interactive medium, it puts the patient in an active role and tailors the information presented to that patient's status, taking into account such factors as age and symptom severity.

The issue of patient preferences will stimulate further methodological development, research, and debate on at least three fronts. First, what are the best techniques for eliciting preferences? Second, what information and other conditions are required for informed and rational statements of preference? Third, what are the technical and policy issues in quantifying individual preferences (utilities) and aggregating preferences for purposes of making judgments about what care is appropriate (Mulley, 1989; IOM, 1991a)? Still, as much as methodological improvements and more research are desirable, they will not resolve policy debates about what preferences should be supported through coverage under public or private health plans and what should be the financial responsibility of the patients.

Conflicting or Inconsistent Guidelines

As the examples and discussion throughout this report make clear, the guidelines enterprise is a singularly pluralistic one. Furthermore, although the topics that might be addressed (clinical conditions, or technologies, or both) are nearly endless, the ones that tend to surface more frequently are those that affect large numbers of individuals, that have large costs associated with them, or that are mired in uncertainty or controversy. As a consequence, more than one group may well be working on guidelines on similar topics. Furthermore, these groups may take different approaches to the methodological issues discussed above, and some may apply cost-effectiveness or other criteria that others do not apply in making recommendations. This pluralism makes it likely that different groups will produce somewhat different and perhaps inconsistent and conflicting guidelines.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

The extent to which inconsistent guidelines are "a menace and an obstacle to improved care" as argued recently (Hinterbuchner, 1991, p. 5) rather than tolerable or possibly desirable depends on the reasons for inconsistencies. One approach to identifying and understanding inconsistencies and weaknesses in specific guidelines has been developed by Margolis, Gottlieb, and their associates (1991). This method, which they call algorithmic analysis, focuses on the completeness and consistency of clinical practice guidelines taken individually and in comparison.8 Hayward and his colleagues (1991) also have systematically compared several guidelines for preventive services.

Both sets of investigations found inconsistencies and conflicts in the guidelines they examined. Some guidelines were simply more complete, internally consistent, and specific than others. Other inconsistencies appear to reflect implicit or explicit differences in attitudes about the use of expert judgment, the risk of false-positives, the importance of costs, and other matters. Finally, some of the differences reflect the impact of new evidence that has appeared in the intervals between publication of different guidelines; for example, between the publication of the first and second asthma guidelines studied by Margolis and coworkers, the recommended sites of care for many specific problems had shifted from inpatient to outpatient settings.

Some committee members argued, when conflicting guidelines are encountered, for deliberately seeking areas of consistency, on the grounds that the stronger, more defensible points will be found there. Others on the committee noted that for certain conditions or technologies about which the science base is limited, conflicting guidelines may not necessarily be "wrong."

8  

In one test of this method on multiple sets of guidelines for measles immunization, breast cancer screening, and management of asthma, each set of guidelines was converted from free text (prose) to flowchart format (the algorithm map) and then restated as clinical rules (if-then statements). The first category of deficiency involved nonspecific terms and phrases, which generally were retained in the flowchart. Among these phrases were the following: "mammography every 1-2 years"; "unless medically indicated sooner"; "quarantine measures"; and ''supportive care." The second type of problem concerned nonspecific phrases that had to be better specified in constructing the algorithm map. Examples included "approximately age 75," which was changed to "age 75"; "at an earlier age," which was modified to read "between age 35 and 39"; and "nurses, nursing and medical students," which became "all employees with patient contact." The third and fourth categories covered missing items. Relevant missing items included recommending the teaching of breast self-examination but not recommending a specific frequency; providing overly abbreviated descriptions of contraindications to measles vaccine; and failing to indicate age ranges to which recommendations applied. The fifth category of deficiencies involved logical inconsistencies that can be regarded as errors. For example, a student in grades kindergarten through twelfth grade cannot have been born before 1957 despite an evident assumption to this effect in one of the guidelines (presumably the result of a failure to update the guideline).

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Sometimes conflicts and inconsistencies will direct attention to critical areas of desirable research to support more consistency and certainty in the future. In other cases, inconsistencies may prompt some groups to consider whether they have declared a position on a matter that might better be treated as a question with only a tentative answer at best. This stance may be psychologically disquieting for those who are accustomed to being (and who are expected to be) definitive.

A challenge remains in how to reconcile conflicting guidelines emanating from two authoritative groups, such as general internal medicine and a subspecialty society. It is particularly germane to certain settings or types of practice that must accommodate both clinical groups—for example, large multispeciality groups or health maintenance organizations. The solution here may be a form of disciplined accommodation in which both guidelines are rewritten in ways that are acceptable to both groups, probably by (1) strenuously seeking areas of agreement, (2) working to make rationales for differences explicit and susceptible to comparison with available evidence, (3) assessing the value of reducing practice variation in accord with the tenets of continuous quality improvement, and (4) allowing options to remain where a case can be made that evidence is inconclusive, professional consensus is split, and variation is unlikely to harm quality of care.

ISSUES AT THE INTERFACE OF DEVELOPMENT AND IMPLEMENTATION

At least three topics are not clearly development or methodological issues in the strictest sense: local adaptation of national guidelines, translation of guidelines into medical review criteria, and formatting and dissemination activities. In each of these areas, those who develop guidelines ought to be sensitive to the needs of users of guidelines so that they do not—through inadvertence, ambiguity, or lack of thoroughness—lead users astray. Conversely, those who are taking guidelines from the development to the application stage should be sensitive to the ideals of credibility and accountability stressed in Chapter 1.

Local Adaptation

Although national efforts to develop guidelines through public and private organizations have received considerable attention, local development and adaptation of national guidelines appear to be both widespread and little examined. This committee does not have extensive documentation of the extent, quality, and impact of local efforts, but it believes there is reason to be concerned about these efforts. The possibility for conflict and inconsistency simply multiplies, if and to the degree that local provider institu-

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

tions, medical societies, and the like adapt and modify guidelines to make them more locally appropriate or palatable.

Reasons for Local Adaptation

From site visits, anecdotal literature, personal experience, and similar sources, the committee identified many ways in which guidelines developed by national professional societies and other groups may be adapted, changed, or even ignored to serve the purposes of particular groups. Such adaptations are termed "local" even when the adjustments serve a multihospital system, nationwide network of HMOs, or other similar groups.

The reasons for adaptation, which are not necessarily defensible, may involve one or more of the following: (1) weaknesses in the guidelines or their dissemination that have no particular relation to local circumstances; (2) specific local or organizational objectives or constraints; (3) strategic judgments about the need to secure practitioner acceptance; and (4) protection of habit or self-interest. In addition, when local organizations face inconsistent guidelines, their choices will almost certainly lead to departures from one or more of these guidelines. For example, a group might choose to follow the ACP in not recommending a baseline mammogram but then follow the American Cancer Society guidelines for screening every 1 or 2 years for women aged 40 to 49.

Local adaptation might also be done on the basis of a thorough decision analysis of a "national" guideline. Such analysis might clarify how patient care options can legitimately differ depending on prevalence of disease, diagnostic test performance, physician skill level, costs, or other variables that might vary from locality to locality. If the "adapted" guideline were based on such an analysis of a national guideline that made such distinctions, it might then be more easily justified than if it had no such analytic underpinnings.

By contrast, some national guidelines are poorly drafted and give inadequate attention to their applicability to specific patient problems, identification of foreseeable exceptions, and clarity or precision. Thus, hospital medical staff who want protocols to guide treatment of patients with particular conditions may have to "fill in" when guidelines lack the specificity needed to make decisions about individual cases. Although such adjustments may be initiated by local organizations, the rationale need not be organization or community specific.

Other adjustments will reflect local conditions because even generally well-developed guidelines are unlikely to foresee or accommodate all the varying characteristics and objectives of potential users of a set of guidelines. For example, if the prevalence of a problem (e.g., human immunodeficiency virus infection) in a community or the characteristics of a partic-

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ular patient population (e.g., mobility of migrant workers) differ dramatically from the situations contemplated by a set of guidelines, some modification in recommended preventive, diagnostic, or treatment regimens may be reasonable.

Similarly, some health delivery systems and institutions may face constraints that are unchangeable in the short term. These constraints might involve regulatory prohibitions, lack of equipment, or shortages of personnel. Such problems may prompt adaptations that define protocols for situations in which care must be provided but the most appropriate course of care is impossible to implement.

Chapter 6 recognized that national guidelines may not incorporate judgments of cost-effectiveness, which some organizations believe they must have to allocate limited resources in a manner consistent with their objectives and environments. Other organizations may seek to apply continuous quality improvement precepts to narrow variations in practice. The result of both these policies may be guidelines that exclude certain options, on the grounds that they are too costly relative to their benefits, or that delineate specific "pathways" or "protocols" that are less variable than those described in a set of national guidelines. A typical example of a narrowing in guidelines occurs when an organization or a public agency (e.g., a state Medicaid program) creates a drug formulary that does not include all of the drugs that are considered reasonable options for treating certain problems. Depending on the extent to which an institution intends to constrain its financial liability for the use of costly but optional forms of care, patient preferences may be accorded greater or lesser weight than they are in national guidelines.

Another rationale given for the adaptation of guidelines is behavioral. Some argue that it is important to secure practitioner (and, less commonly, patient or enrollee) acceptance of guidelines through participation in their adoption. Some departures from national guidelines are viewed as acceptable when it is thought that such variation will lead to the actual use of the most critical elements in guidelines rather than to their rejection. The committee had mixed feelings about this rationale, and this discussion should not be seen as a justification for wholesale or casual departure from well documented, science-based guidelines for clinical practice.

Finally, generally unstated rationales for local adaptation may be to protect professional habits and local customs for their own sake and to protect economic self-interest by endorsing unnecessary care or care that others could provide as well or more economically. For example, a guideline that did not specifically limit the type of practitioner who could perform certain kinds of eye examinations might be reworked to restrict the practice only to physicians or to particular specialists. Committee members were distinctly unsympathetic to such practices and to rationales for guide-

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

lines that were covertly designed to protect habit, "turf," or income at the expense of patients and those who pay for their care.

In short, the main concern here is with fundamental departures from an existing scientifically based, well-documented set of guidelines. Among such changes would be designating certain practices appropriate when national guidelines define them as inappropriate, labeling a practice optional rather than recommended or vice versa, or changing threshold values for making treatment decisions.

When local institutions do adapt national guidelines, one useful step might be for them to notify the originating group and to explain the circumstances that led to their modifications. Whether national guidelines could or should be revised to accommodate or recognize these circumstances will depend on the specifics (for example, the likelihood that the same circumstances will occur more generally). If this process of communication and consideration became established, it would provide an ongoing—if not always systematic—source of feedback for revising and improving guidelines.

Processes for Local Adaptation

Local programs to adapt guidelines vary greatly in the formality of their processes and structures, but they appear generally to be a less sophisticated, less rigorous kind of effort than that endorsed by this committee. One effort located toward the sophisticated, science-based end of this spectrum is the work by Group Health Cooperative of Puget Sound (GHCPS) to develop a preventive care manual for its primary care practitioners. This activity, described in Chapter 6, reflects a specific objective (focusing resources on high-risk groups) and specific organizational characteristics (for example, an enrolled population and integrated patient records). One particular task for GHCPS has been to reconcile or choose from among inconsistent guidelines from different sources, although the materials available to this committee do not fully explicate the basis for different choices.

The less scientific, more behavioral or strategic approach is represented by one of the groups visited by the committee. This organization was developing guidelines or pathways for the care of patients admitted for certain common clinical procedures. Those involved did not employ a systematic process to identify and assess the scientific literature, estimate health outcomes, explain the rationale for the pathway, or document these steps. The pathways were presented as charts to advise clinicians on generally desired practices and to reduce variability in patient care. Although this last process did not incorporate systematic use of the scientific literature on a clinical problem, it was systematic and data oriented in that it identified topics for pathway development based, in part, on the variability

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

in existing practice patterns. Practitioners received periodic reports on how their performance compared with that of their peers and with the pathway.

Within a framework such as that offered by continuous quality improvement, empirical and incremental testing and modification of guidelines may well be appropriate (indeed, even necessary). Such testing may not conform to the highest standards of experimental research design, but it can provide a systematic, practical, and direct means of identifying where guidelines—as well as clinical practice—may need modification. Ideally, this kind of local but systematic information will become part of the broader evolutionary framework for guidelines development and improvement as national and local groups develop communication and tracking mechanisms.

Other local processes may be fairly unsystematic. They involve no analysis of local patterns of care, no explicit formulation of objectives, no literature review, no formal decision making processes, and no documentation of evidence or rationales for decisions. This method might be called a "back of the envelope" approach to guideline development.

Even when the rationale is worthy, this "back of the envelope" approach to adapting or developing guidelines (or medical review criteria) is unacceptable. It offers too much leeway, on the one hand, for uncritical accommodation of local traditions and narrow self-interest and, on the other, for excessive and unwarranted interference with physician-patient decision making.

The Standing of "Adapted" Guidelines

Adaptation processes intended to win physician acceptance of guidelines—the behavioral rationale for adaptation—should be guided and constrained by an expectation that the resulting guidelines and criteria will still be credible in their process, rationale, and documentation. The requirement for systematic and careful procedures applies as well to de novo development activities and efforts to devise medical review criteria. Where carefully developed and documented "national" guidelines exist, local adaptation processes should provide explicit rationales for changes that relate to specific, well-defined local conditions or objectives.

If national guidelines are in one way or another accorded legal stature with respect to malpractice liability (or immunity from liability), then serious attention must be given to the stature of guidelines that are modified to suit local circumstances or preferences and, possibly, to the criteria used to evaluate the quality of care that is rendered. Even if it can be shown that these derivations of existing national guidelines were arrived at through procedures similar to those that produced the original guideline, they may or may not enjoy the same legal stature as the originals. Although the "respectable minority" doctrine described in Chapter 5 could accommodate

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

some differences, it would be troublesome were it to justify departures from guidelines that are based on strong scientific evidence and consensus. Given the evolving views about the relationship between malpractice and guidelines in general, this issue is quite speculative at this time.

Formatting and Dissemination

For the purposes of this report, effective formatting means presenting guidelines in physical arrangements or media that can be readily understood and applied by practitioners, patients, or other intended user groups. Effective dissemination means delivering guidelines to their intended audiences in ways that promote the reception, understanding, acceptance, application, and positive impact of the guidelines. For the purposes of this discussion, effective dissemination presupposes effective formatting, and the discussion centers on the former. Appendix A discusses and illustrates some approaches to formatting guidelines.

Dissemination is in part an answer to the question: "Suppose I want a guideline for something. What do I need to do to find it?" Two broad possibilities exist. First, organizations currently producing guidelines probably have distributed them or related materials, and the questioner may well have filed the documents so that they can be retrieved. Second, the relevant guidelines may have been acquired by a general information resource such as the National Library of Medicine (NLM) and entered into a data base that can be queried on a wide array of topics.

Sponsors and developers of guidelines usually take responsibility for their initial dissemination to major target audiences, often either physicians or nurses. For example, many specialty societies, such as the ACP and the American College of Cardiology, begin their dissemination efforts by publishing individual guidelines in their journals, which all members receive. The GAO survey (1991b) reported that societies also publish in newsletters, the journals of other societies, and other places. For some types of guidelines, particularly for collections dealing with similar clinical issues, the initial step may be direct distribution of the guidelines to members. The American Academy of Pediatrics does this every other year with its Report of the Committee on Infectious Diseases, familiarly known as the Red Book (AAP, 1991). Specialty societies may also distribute guidelines to other societies, to federal agencies, and to selected audiences in the health care and medical education communities. Press conferences and press releases may accompany such publications.

One strength of these kinds of dissemination activities is that they are part of an ongoing process. They have an institutional past and a future that should help build both awareness and acceptance, at least among members of the sponsoring organization and eventually among outsiders as well.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Following initial dissemination steps,9 guideline developers may proceed with an array of activities such as cooperating with other interested parties in disseminating information to patients or consumers. This is where the second response noted above comes into play. The lay press, patient groups, computerized information systems, and directories may begin to make guidelines more widely available or known to practitioners, patients, and others. As noted earlier, the AMA publishes quarterly update listings of guidelines developed by both the AMA and specialty societies. In addition, publications are emerging that reprint or summarize selected guidelines or otherwise report on the field; the Report on Medical Guidelines & Outcomes Research, published by Health & Sciences Communications and now nearing the end of its second year, is an example. The NLM, as described elsewhere in this report, will store, index, and otherwise make available information on practice guidelines, specifically including those from AHCPR panels.

Those involved in the development and use of guidelines are paying increasing attention to a series of strategic "who, what, why, when, and how" questions. Specifically: Who do you want to reach and why? What do they need? How quickly do you want to reach them? What relevant techniques are available, and how do they vary in effectiveness and cost? Answers to these questions will influence some dissemination decisions such as whether to use professional or mass media, direct mailings, or journal publication. The length and complexity of the guideline will also influence the choice of dissemination technique. As noted in Chapter 4, options for dissemination now include a variety of computer-based tools including on-line literature search systems, floppy disks, and CD-ROM disks.

Other decisions will be contingent on a variety of environmental factors. What are the opportunities for dissemination and application within the intended audience? What are the barriers? How can different dissemination strategies be combined and coordinated with other implementation strategies to increase the probability of effective application of guidelines? Answers to these questions will yield ideas about who else will be or should be involved in dissemination, whether it should be a one-time effort or a continuous process, and what resources are needed. Many of the issues raised in the discussion of education in Chapter 4 will apply here as well.

Several specific factors related to dissemination might be considered legitimate and realistic concerns of guidelines developers, even if develop-

9  

In addition to publishing guidelines (in various media) and generally publicizing the availability of the guidelines document, disseminating organizations may also respond to requests for and inquiries about the guidelines and undertake similar tasks. Dissemination should also be understood to include any efforts needed to inform users of mistakes ("errata" or corrections, in publishing terms) and to advise users that existing guidelines are being withdrawn or revised.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ers do not actually carry out dissemination activities. Among these are characteristics of the target audience, timeliness and number of dissemination efforts, and the planned publishing, publicizing, and distribution of the guidelines. Depending on the combinations of these factors, dissemination activities might be considered relatively narrow and weak or relatively broad and robust.

The number of independent dissemination efforts—for instance, a one-shot announcement or several sequenced activities—may also influence the eventual result of the impact. Again, developers may need to be aware of plans in this area so as to be available for comment or interpretation over a longer or shorter term.

The impact of the guideline might also be affected by the timeliness or urgency of the dissemination effort. For example, some guidelines might be rushed into print in a special journal issue or put on a fast-track publication schedule; others may be published in a more routine manner. Developers may need to be sensitive to the significance of their work so that they can accommodate it to the demands or expectations of such schedules. In addition, the nature of the publication(s) may have implications for what guideline developers do (and for their length of service on a guideline panel). Guidelines may appear in their entirety, as synopses, or both; furthermore, they may appear in different formats and languages.

The AHCPR guidelines are a case in point.10 As this report was being prepared, the agency was planning to produce three versions of the guidelines aimed at the professional community: (1) the full technical guideline plus all documentation (biosketches of panel members, description of the processes followed, results of the literature review and analysis, recommendations, references, etc.); (2) a shorter version that includes the full set of recommendations and the entire bibliography; and (3) a pocket-sized, "quick reference" version that summarizes just the recommendations. (These have been referred to variously as "Papa Bear, Mama Bear, and Baby Bear" and the "500-page, 50-page, and 5-page" versions.) The agency appears to be focusing its broadest dissemination efforts on the shortest version as the one most likely to be sought out or read, once it has been noticed. For some topics or conditions that cut across all age groups, these three types of publications will be produced separately for adult and pediatric populations. Plans also call for consumer versions of at least the smallest version. Finally, editions of the consumer brochure in both English and Spanish are planned.

At least one of the shorter versions (probably the medium-length one) will be available through the NLM's on-line capabilities. Those who request the longest (full) technical document from AHCPR's Center for Re-

10  

Dissemination activities will be handled by the Center for Research Dissemination and Liaison at AHCPR, not by the Office of the Forum.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

search Dissemination and Liaison will receive it by mail, although whether the Center will make it available free of charge or for a nominal amount is not yet decided. The NLM will probably forward orders for the full document to the Center for handling.

"Version-specific" dissemination plans are still under discussion. The Journal of the American Medical Association may publish the announcement of the guideline and the shortest, clinician version of it; AHCPR will encourage relevant specialty societies to announce the guideline as well. Some thought is being given to dissemination of the consumer version through mass print media, such as Good Housekeeping, Ladies Home Journal, and the like. Other avenues of dissemination being considered include an 800 telephone number for inquiries (1-800-358-9295); other, more sophisticated marketing strategies are also being explored.

Publicizing the guidelines, as contrasted with publishing them, may be another activity to which developers should be attentive. Public relations and marketing activities in such cases might range from the printing of an announcement of the availability of the guidelines, to a formal press release, briefing, or conference,11 to announcements broadcast through newsletters, journals, and computer bulletin boards, to even more elaborate strategies and combinations of strategies.

A final set of decisions concerning the distribution of guidelines may have little direct effect on what developers do but may well affect the long-run impact of what they produce. These decisions involve the question of whether guidelines documents (or synopses, or both) are made available free of charge or at some price (and, if the latter, what that price might be). For example, guidelines developed under AHCPR auspices and made available through the NLM may be free of charge except for the nominal charges of the NLM for connection times to the relevant bibliographic and retrieval services.

Dissemination of information or guidelines is by itself insufficient to induce use of that information or to change behavior; indeed, excessive distribution of information to physicians or other clinicians can lead to a significant problem of information overload with no redeeming change in practice patterns or habits. Nonetheless, bringing guidelines to people's attention, and making them available as requested or required, are precur-

11  

The production of the first three AHCPR guidelines was accorded such significance that as of late 1991, plans were being developed to convene a press conference at which the Secretary of Health and Human Services would present at least the first of the guidelines (on postoperative pain management). Chairs of the panels and staff of the AHCPR and the Forum would be present and representatives of relevant specialty societies and professional associations would be invited to give statements concerning at least the aim of the effort and the process followed.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

sors to more direct efforts to influence behavior. Recognition of that reality and appropriate planning for dissemination are thus important components of what guideline developers need to do in the future.

Evaluating Impact

If formatting and dissemination operate at the interface at which development begins to shift over to implementation, then evaluation operates at the interface at which the results of implementation are fed back to improve and revise guidelines. Although evaluation of the impact of guidelines is not fundamentally a task for developers, the latter can be presumed to have at least an interest in learning what effects, if any, their work has had. Some groups may, in fact, have sufficient concern about what influence their guidelines are having to carry out various evaluation efforts; others may simply cooperate with outside evaluation activities. This chapter briefly raises the subject, therefore, on the grounds that those in the business of developing guidelines will have concerns about, if not direct involvement in, assessing the effects of their efforts.

As professional societies, public agencies, and others assess their involvement in developing guidelines, they eventually face questions about results. For example, do practitioners, patients, payers, and others even know the guidelines exist? Do they think they are credible and usable? Do they, in fact, use them? How do they affect patient decisions and behavior? Are guidelines having any impact on health outcomes, payment decisions, medical liability, costs, or other factors?

In general, groups have confronted these questions after they have developed several guidelines and have not built evaluation of impact into their programs (Audet et al., 1990). This approach is beginning to change, however, as organizations consider whether their financial and volunteer resources are being constructively used. For example, the GAO (1991b) survey reported that at least four medical societies were interested in evaluating the impact of their guidelines. The ACP self-evaluation of the CEAP activity has already been noted.

Focus groups and surveys are relatively inexpensive means of evaluating results, but they are also relatively weak research strategies in a world where the double-blind randomized clinical trial is the ideal. This report has described one randomized clinical trial involving the use of computer-based reminders for preventive care (McDonald et al., 1984); at least one other similar trial involving hospital admission testing guidelines has been planned (Audet et al., 1990). As noted in Chapter 4, some research has attempted to compare the results of different strategies for informing and educating practitioners about guidelines.

A recently completed but not yet published evaluation sponsored by a large managed care organization examined several questions (Audet et al.,

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

1990): (1) Does practitioner participation in the process of guideline development affect subsequent use by practitioners of the guidelines? (2) Do guidelines decrease resources utilization? (3) If guidelines do reduce utilization, is it at the expense of quality of care, as reflected in patient outcomes? Preliminary results indicate that ''physician generated guidelines codified parsimonious practices, which had a salutary and not negative effect on patient outcomes" (Greenfield, 1991). However, physicians who developed guidelines were no more likely to change practices (for example, to order diagnostic tests more conservatively) than were those who were not involved. The organizational response to these preliminary findings is that involvement in guidelines development is not a sufficient stimulus for change and that they must become part of an integrated quality improvement strategy.

Efforts to evaluate the impact of the guidelines development require both interest and resources. As expensive and methodologically demanding as guideline development is, evaluation of the impact of guidelines is even more demanding. Partly for this reason and partly because the guideline development enterprise is still relatively young, evaluation projects are likely to remain relatively uncommon. To the extent that evaluations are undertaken, they may have more in common with models of program evaluation than with models of clinical biomedical research.

One organization with a clear mandate to undertake evaluation of guidelines is AHCPR. Under OBRA 89, it is required to determine the impact of its first three guidelines on the cost, quality, appropriateness, and effectiveness of health care and to report these findings to Congress by January 1, 1993. As of late 1991, AHCPR's attention was solidly focused on development of guidelines and related medical review criteria; none of the guidelines due by January 1, 1991, had yet been published. Possible activities were still under discussion, and no formal research plan had been made public. (The effort to develop review criteria, described in Chapter 5, includes some provisions for testing their use and impact.) This lack of progress on impact evaluation is not surprising, given the unrealistic deadlines faced by AHCPR.12

The agency's 1993 report to Congress will be a status report of activities in progress and planned; with respect to the actual impact of guidelines, some proxy measures (e.g., media citations) may be generated. In addition, lessons learned as the initial guidelines panels pretested their draft guidelines will be a form of impact evaluation. Among the proposed activities

12  

The timetable is unrealistic for several reasons. First, the guidelines will probably not have had time to make a measurable impact on health, cost, or other outcomes; this would probably be true even if the first three had been published on schedule. Second, even if the guidelines have fairly immediate effects, the data to document such effects will generally be unavailable. For example, insurance claims or other data showing changes in the use of procedures or practices may not be accessible in the time frame specified. Likewise, data on patient outcomes will take time to collect.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

will be a set of internal projects, grants, and contracts with a mix of short and longer-term objectives—for example, to consider the impact of guidelines for preventive services in inner cities, to investigate the impact of dental surgery guidelines, and to evaluate interactive videodisc technologies to encourage behavioral change (Linda Demlo, AHCPR, personal communication, October 1991).

In its 1990 report, the IOM noted that explanations of policy success or failure, in general, needed to consider the following:

  • the validity of the policy premises—for example, the assumption of many policy makers that broader development and use of practice guidelines will achieve significant cost savings;

  • the quality of the implementation process—for example, the extent to which information was disseminated or incentives were created for the use of guidelines;

  • the existence of countervailing events—for example, court decisions limiting the ability of health care organizations or payers to review the appropriateness of care and then deny either practice privileges or payment for practitioners providing inappropriate care; and

  • the nature of supportive or enabling conditions—for example, the breadth of professional interest in the topic covered by the guidelines or a technical breakthrough in access to computer-based information systems.

Even groups that cannot contemplate rigorous evaluation may benefit by considering what would be required to evaluate their guidelines. What would they consider success? What potential adverse consequences should be tracked? What information about the clinical problem, the patient's circumstances and preferences, and the delivery setting should be recorded to permit later evaluation of the processes and outcomes of care? What confounding factors should be considered? Are there intermediate steps that might be usefully monitored? Short of full-scale evaluation, what might users of guidelines do to assess short- or long-term results? Might these users be encouraged to undertake some evaluation on their own or perhaps in collaboration with the guidelines development group?

Some attention to these and similar questions may help developers of guidelines identify previously unsuspected opportunities for evaluation. It may also intensify their interest in finding resources to support evaluation and to refine the way they approach the process of developing guidelines.

SUMMARY

More resources and more systematic procedures do not guarantee good guidelines, but the committee reiterates that guidelines development is a serious enterprise that deserves careful planning and execution. The com-

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

mittee observed several promising efforts at improving structures and processes for developing guidelines, strengthening methods, and incorporating more attention to implementation, evaluation, and revision.

This last step is critical to effective, comprehensive application of guidelines. The attitudes, needs, and circumstances of practitioners, patients, and other users of clinical practice guidelines must be anticipated and considered from the earliest stages of guidelines development, if guidelines are to be applied to achieve their goals. Likewise, evaluation issues—the intended effects of guidelines, means of measuring impact, potential confounding factors—have to be considered when guidelines are being framed rather than dealt with after the fact.

The next, concluding chapter of the report brings together this committee's principal conclusions and recommendations about the clinical practice guidelines enterprise. It does so in some comfort with the progress that the field has made in recent years, taking it as a good omen of the progress that can be made on the many conceptual, practical, methodological, and political challenges that still remain.

Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 163
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 164
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 165
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 166
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 167
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 168
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 169
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 170
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 171
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 172
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 173
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 174
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 175
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 176
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 177
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 178
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 179
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 180
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 181
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 182
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 183
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 184
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 185
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 186
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 187
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 188
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 189
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 190
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 191
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 192
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 193
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 194
Suggested Citation:"7 EVOLUTION IN PROCEDURES AND METHODS FOR DEVELOPING PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 195
Next: 8 A FRAMEWORK FOR THE FUTURE »
Guidelines for Clinical Practice: From Development to Use Get This Book
×
Buy Paperback | $110.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Guidelines for the clinical practice of medicine have been proposed as the solution to the whole range of current health care problems. This new book presents the first balanced and highly practical view of guidelines—their strengths, their limitations, and how they can be used most effectively to benefit health care.

The volume offers:

  • Recommendations and a proposed framework for strengthening development and use of guidelines.
  • Numerous examples of guidelines.
  • A ready-to-use instrument for assessing the soundness of guidelines.
  • Six case studies exploring issues involved when practitioners use guidelines on a daily basis.

With a real-world outlook, the volume reviews efforts by agencies and organizations to disseminate guidelines and examines how well guidelines are functioning—exploring issues such as patient information, liability, costs, computerization, and the adaptation of national guidelines to local needs.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!