Click for next page ( 2


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 1
Summary In the Omnibus Budget Reconciliation Act of 1989 (P.L. 101-239), Congress created the Agency for Health Care Policy and Research (AHCPR). One mission of the agency through its Forum for Quality and Effectiveness in Health Care was to sponsor and encourage the development, dissemination, and evaluation of clinical practice guidelines. Reflecting concerns about the Forum's initial choice of guidelines topics, the 1992 legislation that reauthorized the agency directed it to report to Congress in June 1995 on "optimal methods for setting priorities for guidelines topics" (P.L. 102-410~. The AHCPR, in turn, requested guidance from the Institute of Medicine (IOM). This report presents the Institute's analyses and recommendations as developed by a formally appointed study committee. STUDY APPROACH To undertake the study, the IOM appointed a 12-member committee with ex pertise in Sidelines development and implementation, health services research, health care delivery, and health policy. The committee met twice, once in conjunction with an invitational workshop. It also commissioned three background papers, reviewed relevant literature, and examined priority-setting processes used by other public and private organizations. In assessing processes for setting priorities for guidelines development, the committee considered five principles. The first four were consistency of the process with the organization's mission, implementation feasibility, efficiency, and utility of results to the organization. The fifth principle, which is particularly 1

OCR for page 1
2 SETTING PRIORITIES FOR CLINICAL PRACTICE GUIDELINES important for a public agency, is essentially a corollary of the first: the priority- setting process should be open and defensible. For purposes of this report, clinical practice guidelines are "systematically defined statements to assist practitioner and patient decisions about appropriate health care for specific clinical circumstances." They provide clinical advice (and sometimes ethical, organizational, and other advice), and they may take many forms such as algorithms, computer-based protocols, and policy documents. Technology assessments present information on patient care alternatives for patients, clinicians, and others, or, more narrowly, they provide analyses of individual technologies as "input to decisions" regarding insurance coverage, reimbursement policies, or equipment purchases. Medical technologies include "drugs, devices, medical and surgical procedures, and the organizational and supportive systems" employed in patient care. Although the line between technology assessments and practice guidelines is not sharp, the former are more often focused on individual technologies, intended to support coverage or purchase decisions rather than clinical decisionmal~ing, and prepared by staff rather than expert panels. One reference point for this study was the IOM's 1992 report Setting Priorities for Hearth Technology Assessment: A Model Process. That study was prepared for another unit of AHCPR, the Office of Health Technology Assessment (OHTA), which was originally created to provide assessments that would inform Medicare coverage decisions. Much of this report draws comparisons and contrasts between the priority-setting processes used by the two AHCPR units as a basis for considering future directions for the Forum. FINDINGS AND RECOMMENDATIONS: PRIORITY SETTING The procedures for the selection of AHCPR's first guidelines topics were relatively informal and driven by a tight legislative timetable. The agency has since developed a more formal process. The committee concluded that the current priority-setting approach used by the Forum (Figure 1) is relatively open, fairly explicit, and generally defensible. The committee also concluded that the model process recommended in the 1992 IOM report on technology assessment provides a clear and reasonable alternative framework for an organization willing and able to commit resources for the required data collection and analytic steps. OHTA has provisionally adopted the model process with some modifications. As AHCPR tracks and evaluates the experience of OHTA, the agency will be able to assess the process's strengths and limitations (e.g., in terms of efficiency and usable results), identify ways to improve it, and decide whether and to what extent the process should be adopted or modified by the Forum.

OCR for page 1
SUMMARY ACTIVITY 1. Select priority-setting criteria. RESPONSIBLE PARTY Congress, AHCPR Forum 2. Solicit nominations of topics ~ AHCPR Forum for guidelines development. 3. Use expert practitioner groups to nominate and rank additional topics in selected condition areas. 4. Screen nominations against priority-setting criteria, taking resources and existing guidelines into account. AHCPR Forum, private sector AHCPR Forum 5. Solicit comments on possible ~ AHCPR Forum topics in the Federal Register. 1 1 6. Review comments and propose ~ AHCPR Forum list of priority topics. 7. Designate final topics. 3 AHCPR Administrator -'1 FIGURE 1 Process for setting priorities for guideline development, Office of the Forum for Quality and Effectiveness in Health Care. SOURCE: Adapted from AHCPR, 1993. Format adapted from TOM, 1 992b. In the meantime, the committee agreed that the Forum should move forward in systematizing and improving its current priority-setting process. OHTA should also consider some similar steps.

OCR for page 1
4 SEWING PRIORITIES FOR CLINICAL PRACTICE GUIDELINES Criteria and Their Application Both the Forum and OHTA have made public the criteria they will apply in selecting topics for guidelines development and technology assessment, respectively. Although the criteria now used by the two organizations do not differ radically, the committee believes that more commonality in criteria, definitions, and measures is desirable. Where the Forum and OHTA differ, the differences should be explained by differences in organizational responsibilities or resources. The committee recommended that six general criteria be applied in considering topics for either guidelines development or technology assessment. These criteria are prevalence of the clinical problem (number of affected persons per 1,000 persons in the general U.S. population); burden of illness imposed by the problem (individual mortality, morbidity, or functional impairment); cost (cost per person of managing the problem); variability in practice (significant differences in utilization rates for prevention, diagnosis, or treatment options); potential of a guideline or assessment to improve health outcomes (expected effect on health outcomes); and potential of a guideline or assessment to reduce costs (expected effect on costs to sponsoring organization, other relevant agencies, patients and families, and/or society generally). Unfortunately, the data used to score particular topics on these six criteria will often be incomplete, either because data are not available or because they are too expensive to collect. To limit the distortions that may arise from reliance on obviously incomplete data (e.g., using only mortality data to measure burden of illness), the committee agreed that the agency's process for setting priorities should provide an explicit opportunity for important unmeasured factors to be considered, perhaps by using expert estimates in lieu of data. The Forum now considers as a criterion for topic selection the potential for a guideline to reduce significant clinical variation in the use of services. The committee concluded that the emphasis should be on the potential for a set of guidelines to influence behavior in ways that improve patient care outcomes or increase efficiency without harming patients. On occasion, however, the Forum may reasonably proceed with guidelines when the scientific evidence is clear that an alternative to current practice is preferable-even if it appears that clinicians or patients may resist such guidelines at first. Guidelines may, in such circumstances, be viewed as an initial, educational phase of a long-term effort to change behavior and improve outcomes. The OH1 A has stated that the availability of scientific evidence on a topic should not affect whether a topic is chosen for technology assessment but only how it is assessed (for example, by relying entirely on expert opinion). Although the committee understood the rationale for this position, it noted that one of the comparative advantages of the AHCPR as a health services and outcomes research entity is that it is better situated to mobilize expertise to analyze

OCR for page 1
SUMMARY 5 scientific evidence than most health care delivery and professional organizations. It makes sense for the Forum to capitalize on these strengths and, thus, to consider the availability of evidence as one factor in topic selection. Some consideration of whether an issue is controversial and whether the controversy is amenable to technical clarification is also reasonable. Nonetheless, it may sometimes be appropriate for the Forum to select a topic for which little evidence is available if, for example, a strong case can be made that misleading claims of effectiveness for an intervention need to be countered. In any case, available scientific evidence almost always is insufficient to answer all important questions about a clinical problem, and guidelines can make important contributions by identifying major gaps in clinical knowledge. One question that arose in the committee deliberations is whether the Forum should apply additional criteria to ensure-or at least make it likely that the topic list recommended to the administrator includes subjects related to specific population subgroups (e.g., children), to certain broad clinical problems or diseases (e.g., cardiovascular disease), or to particular legal, ethical, or social concerns (e.g., malpractice). The committee concluded that it was acceptable for the agency to establish separate "tracks" for considering such issues. The Forum has essentially done this in the past when it singled out certain clinical areas such as prenatal care and then convened special panels to nominate and rank topics in these areas. In the future, the committee recommended that the Forum should define more clearly its rationales for designating such special topic categories as worthy of such special consideration. The rationales should reflect explicit judgments about how important it is to have the special categories represented in the topic list presented to the AHCPR administrator for final decisions. The rationales can then be critiqued, which may, in turn, suggest alternative emphases that would be more consistent with program and policy objectives. In addition to giving special consideration to certain topic categories, the Forum also should consider whether past guidelines need to be updated. Some updates may involve the correction of errors, refinements in formatting, or other revisions that can be handled primarily by staff without a large investment of Forum resources. The availability of new evidence, however, raises the question of substantive reassessments and their priority. For the foreseeable future, the committee concluded it would be prudent for the Forum both to include existing guideline topics routinely when it solicits comments on proposed topics for guidelines development and to designate a separate "track" for considering reassessment topics.

OCR for page 1
6 SETTING PRIORITIES FOR CLINICALPR'4CTICE GUIDELINES Procedure Modifications The committee's recommendations for improving the Forum's current procedures focused on (1) methods for obtaining expert judgment and developing consensus positions and (2) the desirability of a basic procedure manual. With respect to the first point, Forum staff should develop model Delphi or Delphi-like procedures for obtaining expert judgments or topic rankings by mail, fax, or electronic mail; devise questions that are specific, explicit, and consistent with standard methods for questionnaire construction and with program purposes; and experiment with more formal procedures to arrive at group judgments during meetings convened as part of priority-setting activities. The dual objectives of these recommendations are, first, to help participants clarify their thinking and, second, to help AHCPR make more productive use of expert judgment. This report includes questionnaires for the Forum to consider when it next surveys practitioners or others for their views on guidelines priorities (see Appendix F). Another useful step the Forum could take is to develop a basic procedure manual for priority setting activities. This manual, which could be developed with assistance from elsewhere in AHCPR, should cover standard activities such as questionnaire construction and analysis and consensus development methods. The purposes would be to simplify and regularize the priority-setting process and to allow continuing and new staff to work more efficiently. The committee agreed that the Forum should also extend the basic procedure manual to cover the guidelines development procedures and methods. Topic Definition During its work, the committee heard repeated concerns that the Forum's priority-setting process needed to define more narrowly and precisely the topics selected for guidelines development. The committee agreed. For example, if the major quality, cost, and other concerns about lower back pain involve the management of acute lower back pain and if it appears that reasonable scientific evidence is available for evaluation, then the priority-setting process should identify that target, not back pain generally. The combination of earlier attention to topic definition and a focus on narrower topics should have several positive effects: (1) the composition and work of the development panel can be more efficiently organized from the outset, permitting the panel to spend more time on content rather than topic

OCR for page 1
SUMMARY 7 definition; (2) some apparent controversies may disappear when clinical issues are more precisely defined; (3) the guidelines product will be more responsive to the problems of most interest to clinicians, patients, and other users; and (4) the implementation and evaluation ofthe guidelines will be easier. To the extent that the focus turns to issues most amenable to clarification through authoritative assessment of scientific evidence, the result should be clearer, more specific guidelines. The committee recognized, however, that even if more attention is paid to topic definition during the topic selection stage, the panels developing guidelines will also need to consider topic boundaries and foci. The committee also heard some suggestions that guidelines developers should abandon the focus on clinical problems and concentrate on specific technologies. The committee did not find the arguments for such a wholesale shift persuasive. An initial focus on clinical conditions encourages a broad view of patient care issues and alternatives (including preventive, diagnostic, and treatment strategies) and an emphasis on health problems and outcomes of care as they are experienced, managed, or evaluated by patients, clinicians, and health care organizations. The committee concluded that the Forum should generally continue to direct its attention toward clinical problems, although the priority- setting process should not preclude the nomination or selection of technology- based topics. FUTURE ROLES FOR AHCPR As this report was being drafted, the AHCPR was engaged in a broad reassessment of its activities, including those of the Forum. One of its questions was: should AHCPR cede the work of guidelines development to others? In this committee's view, the answer at least for now is no. Although the number of groups involved in guidelines development is larger than when AHCPR was created, the Forum's work is more visible and inclusive than most, making its activities more accessible as examples or prototypes from which others can learn both positive and negative lessons. The value ofthis experience may diminish as the methodological, procedural, and other challenges of guidelines development are better resolved and as agreement on preferred approaches grows. Thus, the continuing contribution of AHCPR involvement in the function should be reevaluated periodically. Another rationale for some continuing AHCPR involvement in guidelines development is that as a public agency, AHCPR may consider topics relevant to the problems of uninsured or otherwise disadvantaged populations that would be low priorities for private groups that develop guidelines. In addition, the pressures of health care restructuring may lead private groups to avoid guidelines in areas in which financial incentives may encourage undertreatment or disregard for patient preferences about treatment options.

OCR for page 1
8 SETTING PRIORITIES FOR CLINICAL PRACTICE GUIDELINES National and Local Guidelines Increasing experience with clinical practice guidelines suggests that it is unreasonable- indeed impossible-to expect nationally developed guidelines to cover every operational issue for every kind of setting, practitioner, and patient or to specify care processes in sufficient detail that the guidelines can be directly integrated into local practice. Yet, guidelines that leave too much to be decided at the local level or during implementation run the risk of being ignored, misused, and modified in ways detrimental to patients. The restructuring of the health care system now underway adds both urgency and complexity to the task of defining roles for national and local organizations in guideline development that draw on the strengths of each and minimize their limitations. (In this context, the committee made an inexact but convenient distinction between "national" organizations that develop guidelines for use by others and "local" organizations that both develop and apply them in health care settings.) The committee recommended that the Forum build on its current work in both development and implementation to design and test alternative models of guideline development that include both national and local components. As very generally envisioned by this committee, such models would have two stages, one "national," the other "local." The first stage would be similar to the current guideline development process and would rely either on panels convened by the Forum or on outside contractors. The second phase would shin activity from the national to the local level. Work at the local level would involve a combination of two steps: further "specification" of the guideline (e.g., process-of-care protocols) and operational testing by organizations involved in health care delivery products might still have to be modified to account for local operating differences (e.g., differences in computer-based patient information systems), but the model would challenge those who wish to modify the guidelines to develop evidence that their modifications improved patient care or permitted equivalent care at lower cost. Clearinghouse, Assessment, and Other Activities In addition to the Forum's continued involvement in guidelines development, the committee concluded that the Forum could play a useful role as a guidelines clearinghouse that collected and disseminated guidelines developed by other private and public organizations. The committee also concluded that the Forum needed an explicit, open, and defensible process for assessing the soundness of guidelines developed by other organizations. Both the clearinghouse and assessment Unctions were considered in the 1992 IOM report, Guidelines for Clinical Practice: From Development to Use. The committee noted that a

OCR for page 1
SUMMARY 9 provisional assessment instrument was included as an appendix to that report and that efforts by various groups to apply the instrument warranted study. One caution: it is possible that an assessment process would reveal significant deficiencies in many of the guidelines assessed and that this prospect might generate political opposition to an assessment process from organizations responsible for such guidelines. Another potential role for the Forum involves the encouragement of methods and objectives for the formulation of procedure- or task-specific proficiency or performance guidelines. Unlike practice guidelines that assist practitioners and patients in deciding what course of care is appropriate (e.g., surgery or "watchful waiting"), proficiency or performance guidelines would set forth methods and measures for assessing an individual practitioner's competence in carrying out a specific task (e.g., a surgical procedure). Conceptual and methodological problems have historically troubled efforts by educators, hospital officials, state licensing bodies, and others to assess and ensure professional competence in general and with respect to specific skills or tasks. AHCPR might wish to consider whether some part of the agency should pursue work in this area as an extension of the organization's involvement in outcomes and effectiveness research and guidelines development. The committee was not asked to consider either the implementation of guidelines or the evaluation of their impact. Earlier TOM committees have, however, stressed that planning for successful implementation of guidelines must begin at the development stage and that the effectiveness of guidelines should be evaluated, not assumed. Topic selection and definition are critical first steps in the process of looking ahead to the attitudes, needs, and circumstances of those who must act if guidelines are to have the intended effects.

OCR for page 1