National Academies Press: OpenBook

Guidelines for Clinical Practice: From Development to Use (1992)

Chapter: B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES

« Previous: A: EXAMPLES OF CLINICAL PRACTICE GUIDELINES AND RELATED MATERIALS
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

B A Provisional Instrument for Assessing Clinical Practice Guidelines

KATHLEEN N. LOHR AND MARILYN J. FIELD

Division of Health Care Services

The Institute of Medicine (IOM) of the National Academy of Sciences has been engaged since the beginning of 1990 in two projects relating to the development, implementation, and evaluation of clinical practice guidelines. One IOM committee defined practice guidelines as "systematically developed statements to assist practitioners and patients in choosing appropriate health care for specific clinical conditions." It also delineated several desirable attributes of guidelines that are intended to help users understand the elements of a sound guideline and to recognize good (or not-so-good) guidelines. These aspects of guidelines were discussed in Clinical Practice Guidelines: Directions for a New Program (IOM, 1990) and Guidelines for Clinical Practice: From Development to Use (this report).

The first IOM study committee discovered that no explicit method was available for assessing existing or emerging practice guidelines. At least one instrument was being tested to assess some aspects of guideline development (AMA, 1990), but nothing existed to judge the quality, reliability, and validity of the content of the guideline itself. Therefore, one task the second IOM committee undertook was to develop an "assessment instrument" that could be used by various parties in formal evaluations of guidelines.

The next sections of this document describe, first, the purposes of the "provisional" assessment instrument and, second, its development. The discussion covers several features of the instrument and its application, and notes several cautions and caveats about the present form, all of which warrant further consideration. Finally, the document presents the instru-

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ment itself, in three operational parts—a general information sheet, the full instrument, and a summary evaluation sheet. The instrument is termed provisional because the committee firmly believed that more experience needs to be accumulated by testing it on different kinds of guidelines.

PURPOSES OF THE ASSESSMENT INSTRUMENT

The central purpose of the IOM's instrument for assessing clinical practice guidelines is to provide an explicit method for examining the soundness of such guidelines and to encourage their systematic development. By assessment is meant a prospective judgment of the soundness of both the process used in developing a guideline and the resulting guideline. The intent is to avoid situations in which a guideline that is not consistent with the scientific evidence is nonetheless "rated" as good on procedural criteria alone.1

More concretely, the IOM intended to operationalize its attributes of good guidelines and to provide a standardized approach and structure for the assessment of a guideline document. The resulting form is not simple. Therefore, the IOM does not expect practicing physicians or other clinicians, patients, other nonprofessionals, or policy makers to apply this instrument. Rather it expects individuals (or groups) with three types of expertise to apply it-namely, those with clinical experience with patients who have the conditions or problems covered by the guideline document, those with research experience in the conditions or technologies covered, and those with methodologic skills in developing guidelines. Any final or overall judgments of a guideline document emerging from the application of this instrument would be reported in simpler, summary form in ways that would convey the relative soundness (or lack of it) of a given guideline document to all potential users of the guideline.

The IOM committee sees three possible uses of this instrument: as an educational tool, as a self-assessment tool, and as a means of judging guidelines before their adoption. The Agency for Health Care Policy and Research (AHCPR) may want to use this instrument, or one like it, in directing the work of its guidelines-development panels; the agency may also wish to employ it in judging the products of those panels or the guidelines developed by other groups, such as medical specialty societies. Furthermore, other groups may wish to review existing or draft guidelines of their own against this instrument, in an effort to identify guidelines warranting revision or defects in draft guidelines that warrant correction before they are put into final form. Finally, if an organization were to be created for the

1  

Evaluation of the eventual impact of guidelines is a separate step. Both IOM reports (1990 and this one) include discussions of evaluation.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

express purpose of certifying or otherwise reporting on the soundness of particular guidelines, it might wish to employ the instrument as part of its review activities.

DEVELOPMENT PROCESS

The concept of the instrument originated during discussions with members of the AHCPR staff about their responsibilities for practice guidelines under the Omnibus Budget Reconciliation Act of 1989. In seeking to respond to those early ideas, the IOM committee went through four steps.

First, with the aid of an outside consultant, staff drafted a set of questions to operationalize the eight conceptual attributes of good practice guidelines identified in the IOM's 1990 report on guidelines. 2 (Discussions of these attributes introduce sections of the assessment instrument.) Second, these questions were combined with background information and instructions for users and subjected to considerable internal and external review, as described below. Members of the IOM committee twice reviewed drafts of the assessment instrument during this time.

Third, IOM staff used the critiques and suggestions of reviewers to revise the instrument, and it, together with background material, was subjected to external review according to IOM and National Research Council (NRC) procedures. Fourth, the instrument was revised in response to that external review, resulting in the provisional questionnaire and other forms incorporated into this document.

Initial Reviews

An interim draft of the instrument was sent to AHCPR in December 1990 and was subsequently forwarded to several professional societies that had volunteered to review and test the instrument against guidelines of their own. By June 1991, the committee had received more than 15 separate responses and commentaries. Responses included a lengthy summary review provided by IMCARE (Internal Medicine Center to Advance Research and Education), which had solicited reviews from the 231 internists then in its Guideline Network. IMCARE also sent the document to 147 physicians who requested more information and received 65 responses from network members who reviewed and, in many cases, applied the instruments to real guidelines.

Reactions to the draft instrument were extremely varied. With respect to format, instructions, ease of use, and similar issues, positive comments included the following:

2  

Anne-Marie Audet, M.D., of the Health Institute, New England Medical Center, Inc., served as consultant during the initial stage of the project.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
  • "Well written—concise."

  • "The instrument was easy to apply and the definitions of attributes were helpful in driving the assessment."

  • "The meaning of each attribute, the instructions, and response categories were generally clear."

  • "Overall, the instrument provides a precise algorithm for examining the degree to which a clinical guideline meets defined aspects of seven attributes. In our opinion, the instrument can be used for its intended purpose. In addition, application of an instrument such as this is, in itself, a thought-provoking exercise for individuals active in developing clinical practice guidelines."

By contrast, negative comments were of the following kind:

  • The form seems excessively lengthy and not particularly user-friendly."

  • "The instructions are verbose and redundant, and almost legalistic. They are not user-friendly."

  • "This instrument was very confusing, almost 'impossible."'

  • "Instrument of intellectual torture. Beyond Bureaucracy! . . . This makes me feel stupid!"

A form that accompanied the instrument asked reviewers to indicate the time (in person-hours) needed to apply the assessment instrument to a guideline of their choice. Among 59 individuals in the IMCARE group who evidently applied the instrument to an actual guideline and completed the form, 13 said that the instrument took under two hours to apply, 38 said two to three hours, and 8 said four hours or more. Several commentators indicated that their learning curve was quite steep and that repeated use of the instrument would make it simpler and less time-consuming to apply.

In rating the overall difficulty of understanding or using the assessment instrument, the majority of respondents indicated that it was moderately to very difficult. In addition, the great majority found the instrument good or at least somewhat helpful in helping them reach an overall judgment of the strengths and weaknesses of the guideline they were evaluating. Finally, more than half indicated that the instrument definitely should be revised; most of the rest were uncertain or had no opinion, and only about 1 in 10 advised abandoning the effort.

Among the more concrete recommendations for revisions were the following: (1) add more "don't know" or "not applicable" responses to certain questions; (2) simplify and shorten the instructions; (3) include a "prologue'' containing pertinent information from the first IOM (1990) report on guidelines, which defines key terms and similar concepts; (4) consider adding an attribute related to endorsement by appropriate affiliated or outside organizations; and (5) clarify exactly who the intended users are. In addition, many reviewers offered comments on the draft instrument itself, chief-

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ly observations about the wording of questions and the addition of response categories. More general suggestions included employing the instrument to help generate guidelines (rather than rate them) and using the instrument to guide an assessment process (without necessarily requiring that the instrument be completed in full).

Final Reviews

In accordance with IOM and NRC procedures, the revised instrument was subjected to an external, anonymous review by a panel similar to the full IOM committee. The "provisional" form in this document reflects the reactions of this group of seven experts, and many of their comments have been incorporated into the discussion presented here.

IMPORTANT FEATURES OF THE PROVISIONAL ASSESSMENT INSTRUMENT

Attributes of Practice Guidelines
Types of Attributes

Four attributes identified in the first IOM report on practice guidelines concern the substance of the guidelines—clinical applicability or scope, clinical flexibility, reliability/reproducibility, and validity. Four others— clarity, multidisciplinary process, scheduled review, and documentation— have more to do with process. This instrument explicitly incorporates all attributes but documentation; that attribute is captured in questions directed at the other seven. Each attribute is described in the text of the instrument.

Implicit Weight Accorded to Different Attributes

As discussed below, this instrument has no explicit or quantitative scoring system. The attributes are implicitly weighted, however, according to the number of main questions used to cover them. Of a total of 46 questions, validity has 22 questions; clarity, 8; multidisciplinary process, 4; clinical flexibility, 4; reliability and reproducibility, 4; clinical adaptability, 3; and scheduled review, 1. By this rough metric, validity is accorded by far the major emphasis in the document, reflecting the committee's concern for finding a way to judge the soundness of the guidelines themselves rather than just the acceptability of the development process.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Questions and Response Categories
Questions

Most of the seven attributes are dealt with through one or more questions that tap specific "dimensions." For example, validity is divided into five issues: (1) strength of the scientific evidence and professional consensus; (2) qualitative and quantitative statements about health benefits and harms or risks; (3) qualitative and quantitative statements about expected health costs or expenditures; (4) the extent to which specific recommendations are justified by the estimates of benefits, harms, and costs provided in the guidelines and the extent to which those estimates are supported by the evidence amassed in the guidelines document; and (5) potential conflicts among existing guidelines, if any.

The instrument has 46 descriptive questions related to the seven attributes. Generally they pertain to the presence of information about the attribute or about a particular dimension of an attribute. Several questions have additional "items" designed to help assessors think about key points implied by the main question and whether a particular dimension of an attribute is satisfactory or not.

Responses to Questions

Responses to each question are typically "yes" or "no" for questions asking about the presence or absence of certain information, features, or development processes. If the answer is "yes,'' a follow-up question asks about the quality of that information, feature, or development process--essentially whether the information provided is satisfactory or not. If the answer to the main question is "no," the follow-up question probes the significance of the absence of information, a particular feature, or development process and asks whether the omission is important or not.

Satisfactory. Assessors can judge information about a particular attribute as satisfactory if all critical elements have been considered and presented. For example, the discussion or description should be thorough and comprehensive; the guideline developers should have based their work on appropriate and correct information (e.g., from the literature review); and they should have used appropriate methods (e.g., for evaluating the strength of the scientific evidence or reaching professional peer consensus).

Conditionally satisfactory. The description or discussion of an attribute or dimension is conditionally satisfactory if some, but not all, of the critical elements have been considered and presented. For example, the discussion of a particular aspect of an attribute such as scheduled review may be vague

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

or incomplete. Alternatively, guidelines developers may have disregarded important information (about, for instance, likely risks to patients from the use of a technology) in reaching their recommendations, or they may have improperly used certain kinds of methods. These problems with the guideline document may not prevent a clinician from using it or understanding its recommendations, but they may affect its overall usefulness or call certain recommendations into question. Revisions would be presumed to improve the guidelines, but they would not be mandatory.

Unsatisfactory. The description or discussion about a specific attribute is unsatisfactory if most of the critical elements have not been considered or presented. For example, well-known pieces of clinical information (or the views of multiple specialists with an interest in the guideline topic) may have been ignored; methods of analysis may have been misapplied; or recommendations may be based on faulty information or poor logic, or both. In such a case, it would be difficult if not impossible to judge the quality of the process of guideline development or the soundness of the resulting guidelines and recommendations (or both); certainly assessors could not mark those attributes as satisfactory. Serious thought must be given to augmenting or revising the guidelines document before it is promoted further.

Omissions. Omitting a description or discussion about a specific attribute or dimension may be unimportant if that omission seems likely to have no demonstrable effect either on the ability of a guideline user to apply the guideline effectively in the clinical decision-making process or on the capacity of the assessor to make an independent assessment of the quality of that attribute. Omitting such a description or discussion is of minor importance if it seems likely to affect negatively the ability of a guideline user or of evaluators to apply the guideline effectively or independently to assess its quality. Finally, omitting such a description or discussion is a major omission if the absence of such information essentially prevents guideline users or evaluators from applying the guideline effectively or even making an independent evaluation about the soundness of the guidelines document itself (at least on that particular feature).

Special Cases

Special cases may arise in which information appropriately is omitted from the guideline because the question or item is not applicable or is inappropriate (given responses to earlier items, for instance). In such an instance, the assessor is asked to mark the response category most appropriate for the given case (e.g., not applicable). In other situations, assessors may find it difficult to arrive at a single answer to the question, especially if the guidelines document being evaluated is very complex or if necessary background information appears to be missing. "Comments" sections are provided throughout for assessors to record additional remarks or qualifying

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

statements, to highlight areas not well covered by the instrument, and to note special factors that should either be followed up or taken into account in the overall judgment about the guidelines document. Finally, if assessors conclude that the guidelines document is so complex, clinically esoteric, or methodologically sophisticated that it warrants additional, outside expert review, they are asked to note that at the end of the full instrument and also on the summary evaluation sheet.

Alternative Approaches to Responses

The main type of response used in this form is categorical (e.g., satisfactory, conditionally satisfactory, and unsatisfactory). Some reviewers noted that this approach is inherently constraining and requires definitions of the three categories, which may not be interpreted consistently. Furthermore, these categories do not allow assessors to distinguish guidelines that more properly should be characterized as excellent or outstanding. To overcome some of these drawbacks, an approach to responses based on a scale might be tested.

For example, a five- or even seven-point scale might be adopted, with one end of the range described as excellent (exemplary, highly satisfactory, or a similar superlative) and the opposite end described as poor (inferior, or very unsatisfactory). The equivalent approach might also be tried for the responses concerned with omissions of information. The committee believes this change warrants testing at some point in the future development of this form.

Response Scoring

After considerable debate and consideration of reviewers' comments, the committee concluded that this instrument should not be "scored" in any quantitative way. Thus, it does not propose any formal weighting or numerical scoring scheme for the main questions, nor does it suggest a particular threshold, cut-point, or floor against which current guidelines might be judged acceptable or unacceptable. If most responses to the questions are "satisfactory" (or "unimportant omissions"), however, one might reasonably conclude that such a guidelines document would be sufficient for most clinical situations. Alternatively, if most responses were unsatisfactory (or major omissions), one would probably argue that the guidelines document needed to be revised before it could be used effectively.

The committee was of the view that a defensible scoring system could not be designed a priori in any case, regardless of whether scoring would be purely categorical or more quantitative. Review and testing of the assessment instrument itself—with revisions as necessary—will be required before a sensible scoring system can be proposed. Moreover, different

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

users of the assessment instrument may have legitimate reasons to differ on where they would establish such cut-points. Provision of information on the quality of guidelines documents appears to be more in the public interest than is making "one-size-fits-all" judgments on behalf of others.

In the same vein, no single question is treated as the signal of a "fatal flaw." That is, for no question will a response of "no," "unsatisfactory," or "major omission'' by itself render the guidelines document unacceptable. Some committee members believed that certain questions, especially those relating to validity, should be so designated. However, the questions that seem to be the most likely candidates for this level of decisiveness3 were added in response to the external review of the draft document; therefore they have not yet been reviewed or tested further. The committee believes designating these (or other) items as potentially "fatal" is premature.

Response Aggregation and Display
General Comments

Information obtained from applying the instrument might eventually be arrayed in one or more qualitative, summary displays or tables, as might be done, for instance, by a Consumer Report article. This might provide a rough indicator of whether the guidelines document could be used effectively in clinical situations. One reviewer, for example, suggested that a report for busy practicing physicians might usefully include "a graphic summary of the degree to which each attribute was successfully achieved, e.g., a bar representing the percent of key items within each attribute that were deemed satisfactory ...; and . . . a brief, narrative summary assessment."

Summary Evaluation Sheet

The committee did not pursue the design of such displays, chiefly because such an effort was seen as premature for an instrument that itself warrants additional testing and application. As an intermediate step, however, the instrument does include a "summary evaluation sheet," which is actually a set of pages that condense the findings of the assessment from the primary questions in the instrument. It is filled out only after the full instrument has been completed.

3  

Two questions that might be candidate "fatal flaw" items, when responses to them were unacceptable (i.e., "no"), are the following from the validity section: * Generally, the estimates of benefits, harms, and costs are consistent with the evidence presented in the guidelines document. (Yes, completely; yes, partially; or no) (Question 28) * Each major recommendation is consistent with the estimated benefits, harms, and costs of the service or intervention (and thus with the strength of the evidence). (Yes, completely; yes, partially; or no) (Question 31).

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

In the present version, "better" answers are recorded to the left of the response column, "worse" answers to the right. Thus, a quick scan of this sheet may provide an overall sense of the quality of the guidelines. Put another way, a clinician or other user ought to be able to apply the guideline effectively if all dimensions of the seven attributes (or, at a minimum, all seven attributes) are judged to be "satisfactory" and all omissions of information are considered ''unimportant," as those terms were defined earlier. In this (ideal) situation, all notations on the summary sheet would be on the far left. By contrast, if many or most notations are on the right side of the response column, the user might wish to employ the guidelines only selectively or to request clarifications or revisions.

Several reviewers noted that completing the summary evaluation sheet is essentially a clerical task, provided the main part of the assessment instrument has been legibly and fully completed by one or more experts (as discussed earlier). The committee agrees and thus suggests that junior or clerical staff be given this responsibility, and that the instructions on the form indicate that users might wish to do so. Alternatively, the instrument (or at least the recording of responses to its questions) might be computerized. In that case, the summary sheet could be an automatic product of the computer program. The value in pursuing more fully the possibilities of computerization of this form might be considerable.

FINAL CAUTIONS AND CAVEATS

The Ideal: Enemy of the Good

Some commentators noted the IOM's recognition in its 1990 report that most (if not all) guidelines in existence today would "fail" to meet the ideal of this instrument. Reviewers were concerned that "prematurely imposing excessive rigor" would discourage some guideline developers.4

4  

Comments from the American Nurses Association were particularly to the point: the criteria used to judge the adequacy of guidelines establishes a high standard that likely would seldom be achieved in reality. Several questions need careful consideration prior to accepting the criteria in this instrument: 1. Do the criteria . . . create false expectations of quality which is not achievable with current fiscal restrictions in health care? 2. What are the potential legal and regulatory ramifications of accepting these criteria as representative of quality practice? 3. How would these criteria eventually influence the costs of care through the pursuit of considerable evidence regarding the "best" method of treatment? 4. Are these guidelines intended to weed out bad practice, or is the intention to demonstrate the "best," often misinterpreted as the "only" way of practicing?" (K. S. O'Connor, Division of Nursing Practice and Economics, American Nurses Association, in a letter to Marilyn Field, IOM study director, dated May 20, 1991.)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

At least one reviewer warned that the assessment process should not be used as a "second level expert panel" and cautioned that designing the assessment instrument process itself would take some care. The IOM committee agrees and, in that light, emphasizes that the educational uses of the instrument are more important than its assessment applications (in the near term at least).

Users of the Assessment Instrument

Busy practicing physicians or other clinicians are not the intended or anticipated appliers of this assessment instrument. Neither are policy makers, patients, or other nonprofessionals, although all may have some interest in the results. Assessors are expected to have, individually or collectively, expertise in three areas: clinical experience with patient populations covered by the guideline, research experience about the conditions or technologies covered by the guideline, and methodological expertise with techniques and processes of guideline development.

Because no one individual is likely to possess all three kinds of expertise, experience with the instrument may suggest that a "group," "panel," or "study section" approach will be needed to apply it satisfactorily. In this way, different individuals would be responsible for different parts of the assessment (particularly to determine validity). Furthermore, turning the full assessment into a review or evaluation that would be understandable to patients, practitioners, or policy makers will be a separate step, as noted earlier.

This provisional instrument thus proceeds on several assumptions. First, assessors (individually or collectively) are sufficiently schooled either in the methodologic issues inherent in guidelines development or in the clinical issues related to the main topic of the guidelines document that they are able to complete the bulk of the assessment instrument unaided. Second, questions about clinical topics or methods can be referred to appropriate experts when necessary and without undue delay. This kind of referral is particularly important if the AHCPR or some other entity acquires a specific mandate to certify or ratify guidelines from whatever source. Third, in some cases, having a dual or parallel (i.e., simultaneous) review may be a desirable tactic. Fourth, junior staff may well be used to assemble relevant material, perhaps to do an initial check of the document itself, perhaps to evaluate the document for the attribute of clarity, and to complete the summary evaluation sheet. Finally, the experts assembled or asked to apply the instrument (in its current form or any future, modified form) will be carefully trained in its use.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Availability of Supporting Material

Guidelines documents would become unmanageably long and unworkable for busy clinicians if all the information leading to specific recommendations were included in the guideline itself. Nevertheless, the availability of such information somewhere is important. Meeting this expectation presents a difficult conflict for guideline developers, and it seriously complicates the task of assessing guidelines.5

At this time, the committee takes the position that as much information as possible should be synthesized into a guideline document, even if the formal guideline made widely available to practitioners and clinicians is a streamlined version. The assessment process is then to be directed at the complete document (with whatever supporting materials may be submitted with it), not the clinician's version. This stance accords with the committee's general goal of assessing the underlying quality of the guideline itself, not just the process by which it was developed.

For any of the uses to which the present instrument is put, the committee thus assumes that relevant documentation will be in the assessors' hands. This assumption is particularly important when the guidelines to be assessed have been developed by others, and especially if those organizations have approached AHCPR or another certifying entity with a specific request for ratification of the guidelines. Hence this instrument assumes that the pertinent information concerning the guidelines document—including information related to the process of development itself—is available for any review effort, with no provision for "later" or "on request" submission of information.

A consequence of this assumption is that this instrument is directed at "guideline documents," however those documents might be construed by the developers. Some guidelines may be contained within a single report, monograph, or other publication. Other guidelines may incorporate related publications by reference, particularly when developers have used a standardized methodology that is described elsewhere. Primary and secondary publications, reports, and records relating to the development of the guideline document being assessed should be assembled before the assessment exercise begins. This might include, for example, reviews and syntheses of

5  

For example, the AHCPR Forum panel that has been working on the issue of managing depression in community-based settings started originally with 50,000 citations to the literature, reviewed between 4,000 and 6,000 articles, and based its guidelines document on about 400 relevant articles (J. J. Strain, member of the IOM committee, in a memorandum to Kathleen Lohr and Marilyn Field, dated June 17, 1991). There is no possibility that assessors of the guideline document could replicate that experience or even undertake to review the final set of relevant articles.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

the relevant scientific literature, but such a requirement would not extend to individual research reports and articles themselves.

Nevertheless, the committee recognizes that published guidelines may be "incomplete" because of limitations placed on the authors by editors and publishers (e.g., space constraints) and that important documentation may not be present. Therefore, if the instrument is applied to published guidelines developed by a group that does not deliberately seek to have its guidelines so assessed (and volunteer the supporting material as assumed above), additional material may need to be gathered from those authors in order to apply this instrument fairly.

Standardized Format Versus Narrative Evaluation

Regardless of what approach to assessing guidelines is finally adopted and what level of expertise the evaluators possess, several basic complexities must be acknowledged. For instance, simply assessing whether guideline developers explain or document a certain piece of information does not allow one to discriminate a comprehensive disclosure from one of poor quality. Similarly, lack of disclosure may have a significant or only a trivial impact on the clinical usefulness and validity of a guideline. No structured instrument of practical length is likely to be able to accommodate these nuances across guideline documents of many different types. Thus, some narrative, global assessment may always be desirable, if not absolutely necessary, if assessments of guidelines are to be useful for a wide set of audiences.

In developing the instrument, the committee asked the first set of reviewers to comment on a "handbook" approach as an alternative to the formal instrument. This approach would provide guidelines assessors with some instructions about the attributes of guidelines to be evaluated and would require them to prepare a narrative evaluation statement, but it would not produce specific responses to specific questions.

Some respondents preferred the "objective, criterion-based" review (i.e., the formal instrument), noting that it might yield "a more standardized evaluation strategy" and "potential benefits such as greater efficiency and reliability, a more readily digested assessment of the strengths and weaknesses of a guideline, and the ability to draw more 'objective' comparisons among a collection of guidelines." Given that no clear preference for the handbook approach emerged, the committee did not pursue this approach further. However, the desire expressed by several reviewers for a narrative, summary statement about a guideline document probably reflects some discomfort with an assessment strategy based solely on the question-and-answer format of the present instrument.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Further Pretesting and Experience with the Instrument

In developing this instrument, the committee recognized the need for more practical experience with it. The present version incorporates revisions suggested by the large number of reviewers, mainly from medical specialty societies, who critiqued an earlier version and in some cases applied it to actual guidelines, as well as changes pursuant to the IOM/NRC review. Nevertheless, the committee takes the view that further application and pretesting of this provisional form should be conducted.

That testing should determine answers to the following questions: Is the instrument too long and too complicated for practical routine use? Does experience applying the instrument as an assessment tool make it easier to use, as several reviewers believed it might? Can shortcuts be found in applying it? For instance, is it useful to have junior staff make an initial check to determine whether all relevant materials appear to be in the guidelines document package or to make a first-assessment pass through the guidelines document itself? Are the results of the assessment consistent with results of any pretests or early evaluations of the guidelines in actual practice?

The present committee takes no stand on how extensive such pre- or pilot-testing might be—for instance, on the number of guidelines that should be assessed to determine the reliability, validity, and practicality of the current form. Two factors are relevant. First, the committee had neither the time nor the resources to pursue these issues further (and certainly not to carry out such activities itself). Second, it considers that such testing might need to be specific to the potential user groups and that setting a priori rules risks making them too rigorous or too confining for all purposes.

THE PROVISIONAL ASSESSMENT INSTRUMENT

The form reproduced in the last part of this appendix has three main parts. First is a general information sheet, with space for the following items to be briefly described: clinical diagnoses or conditions; health practices, services, or technologies; target populations; primary settings of care; primary types of clinicians targeted; stated purposes of the guideline; source, author, or developer of the guideline document; person to contact for further information about the guideline document; date of issue of the guideline document; and name/affiliation of assessor(s). The second section is the full instrument itself, with self-contained instructions. The third section is the summary evaluation sheet.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ACKNOWLEDGMENTS

The idea of creating an instrument by which to assess the soundness of clinical practice guidelines can be traced through the work of three Institute of Medicine committees: the Committee to Design a Strategy for Quality Review and Assurance in Medicare, the Committee to Advise the Public Health Service on Clinical Practice Guidelines, and the present Committee on Clinical Practice Guidelines. The present committee wishes, therefore, to acknowledge the groundbreaking efforts of its predecessor panels in this rapidly evolving arena.

Production of this provisional instrument would not have been possible without the assistance of many individuals and organizations. We are indebted to the members of various medical specialty societies who reviewed and in some cases voluntarily applied and tested an earlier draft instrument; we especially thank Betty King, executive director of IMCARE (Internal Medicine Center to Advance Research and Education) for her efforts in organizing a broad review of the instrument by the IMCARE task force on practice guidelines. We thank Anne-Marie Audet of the Health Institute, New England Medical Center Hospitals, for her enthusiastic efforts with the first draft of the instrument.

The project under which this instrument was developed was supported by The John A. Hartford Foundation and by the Agency for Health Care Policy and Research, U.S. Department of Health and Human Services, under Contract No. 282-900018. The views presented are those of the Institute of Medicine Committee on Clinical Practice Guidelines and the authors, and are not necessarily those of the funding organizations.

Finally, we extend our appreciation to our Institute of Medicine (IOM) colleagues on this project, Molla Donaldson and Holly Dawkins; to the project's senior secretaries, Theresa Nally and Donna Thompson; and to other members of the IOM staff who provided timely and helpful comments, including Christopher Howson, Michael Stoto, and Malin VanAntwerp.

REFERENCES

AMA (American Medical Association). "Evaluation of a Recently Completed Practice Parameter." Chicago: American Medical Association, 1990.


IOM (Institute of Medicine). Clinical Practice Guidelines: Directions for a New Program.


M.J. Field and K.N. Lohr, editors. Washington, DC: National Academy Press, 1990.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ASSESSMENT INSTRUMENT

PART ONE. GENERAL INFORMATION SHEET

TO THE ASSESSOR: Please complete this sheet with brief statements about the content of the guideline document you are reviewing. Use whatever information can be found in the document. If you cannot find the relevant information or are uncertain about the appropriate response, indicate "not specified" or "uncertain."

TITLE OF GUIDELINE DOCUMENT_____________________________________________

  1. Clinical diagnoses or conditions

  2. Main health practices, services, or technologies considered

  3. Target populations (e.g., age, sex, income level, health status)

  4. Primary settings of care (e.g., primary or specialty; nursing home)

  5. Primary types of clinicians targeted (e.g., profession; specialty)

  6. Stated purposes, aims, or goals of the guideline document

  7. Source, author, or developer of guideline document

  8. Individual to contact for further information about the guideline document (name, organization, phone number)

  9. Date of issue of the guideline document

  10. Name/affiliation of assessor(s)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

PART TWO. ASSESSMENT INSTRUMENT

Background

This instrument itself has seven sections, each corresponding to one of the seven attributes of guidelines to be evaluated. Each section begins with a brief definition of the attribute and then is divided into segments that deal with important dimensions of that attribute.

Instructions: Illustrative Example

Each segment begins with a descriptive question that you should answer yes or no. The responses will then direct you to move to a specific next question. Space is provided for "Comments" throughout the instrument.

For example, in the section on clinical applicability, the first question reads (in part) as shown below, and you are instructed to check "yes" or "no" and then answer the appropriate subquestion:

1. THE GUIDELINE DOCUMENT DESCRIBES THE PATIENT POPULATIONS TO WHICH THE GUIDELINES ARE MEANT TO APPLY.

_____Yes (Go to 1.1) _____No (Go to 1.2)

1.1. THE DESCRIPTION OF THE PATIENT POPULATIONS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 2 >>

1.2. OMISSION OF A DESCRIPTION OF THE PATIENT

POPULATIONS IS:

_____Unimportant _____Minor ______Major

omission omission

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

In some circumstances you are asked to judge a set of variables-specific elements to consider in evaluating an attribute-and to arrive at a "global" answer to a particular question. When this occurs, you should (1) start with the set of items that are identified alphanumerically (e.g., Ia, Ib, . . .) and answer them directly and then (2) combine those answers into a summary evaluation to determine whether the guideline document has dealt with that particular issue in a satisfactory, conditionally satisfactory, or unsatisfactory manner.

You should then go on to the next question or to the next section, as directed. In the absence of a specific direction, go to the very next question.

Definitions of Terms

The questions in this instrument ask for three different types of responses. The meaning of these response categories is as follows:

Yes and no. Most of the main questions concern the presence of a discussion or piece of information about a particular attribute. Generally, the "yes" and "no" responses direct you to answer follow-up questions. For these items, response choices are "satisfactory," ''conditionally satisfactory," and "unsatisfactory," or "unimportant omission," "minor omission," and "major omission." These terms are further defined below.

Satisfactory. You can judge information about a particular attribute as satisfactory if all critical elements have been considered and presented. For example, the discussion or description should be thorough and comprehensive; the guideline developers should have based their work on appropriate and correct information; and they should have used appropriate methods.

Conditionally satisfactory. The description or discussion of an attribute or dimension is conditionally satisfactory if some, but not all, of the critical elements have been considered and presented. For example, the discussion of a particular aspect of the attribute may be vague or incomplete; alternatively, the guidelines developers may have disregarded important information in reaching their recommendations or improperly used certain kinds of methods. These problems with the guideline document may not prevent a clinician from using it or understanding its recommendations, but they may affect its overall usefulness or call certain recommendations into question. Revisions would be presumed to improve the guidelines, but they would not be mandatory or essential.

Unsatisfactory. You can determine the description or discussion about a specific attribute to be unsatisfactory if most of the critical elements have not been considered or presented. For example, well-known pieces of clin-

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

ical information (or the views of multiple specialists with an interest in the guidelines topic) may have been ignored; methods of analysis may have been misapplied; or recommendations may be based on faulty information, poor logic, or both. In such a case, it would be difficult if not impossible to judge the quality of the process of guideline development or the soundness of the resulting guidelines and recommendations, or both; certainly they could not be assessed as satisfactory, and serious thought must be given to augmenting or revising the guideline document before it is promoted further.

Unimportant omissions. You can regard the omission of a description or discussion about a specific attribute or dimension of an attribute as unimportant if it (1) is likely to have no meaningful impact on the ability of a guideline user, such as a practitioner or patient, to apply the guideline effectively in the clinical decision-making process and (2) does not prevent you from easily and independently assessing that aspect of the guideline document.

Minor omissions. The omission of a description or discussion about a specific attribute or dimension is of minor importance if it (1) is likely to have only a little negative impact on the ability of a guideline user to apply the guideline effectively in the clinical decision-making process and (2) does not prevent you from assessing that aspect of the guideline document.

Major omissions. You can determine the omission of a description or discussion about a specific attribute or dimension to be a major problem if it (1) is likely to prevent a guideline user from applying the guideline effectively in the clinical decision-making process or (2) prevents you from making an independent assessment about that aspect of the guideline document.

Not applicable or don't know. In some situations, the question may not be applicable to the guideline document you are evaluating. When that occurs, simply mark "NA" for "not applicable" or "DK" for ''don't know."

Comments. In other situations, you may find it difficult to arrive at a single answer to the question, especially if the guideline document you are evaluating is very complex or if necessary background information appears to be missing. In these cases, you can record additional remarks or qualifying statements about your response in the "Comments" sections.

Finally, if you believe that the guideline document is so complex, clinically esoteric, or methodologically sophisticated that it warrants additional, outside expert review, please note your comments at the end of the full instrument.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

I. CLINICAL APPLICABILITY

Clinical applicability, or the scope of the guideline, means three things in the context of this instrument. First, guidelines should be written to cover as inclusive a patient population as possible, consistent with knowledge about critical clinical and sociodemographic factors relevant for the condition or technology in question. To that end, the patient population(s) covered should be described as accurately and precisely as possible. Second, if patient populations that might be expected to be covered by the guideline are not, then the document discusses why those populations have been excluded; that is, it identifies the patient populations the guidelines are not meant to serve or apply to. Third, when the clinical conditions or problems covered by the guideline are likely to be complex, or when the guideline recommendations may be contingent on complex patterns of clinical factors, those points should be explicitly covered in the guideline document.

This attribute requires that two things be true about the guideline document. First, the guideline document accurately and precisely states how broad or narrow the patient population(s) are to which the guidelines are meant to apply, describes the actual population(s) to which statements apply, and describes the population(s) to which statements are not meant to apply. Population(s) may be described in terms of diagnosis, pathophysiology, severity of primary disease, presence of coexisting diseases, age, sex, race, social support systems, and other characteristics. Second, it notes and discusses any complex clinical issues that may arise for this patient population.

1. THE GUIDELINE DOCUMENT DESCRIBES THE PATIENT POPULATIONS TO WHICH THE GUIDELINES ARE MEANT TO APPLY.

_____Yes (Go to Question 1.1) _____No (Go to Question 1.2)

1.1. THE DESCRIPTION OF THE PATIENT POPULATIONS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 2 >>

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

1.2. OMISSION OF THE DESCRIPTION OF THE PATIENT POPULATION(S) IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

2. THE GUIDELINE DOCUMENT DISCUSSES COMPLEX CLINICAL PROBLEMS THAT CAN BE EXPECTED FOR THE POPULATION(S) COVERED BY THE GUIDELINES.

_____Yes (Go to Question 2.1)

_____No (Go to Question 2.2)

_____Not Applicable (Go to Question 3)

2.1. THE DISCUSSION OF EXPECTED COMPLEX CLINICAL PROBLEMS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 3 >>

2.2. OMISSION OF THE DISCUSSION OF EXPECTED COMPLEX CLINICAL PROBLEMS IS:

_____Unimportant _____Minor ______Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

3. THE GUIDELINE DOCUMENT GIVES A RATIONALE FOR EXCLUDING PATIENT POPULATION(S).

_____Yes (Go to Question 3.1) _____No (Go to Question 3.2)

3.1. THE RATIONALE FOR EXCLUDING CERTAIN PATIENT POPULATION(S) IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO II. CLINICAL FLEXIBILITY >>

3.2. OMISSION OF THE RATIONALE FOR EXCLUDING CERTAIN PATIENT POPULATION(S) IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

II. CLINICAL FLEXIBILITY

Clinical flexibility means that two mediating factors should be addressed in the guideline document. First, it should identify major foreseeable exceptions to or options for applying the guidelines, if any exist. Second, it should discuss the role of patient preferences for different courses of health care for those conditions or technologies in which patient values and preferences may be important decision-making factors (for example, being able to choose in an informed way between surgery and watchful waiting).

This attribute requires the guideline document to discuss two topics. First are situations (if any) in which socially relevant factors permit an exception to be made in applying the guidelines. These factors could include the home and family situation of the patient, clinical constraints on the health care delivery setting (e.g., no intensive care beds, no 24-hour anesthesiologist), nonclinical constraints on the health care delivery setting (e.g., inadequate information systems), or all of these; if no such factors exist, the guideline document should say so. Second is the role of patient preferences for different possible outcomes of care, when the appropriateness of a clinical intervention involves a substantial element of personal choice or values on the part of the patient. For example, this discussion may include information as to major points on which preferences may diverge for the case in hand, specific points to consider in eliciting patient preferences, and means of integrating patient views in the decision making process.

4. THE GUIDELINE DOCUMENT PROVIDES SPECIFIC INFORMATION ABOUT SITUATIONS IN WHICH CLINICAL EXCEPTIONS MIGHT BE MADE IN APPLYING THE GUIDELINES.

_____Yes, the document gives information about clinical exceptions (Go to Question 4.1)

_____No, the document says nothing about clinical exceptions (Go to Question 4.2)

4.1. THE INFORMATION OR STATEMENT ABOUT CLINICAL EXCEPTIONS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

>> GO TO QUESTION 5 >>

4.2. OMISSION OF INFORMATION OR A STATEMENT ABOUT CLINICAL EXCEPTIONS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

5. THE GUIDELINE DOCUMENT PROVIDES SPECIFIC INFORMATION ABOUT NONCLINICAL SITUATIONS IN WHICH EXCEPTIONS MIGHT BE MADE IN APPLYING THE GUIDELINES.

_____Yes, the document gives information about nonclinical exceptions (Go to Question 5.1)

_____No, the document says nothing about nonclinical exceptions (Go to Question 5.2)

5.1. THE INFORMATION OR STATEMENT ABOUT NONCLINICAL EXCEPTIONS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 6 >>

_____Unimportant _____Minor _____Major

omission omission

5.2. OMISSION OF INFORMATION OR A STATEMENT ABOUT NONCLINICAL EXCEPTIONS IS:

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

6. THE GUIDELINE DOCUMENT DISCUSSES THE ROLE OF PATIENT PREFERENCES, AS THEY RELATE TO HEALTH CARE DECISIONS IN THE PARTICULAR CASE THAT THE GUIDELINES COVER.

_____Yes (Go to Question 6.1) _____No (Go to Question 6.2)

6.1. THE DISCUSSION OF PATIENT PREFERENCES IS:

_____Satisfactory _____ Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 7 >>

6.2. OMISSION OF DISCUSSION OF PATIENT PREFERENCES IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

>> GO TO III. RELIABILITY/REPRODUCIBILITY >>

7. THE GUIDELINE DOCUMENT DESCRIBES HOW PATIENT PREFERENCES WERE TAKEN INTO ACCOUNT DURING THE GUIDELINE DEVELOPMENT PROCESS.

_____Yes (Go to Question 7.1) _____No (Go to Question 7.2)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

7.1. THE DISCUSSION OF HOW PATIENT PREFERENCES WERE CONSIDERED IN DEVELOPING THE GUIDELINE IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO III. RELIABILITY/REPRODUCIBILITY >>

7.2. OMISSION OF THE DISCUSSION OF PATIENT PREFERENCES IN DEVELOPING THE GUIDELINE IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

III. RELIABILITY/REPRODUCIBILITY

Reliability and reproducibility for the purpose of assessing guidelines means that, given the same circumstances, essentially the same set of guidelines would be developed by a second group; further, the terms mean that, ideally, the guidelines are or would be interpreted and applied consistently by practitioners or other appropriate parties.

Reliability and reproducibility of a guideline document is not likely ever to be assessable empirically. To approach these concepts, therefore, this attribute requires either that guidelines be subjected to some form of explicit, independent review by a group (or groups) other than the original developers, where that group (or groups) is equivalent in expertise and other factors to the original developers, or that the guideline recommendations have been pretested in some manner, or both. (Pretesting can be done in actual delivery settings or on prototypical cases.) If no such review or pretesting has been done, then the guidelines must explain the reasons.

8. THE GUIDELINES WERE SUBJECTED TO INDEPENDENT REVIEW BY EXPERTS OR OUTSIDE PANELS.

_____Yes (Go to Question 8.1) _____No (Go to Question 9)

8.1. THE DISCUSSION OF INDEPENDENT REVIEW IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 10 >>

9. THE GUIDELINE DOCUMENT EXPLAINS THE LACK OF INDEPENDENT REVIEW.

_____Yes (Go to Question 9.1) _____No (Go to Question 9.2)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

9.1. THE EXPLANATION OF THE LACK OF INDEPENDENT REVIEW IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 10 >>

9.2. OMISSION OF AN EXPLANATION OF THE LACK OF INDEPENDENT REVIEW IS:

_____Unimportant _____Minor _____Major

omission omission

10. THE GUIDELINES WERE PRETESTED IN SOME MANNER.

______Yes (Go to Question 10.1) _____ No (Go to Question 11)

10.1. THE DISCUSSION OF PRETESTING IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO IV. VALIDITY >>

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

11. THE GUIDELINE DOCUMENT EXPLAINS THE LACK OF PRETESTING.

_____Yes (Go to Question 11.1) _____No (Go to Question 11.2)

11.1. THE EXPLANATION OF THE LACK OF PRETESTING IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO IV. VALIDITY >>

11.2. OMISSION OF AN EXPLANATION OF THE LACK OF PRETESTING IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

IV. VALIDITY: DEFINITION AND EVALUATION QUESTIONS

Validity of practice guidelines means, conceptually, that if they are followed, then they will lead to the health and cost outcomes projected for them. Validity must be judged primarily by reference to the substance and quality of the evidence cited, the means used to evaluate the evidence, and the relationship between the evidence and the recommendations. Validity is the most critical attribute and the most difficult to assess. Although this section contains 22 questions, questions 28 and 31 are, together, of special importance because they constitute an overall evaluation of this attribute.

This attribute requires that five things be true for the guideline document. First, the collection, synthesis, and interpretation of scientific evidence must be documented and of satisfactory quality; ideally, each major recommendation will be described as based on ''excellent," "acceptable," or "weak" evidence, or with a similar set of descriptive terms.

Second, both qualitative and quantitative statements about health benefits and harms/risks appear in the guideline document, and insofar as possible those estimates are tied to and justified by the evidence amassed as part of the literature review and analysis. For example, a qualitative statement about benefits might read "screening mammography should lead to a decrease in breast cancer mortality"; a similar statement about harms and risks might read "screening mammography can lead to false-positive results and to unnecessary workup and anxiety." Quantitative statements might read, respectively, "screening mammography in women 50 years of age may reduce mortality from 20 percent to 60 percent" and "among one million women 40 to 50 years of age, radiation from 10 mammography examinations can be expected to cause about 60 new breast cancers." In all cases, such statements should be based on evidentiary information insofar as possible, and appropriate qualifiers or caveats noted when the evidence is weak or conflicting or when the estimates are based on consensus techniques such as expert panels or group judgment methods.

Third, both qualitative and quantitative statements about expected health costs or expenditures appear in the guideline document; the same requirements about the link between the guideline estimates and the data sources should be met, and the same degree of specificity about patient groups should be observed. In addition, the document should be clear as to whether costs referred to are the total for the patient group or the per-patient figure. For example, "use of laparoscopic techniques to treat cholecystitis should reduce the direct and indirect costs associated with using cholecystectomy as the main patient management approach" might be a suitable qualitative statement concerning costs, and "use of laparoscopic techniques in the treatment of cholecystitis may reduce the costs of treatment as much

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

as 75 percent by the end of the decade by reducing hospitalization and time for post-operative (i.e., post-cholecystectomy) morbidity and recovery" might be an appropriate quantitative statement about estimated costs.

Fourth, specific recommendations are clearly tied to and justified by the estimated benefits, harms, and costs provided within the document.

Fifth, conflicts between this set of guidelines and any other independent sets (and their respective recommendations), if any, must be explicitly discussed.

Strength of Scientific Evidence and Professional Consensus

12. THE GUIDELINE DOCUMENT SPECIFICALLY DESCRIBES THE METHOD(S) USED TO COLLECT (I.E., IDENTIFY AND RETRIEVE) THE SCIENTIFIC EVIDENCE ON WHICH RECOMMENDATIONS ARE BASED.

_____Yes (Go to Question 12.1) _____No (Go to Question 12.2)

12.1. ASSESSOR: Respond to Items 12a-d, below, to assess the methods for collecting scientific evidence; then answer Question 12.1, using your best judgment as to the overall rating for this element of validity. Other factors you judge important should be specifically recorded under "Comments or Other Factors."

12a. The criteria used to include and/or exclude studies are:

_____Adequate _____Inadequate _____Not given/described

12b. The search strategy is:

_____Adequate _____Inadequate _____Not given/described

12c. The sources of information are:

_____Adequate _____Inadequate _____Not given/described

12d. Major studies or other sources of information have been identified.

_____Yes _____No _____Don't know

(Specify)

Now answer:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

12.1. THE METHOD(S) OF COLLECTING SCIENTIFIC EVIDENCE IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments or Other Factors:

>> GO TO QUESTION 13 >>

12.2. THE LACK OF A CLEAR METHOD FOR COLLECTING THE SCIENTIFIC EVIDENCE IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

13. THE GUIDELINE DOCUMENT GIVES ADEQUATE REFERENCES OR CITATIONS TO THE SOURCES OF INFORMATION USED IN DEVELOPING THE GUIDELINES.

_____Yes _____No

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

14. THE GUIDELINE DOCUMENT DISCUSSES IN GENERAL TERMS THE STRENGTH OF THE SCIENTIFIC EVIDENCE ON WHICH RECOMMENDATIONS ARE BASED.

_____Yes _____No

Comments:

15. THE GUIDELINE DOCUMENT EXPLICITLY RATES THE STRENGTH OF THE SCIENTIFIC EVIDENCE.

_____Yes (Go to Question 15.1) _____No (Go to Question 15.2)

15.1 ASSESSOR: Respond to Items 15a-15f, below, to determine whether the method used to rate the strength of the scientific evidence is adequate; then answer Question 15.1 below, using your best judgment as to the overall rating for this element of validity. Other factors you judge important should be specifically recorded under "Comments or Other Factors."

15a. Characteristics of studies used as a basis for guidelines have been described.

_____Yes _____No

15b. Strengths and weaknesses of studies used as a basis for guidelines have been noted.

_____Yes _____No

15c. The way the characteristics, strengths, and weaknesses of studies used as a basis for guidelines have been taken into account (for instance, an explicit weighting scheme) has been clearly described.

_____Yes _____No

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

15d. he way the characteristics, strengths, and weaknesses of studies used as a basis for guidelines have been taken into account (for instance, an explicit weighting scheme) is:

_____Adequate _____Inadequate _____Not given/described

15e. The discussion in the document of possible threats to internal validity and reliability of studies included in the scientific evidence supporting the guidelines is:

_____Adequate _____Inadequate _____No discussion given

15f. The discussion in the document of possible threats to external validity and generalizability of studies included in the scientific evidence supporting the guidelines is:

_____Adequate _____Inadequate _____No discussion given

Now answer:

15.1. OVERALL, THE METHOD USED TO RATE OR WEIGHT THE SCIENTIFIC EVIDENCE IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments or Other Factors:

>> GO TO QUESTION 16>>

15.2. THE LACK OF ANY GENERAL DISCUSSION OR EXPLICIT RATING OF THE STRENGTH OF THE SCIENTIFIC EVIDENCE IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

16. IF A FORMAL METHOD OF SYNTHESIS IS USED TO COMBINE THE SCIENTIFIC EVIDENCE QUANTITATIVELY OR OTHERWISE TO DEVELOP SUMMARY OUTCOME MEASURES THAT REFLECT THE STRENGTH OF THE SCIENTIFIC EVIDENCE, THEN THE GUIDELINE DOCUMENT EXPLICITLY DESCRIBES THE METHOD.

_____Yes, method used and described (Go to Question 16.1)

_____No, method used but not described (Go to Question 16.2)

_____No, no formal method of synthesis used (Go to Question 18)

16.1. ASSESSOR: Respond to Items 16a-16c, below, to determine whether formal methods for synthesizing scientific evidence are satisfactory; then answer Question 16.1 below, using your best judgment as to the overall rating for this element of validity. Other factors you judge important should be specifically recorded under "Comments or Other Factors."

16a. The meta-analytic method(s) is:

_____Adequate _____Inadequate _____Not applicable/used

16b. The decision-analytic model(s) is:

_____Adequate _____Inadequate _____Not applicable/used

16c. Other systematic information synthesis method(s) is:

_____Adequate _____Inadequate _____Not applicable/used

Now answer:

16.1 OVERALL, THE FORMAL METHOD(S) USED TO SYNTHESIZE OR COMBINE SCIENTIFIC EVIDENCE IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments or Other Factors:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

>> GO TO QUESTION 17 >>

16.2. OMISSION OF A DESCRIPTION OF THE METHOD(S) OF SYNTHESIZING THE SCIENTIFIC EVIDENCE IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

17. GIVEN THAT A FORMAL METHOD OF SYNTHESIS IS USED TO COMBINE THE SCIENTIFIC EVIDENCE QUANTITATIVELY OR OTHERWISE TO DEVELOP SUMMARY OUTCOMES MEASURES, THE GUIDELINE DOCUMENT EXPLICITLY REPORTS THE RESULTS OF THAT SYNTHESIS.

_____Yes, method used and results reported (Go to Question 17.1)

_____No, method used but results not reported (Go to Question 17.2)

17.1. RESULTS OF INFORMATION SYNTHESIS ARE:

_____Satisfactory (e.g., summary outcome measure(s) with confidence intervals or discussion of

uncertainty)

_____Conditionally satisfactory (e.g., summary outcome measure(s) without confidence intervals or discussion of uncertainty)

_____Unsatisfactory (e.g., outcome measure(s) are not interpretable, are inconsistent, or are otherwise questionable or erroneous). (Specify)

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

>> GO TO QUESTION 18 >>

17.2. OMISSION OF RESULTS OF SYNTHESIS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

18. IF FORMAL EXPERT OR GROUP JUDGMENT TECHNIQUES ARE USED TO REACH PROFESSIONAL CONSENSUS, THEN THE GUIDELINE DOCUMENT EXPLICITLY DESCRIBES THE TECHNIQUES.

_____Yes, techniques used and described (Go to Question 18.1)

_____No, techniques used but not described (Go to Question 18.2)

_____No, no formal expert or group judgment techniques used (Go to Question 19)

18.1. THE EXPERT OR GROUP JUDGMENT TECHNIQUES ARE:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 19 >>

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

18.2. OMISSION OF A DESCRIPTION OF THE EXPERT OR GROUP JUDGMENT TECHNIQUES IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

19. GIVEN THAT EXPERT OR GROUP JUDGMENT METHOD(S) ARE USED TO REACH PROFESSIONAL CONSENSUS, THE GUIDELINE DOCUMENT EXPLICITLY GIVES INFORMATION ABOUT THE STRENGTH OF PROFESSIONAL CONSENSUS.

_____Yes, techniques used and information given (Go to Question 19.1)

_____No, techniques used but information not given (Go to Question 19.2)

19.1. THE INFORMATION ABOUT THE STRENGTH OF PROFESSIONAL CONSENSUS IS:

_____Satisfactory (e.g., levels of professional consensus given for all major points in the guidelines)

_____Conditionally satisfactory (e.g., levels of professional consensus given for some, but not all, major points in the guidelines)

_____Unsatisfactory (e.g., levels of professional consensus are not interpretable, are inconsistent, or are otherwise questionable or erroneous). (Specify)

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

>> GO TO QUESTION 20 >>

19.2. OMISSION OF EXPLICIT INFORMATION ABOUT THE STRENGTH OF PROFESSIONAL CONSENSUS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Health Benefits and Harms/Risks: Qualitative Descriptions

20. THE GUIDELINE DOCUMENT PROVIDES A QUALITATIVE DESCRIPTION OF THE HEALTH BENEFITS THAT ARE EXPECTED FROM A SPECIFIC HEALTH PRACTICE.

_____Yes (Go to Question 20.1) _____No (Go to Question 20.2)

20.1. THE QUALITATIVE DESCRIPTION OF HEALTH BENEFITS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 21 >>

20.2. OMISSION OF A QUALITATIVE DESCRIPTION OF HEALTH BENEFITS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

21. THE GUIDELINE DOCUMENT PROVIDES A QUALITATIVE DESCRIPTION OF THE POTENTIAL HARMS OR RISKS THAT MAY OCCUR AS A RESULT OF A SPECIFIC HEALTH PRACTICE.

_____Yes (Go to Question 21.1) _____No (Go to Question 21.2)

21.1. THE QUALITATIVE DESCRIPTION OF POTENTIAL HARMS OR RISKS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 22 >>

21.2. OMISSION OF A QUALITATIVE DESCRIPTION OF POTENTIAL HARMS OR RISKS IS:

_____Unimportant _____Minor _____Major

omission_____omission

Comments:

Health Benefits and Harms/Risks: Quantitative Information

22. THE GUIDELINE DOCUMENT PROVIDES QUANTITATIVE INFORMATION OR ESTIMATES ABOUT THE HEALTH BENEFITS TO BE EXPECTED AS A RESULT OF A SPECIFIC HEALTH PRACTICE.

_____Yes (Go to Question 22.1) _____No (Go to Question 22.2)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

22.1. THE QUANTITATIVE INFORMATION ABOUT THE HEALTH BENEFITS IS:

_____Satisfactory (e.g., one or more measures of benefits, including accurate summary or composite measures, with confidence intervals or discussion of uncertainty)

_____Conditionally satisfactory (e.g., one or more measures of benefits, without confidence intervals or discussion of uncertainty)

_____Unsatisfactory (e.g., measures are not interpretable, are inconsistent, or are otherwise questionable or erroneous). (Specify)

Comments:

>> GO TO QUESTION 23 >>

22.2. OMISSION OF QUANTITATIVE INFORMATION AND ESTIMATION OF HEALTH BENEFITS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

>> GO TO QUESTION 24 >>

23. THE GUIDELINE DOCUMENT PROJECTS HEALTH BENEFITS OR OUTCOMES IN TERMS OF ADDITIONAL LIFE EXPECTANCY OR SIMILAR MEASURES, SUCH AS QUALITY-ADJUSTED LIFE YEARS.

_____Yes _____No _____Not applicable/not necessary

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

24. THE GUIDELINE DOCUMENT PROVIDES QUANTITATIVE INFORMATION OR ESTIMATES ABOUT THE POTENTIAL HARMS OR RISKS OCCURRING AS A RESULT OF A SPECIFIC HEALTH PRACTICE.

_____Yes (Go to Question 20.1) _____No (Go to Question 20.2)

24.1. THE QUANTITATIVE INFORMATION ABOUT POTENTIAL HARMS OR RISKS OCCURRING AS A RESULT OF A SPECIFIC HEALTH PRACTICE IS:

_____Satisfactory (e.g., one or more measures of harms or risks, including summary or composite measures, with confidence intervals or discussion of uncertainty)

_____Conditionally satisfactory (e.g., one or more measures of harms or risks, without confidence intervals or discussion of uncertainty)

_____Unsatisfactory (e.g., measure(s) are not interpretable, are inconsistent, or are otherwise questionable or erroneous). (Specify)

Comments:

>> GO TO QUESTION 25 >>

24.2. OMISSION OF QUANTITATIVE INFORMATION ABOUT POTENTIAL HARMS OR RISKS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Health Costs: Qualitative Description

25. THE GUIDELINE DOCUMENT PROVIDES A QUALITATIVE DESCRIPTION OF THE HEALTH COSTS OR EXPENDITURES THAT ARE EXPECTED FROM A SPECIFIC HEALTH PRACTICE.

_____Yes (Go to Question 25.1) _____No (Go to Question 25.2)

25.1. THE QUALITATIVE DESCRIPTION OF EXPECTED HEALTH COSTS OR EXPENDITURES IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO QUESTION 26 >>

25.2. OMISSION OF A QUALITATIVE DESCRIPTION OF EXPECTED HEALTH COSTS OR EXPENDITURES IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Health Costs: Quantitative Description

26. THE GUIDELINE DOCUMENT PROVIDES QUANTITATIVE INFORMATION OR ESTIMATES ABOUT THE HEALTH COSTS OR EXPENDITURES THAT ARE EXPECTED AS A RESULT OF A SPECIFIC HEALTH PRACTICE.

_____Yes (Go to Question 26.1) _____No (Go to Question 26.2)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

26.1. ASSESSOR: Respond to Items 26a-26e, below, to determine whether potential costs and expenditures have been estimated in a satisfactory manner; then answer Question 26.1, using your best judgment as to the overall rating for this element of validity. Other factors you judge important should be specifically recorded under ''Comments or Other Factors."

26a. The cost estimates are done for major subgroups of the patient population, e.g., major risk groups, and for major clinical (diagnostic, therapeutic, etc.) alternatives.

_____Yes _____No

26b. The cost estimates include all the services necessary to achieve the health benefits that are assumed to be achievable.

_____Yes _____No

26c. The cost estimates specify number(s) of services that may be added, substituted, and/or eliminated if the guideline recommendations are followed.

_____Yes _____No

26d. The cost estimates specify charges, production costs, or similar information for the services that may be added, substituted, and/ or eliminated if the guideline recommendations are followed.

_____Yes _____No

26e. The quantitative method(s) used to estimate costs is:

_____Appropriate _____Inappropriate

Now answer:

26.1 THE QUANTITATIVE INFORMATION ABOUT EXPECTED HEALTH COSTS OR EXPENDITURES IS:

_____Satisfactory (e.g., one or more estimates of costs, including accurate summary or composite measures, with ranges of uncertainty)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

_____Conditionally satisfactory (e.g., one or more estimates of costs, without ranges of uncertainty)

_____Unsatisfactory (e.g., cost estimates are not interpretable, are inconsistent, or are otherwise questionable or erroneous). (Specify)

Comments or Other Factors:

>> GO TO QUESTION 27 >>

26.2. OMISSION OF QUANTITATIVE INFORMATION ABOUT EXPECTED HEALTH COSTS OR EXPENDITURES IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

>> GO TO QUESTION 28 >>

27. IF HEALTH BENEFITS ARE PROJECTED IN TERMS OF ADDITIONAL LIFE EXPECTANCY OR SIMILAR MEASURES, SUCH AS QUALITY-ADJUSTED LIFE YEARS, THEN THE COST PER UNIT OF EACH IDENTIFIED BENEFIT IS ESTIMATED.

_____Yes, benefits projected in such terms and cost per unit estimated

_____No, benefits projected in such terms but cost per unit not estimated

_____Not applicable, benefits not so projected and cost per unit not estimated

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

28. GENERALLY, THE ESTIMATES OF BENEFITS, HARMS, AND COSTS ARE CONSISTENT WITH THE STRENGTH OF THE EVIDENCE PRESENTED IN THE GUIDELINE DOCUMENT.

_____Yes, completely _____Yes, partially _____No

Comments:

29. DOES THE GUIDELINE DOCUMENT MAKE MAJOR RECOMMENDATIONS?

_____Yes (List below, and then go to Question 30)

_____No (Go to Question 31)

ASSESSOR: Briefly list in the space below the recommendations from the guideline document that the developers consider major. If the developers have not specifically indicated which are their major recommendations, please list those that you have used in answering the questions about the strength of scientific evidence.

30. THE GUIDELINE DOCUMENT EXPLICITLY DISCUSSES THE STRENGTH OF THE SCIENTIFIC EVIDENCE ON WHICH EACH MAJOR RECOMMENDATION IS BASED.

_____Yes (Go to Question 30.1)

_____No (Go to Question 30.2)

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

30.1. THE DISCUSSION OF THE STRENGTH OF THE EVIDENCE ON WHICH EACH MAJOR RECOMMENDATION IS BASED IS:

_____Satisfactory for all recommendations

_____Conditionally satisfactory—i.e., satisfactory for some but not all recommendations

_____Unsatisfactory—i.e., not satisfactory for most or all recommendations

_____Comments:

>> GO TO QUESTION 31 >>

30.2. OMISSION OF A DISCUSSION OF THE STRENGTH OF THE SCIENTIFIC EVIDENCE FOR EACH MAJOR RECOMMENDATION IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

31. EACH MAJOR RECOMMENDATION IS CONSISTENT WITH THE ESTIMATED BENEFITS, HARMS, AND COSTS OF THE SERVICE OR INTERVENTION (AND THUS WITH THE STRENGTH OF THE EVIDENCE).

_____Yes, completely _____Yes, partially _____No

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Potential Conflict Among Similar Sets of Guidelines

32. THE GUIDELINE DOCUMENT IDENTIFIES OTHER SETS OF GUIDELINES THAT DEAL WITH THE SAME CLINICAL CONDITION, TECHNOLOGY, OR TOPIC.

_____Yes (Go to Question 33)

_____No, but similar sets of guidelines are known to exist (Specify below and go to Question 33.2)

_____Not applicable, no similar sets of guidelines are known to exist (Go to V. CLARITY)

33. THE GUIDELINE DOCUMENT IDENTIFIES POSSIBLE CONFLICTS AMONG EXISTING GUIDELINES AND THE REASONS FOR THEM.

_____Yes (Go to Question 33.1) _____No (Go to Question 33.2)

33.1. THE DISCUSSION OF POSSIBLE CONFLICTS AMONG GUIDELINES IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

>> GO TO V. CLARITY >>

33.2. OMISSION OF A DISCUSSION OF SIMILAR GUIDELINES, OR OF POSSIBLE CONFLICTS AMONG GUIDELINES, IS:

_____Unimportant _____Minor _____Major

omission _____omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

V. CLARITY

Clarity means that guidelines are written in unambiguous language and terms, that the logic of the recommendations is clear and straightforward, and that the guideline document has a clear and easy-to-understand structure and format. That is, clarity encompasses the language and the logic with which the guideline document is written and the way it is physically presented. Clarity applies to three content areas of guidelines: (1) a general framework in which health condition(s), health practice(s), patient care goals, and similar topics are defined and discussed; (2) presentation and discussion of the evidence used in developing the guidelines; and (3) recommendations.

More specifically, this attribute requires that, as described below, certain things about language and terms, logic, and structure must be true.

Language and Terms

The guidelines are written in unambiguous language. Vague terms are avoided when describing the patient populations, health conditions, the health interventions, and the recommendations. For example, expressions such as "severe bleeding" are avoided in favor of (or at least qualified by) more precise language, such as a "drop in hematocrit of more than 6 percent in less than 8 hours." Or, for instance, a recommendation such as ''thyroid function tests should be obtained whenever appropriate" is replaced by a recommendation that includes the type of test, its frequency, and the specific circumstances under which it should be used, such as "once every 5 years in otherwise healthy adults more than 65 years of age."

34. THE GUIDELINES DESCRIBE THE HEALTH CONDITION TO BE PREVENTED, DETECTED, OR TREATED IN UNAMBIGUOUS TERMS.

_____Yes _____No

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

35. THE GUIDELINES DESCRIBE THE OPTIONS FOR MANAGEMENT OF THE HEALTH CONDITION (I.E., THE HEALTH PRACTICE AND ITS ALTERNATIVES) IN UNAMBIGUOUS TERMS.

_____Yes _____No

Comments:

36. IF THE GUIDELINES GIVE MAJOR RECOMMENDATIONS, EACH IS WRITTEN IN UNAMBIGUOUS TERMS.

ASSESSOR: Refer to the list you developed for Question 29 in answering this question.

Yes _____No _____Not applicable, no major recommendations given

Comments:

Logic

The guidelines are as comprehensive as possible in keeping with the attributes "clinical adaptability" and "clinical flexibility." Thus, the logic of the guidelines is such that all clinically important and relevant situations are handled in a consistent, reasonable, and easy-to-follow manner and that situations that are not covered are explained in a logically appropriate place in the guideline statement.

Recommendations are mutually exclusive; that is, they are consistent with each other. For example, a guideline does not recommend "aortic valvuloplasty for an 80-year-old man with end stage renal disease" in one place and "aortic valve replacement for an 80-year-old man with end stage renal disease" in another.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

37. RECOMMENDATIONS ARE COMPREHENSIVE, INSOFAR AS THE EVIDENCE PERMITS, AND RECOMMENDATIONS THAT MIGHT BE EXPECTED ARE GIVEN.

(That is, the recommendations collectively cover all clinically relevant circumstances.)

_____Yes (Go to Question 38) _____No (Go to Question 37.1)

Comments:

37.1 IF EXPECTED RECOMMENDATIONS SEEM TO BE MISSING, THE GUIDELINE DOCUMENT DISCUSSES WHY.

_____Yes _____No

Comments:

38. RECOMMENDATIONS ARE CONSISTENT.

(That is, no two recommendations in the guidelines conflict with each other.)

_____Yes

_____No (at least two recommendations appear to conflict with each other)

_____Not applicable, no recommendations given

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Structural Clarity

The overall organization and appearance of the guideline document and the mode of presentation of the recommendations are easy for users to understand and follow. A structurally clear guideline is one in which the recommendations are easily accessible to the prospective user. That is, clinicians should not have to read, analyze critically, and distill a detailed manuscript in order to find needed recommendations. Structural clarity may be achieved through the use of a summary, special highlighting techniques, algorithms, or other methods.

39. THE GUIDELINE DOCUMENT USES CLEAR HEADINGS, INDEXES, LISTS, FLOW CHARTS, OR OTHER DEVICES TO IDENTIFY MAJOR TOPICS DISCUSSED.

_____Yes _____No

Comments:

40. THE GUIDELINE DOCUMENT HAS A SUMMARY OR ABSTRACT THAT ACCURATELY REFLECTS THE METHODS, CONTENT, AND RECOMMENDATIONS OF THE ENTIRE DOCUMENT.

_____Yes _____No

Comments:

41. A USER OF THE GUIDELINE DOCUMENT CAN EASILY FIND EACH MAJOR RECOMMENDATION.

ASSESSOR: Refer to the list developed for Question 29 in answering this question.

_____Yes _____No _____Not applicable, no major recommendation given

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

VI. SCHEDULED REVIEW

Scheduled review means that a statement specifying a date for review and possible revision of the guideline has been included in the guideline document. Revisions to guidelines should reflect new clinical evidence or changing professional consensus.

This attribute requires that the guideline document either (1) give a specific date for review and possible revision of the guidelines or (2) describe a process by which such a date might be established and the review and possible revision performed.

42. THE GUIDELINE DOCUMENT GIVES A SPECIFIC DATE FOR SCHEDULED REVIEW, GIVES OTHER INFORMATION CONCERNING A PROCEDURE BY WHICH SCHEDULED REVIEW MIGHT BE DONE, OR GIVES A SUNSET OR EXPIRATION DATE.

_____Yes (Go to Question 42.1) _____No (Go to Question 42.2)

42.1. ASSESSOR: Respond to Items 42a-42d, below, to determine whether the scheduled review date information is satisfactory, then answer Question 42.1 below, using your best judgment as to the overall rating for this attribute of scheduled review. Other factors you judge important should be specifically recorded under "Comments or Other Factors."

42a. The target date for review is:

_____Appropriate _____Inappropriate _____None given/discussed

42b. The rationale for the target date is:

_____Adequate _____Inadequate _____Not applicable

42c. The procedures suggested for determining when the guidelines should be reviewed are:

_____Appropriate _____Inappropriate _____None given/discussed

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

42d. The guideline has a sunset provision that may dictate when a scheduled review should take place or that may indicate when the guideline will expire.

_____Yes _____No

Now answer:

42.1. THE SCHEDULED REVIEW DATE OR PROCEDURE FOR SETTING IT IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments or Other Factors:

>> Go to VII. MULTI-DISCIPLINARY PROCESS >>

42.2. THE LACK OF A SCHEDULED REVIEW DATE OR PROCEDURE FOR SETTING IT IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

VII. MULTI-DISCIPLINARY PROCESS

A multi-disciplinary process for practice guidelines means that representatives of a broad range of practitioners, consumers or patients, and other groups likely to be affected by the guidelines have participated in the development process at some stage. These representatives can be individuals who have had direct responsibility for the guideline document or individuals who have reviewed that document or in other ways have contributed to it. This attribute intends that both methodologic and clinical disciplines be involved in the guideline-development process. This document cannot identify in advance all relevant participants, interested parties, or disciplines because each set of guidelines will differ in this respect.

This attribute requires that five things be true. First, some combination of individuals directly responsible for guidelines and those who have otherwise contributed to their development collectively represents all the key groups likely to affect or to be affected by the guidelines. Second, the guideline document describes the parties involved (including their credentials and potential biases); "the parties involved" is understood to mean participants in the actual development panel and those in review panels, public hearings, or other review forums. Third, potential biases and conflicts of interests have been discussed or otherwise appropriately taken account of. Fourth, the methods used to solicit panelists' views and arrive at group judgments have been described and are adequate and appropriate to the task of balancing views and potential biases. Fifth, the methods used to solicit outside review comments and present those to panelists have been described and are adequate to the task of making outside views clear to panelists.

43. PERSONS WITH APPROPRIATE CLINICAL AND METHODOLOGIC DISCIPLINES PARTICIPATED IN DEVELOPING THE GUIDELINE DOCUMENT-THAT IS, A MULTI-DISCIPLINARY APPROACH WAS FOLLOWED.

_____Yes (Go to Question 43.1)

_____No (Go to Question 43.2)

_____Don't know or can't tell (Go to Question 43.2)

43.1. ASSESSOR: Respond to Items 43a-43i, below, to determine whether the multi-disciplinary process is satisfactory; then answer Question 43.1 below, using your best judgment as to the overall rating for this element of multi-disciplinary process. Other factors you judge important should be specifically recorded under "Comments or Other Factors."

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

43a. An explanation, discussion, or rationale for selecting the guideline panel chairperson is given.

_____Yes _____No

43b. An explanation, discussion, or rationale for selecting the members of the guideline panel is given.

_____Yes _____No

43c. An explanation, discussion, or rationale for selecting other individuals directly responsible for the guideline document (such as consultants) is given.

_____Yes _____No

43d. The explanation(s), discussion(s), or rationale(s) for selecting the individuals covered in 43a-c is (are):

_____Adequate _____Inadequate _____Not applicable

43e. These individuals reflect all appropriate interest groups and disciplines.

_____Yes _____No _____Can't tell

43f. One or more outside review panel(s) commented on or reviewed draft guidelines.

_____Yes _____No _____Can't tell

43g. One or more public hearing(s) or similar review mechanism(s) were held to allow comment or review on draft guidelines.

_____Yes _____No _____Can't tell

43h. Collectively, the review panel(s), public hearing(s), or other review mechanisms reflected all appropriate interest groups and disciplines.

_____Yes _____No _____Can't tell

43i. If the answer to either question 43e or question 43h is "No" or "Can't tell," please record what groups or disciplines appear to have been omitted.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Now answer:

43.1. THE MULTI-DISCIPLINARY APPROACH TO THE GUIDELINES DEVELOPMENT PROCESS IS:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments or Other Factors:

43.2. THE LACK (OR APPARENT LACK) OF A MULTIDISCIPLINARY PROCESS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

44. THE GUIDELINE DOCUMENT EXPLICITLY NOTES ANY POTENTIAL BIASES AND/OR CONFLICTS OF INTERESTS OF THE PANEL MEMBERS, OR STATES THAT BIASES AND CONFLICTS OF INTEREST WERE DISCUSSED AMONG PANEL MEMBERS OR OTHERWISE TAKEN INTO ACCOUNT.

_____Yes, potential biases and/or conflicts of interest are noted

_____Yes, a statement that biases and/or conflicts of interest were discussed is given

_____No, no note or statement about biases and/or conflicts of interest is given

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

45. OVERALL, POTENTIAL BIASES AND/OR CONFLICTS OF INTEREST APPEAR TO BE ADEQUATELY BALANCED OR OTHERWISE ACCOUNTED FOR IN THE GUIDELINE DEVELOPMENT PROCESS.

_____Yes _____No _____Don't know or can't tell

(Specify)

Comments:

46. THE GUIDELINE DOCUMENT DESCRIBES THE METHODS USED TO SOLICIT VIEWS OF INTERESTED PARTIES NOT ON THE GUIDELINES DEVELOPMENT PANEL AND TO PRESENT THOSE VIEWS TO THE MEMBERS OF PANEL.

_____Yes (Go to Question 46.1) _____No (Go to Question 46.2)

46.1. THE METHODS USED TO SOLICIT VIEWS OF THOSE NOT ON THE PANELS AND PRESENT THOSE VIEWS TO PANELS ARE:

_____Satisfactory _____Conditionally _____Unsatisfactory

satisfactory (Specify)

Comments:

46.2. THE LACK OF A DESCRIPTION OF THE METHODS USED TO SOLICIT VIEWS OF THOSE NOT ON THE PANELS AND TO PRESENT THOSE VIEWS TO PANELS IS:

_____Unimportant _____Minor _____Major

omission omission

Comments:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

PLEASE RECORD ANY SUMMARY JUDGMENTS OR OTHER COMMENTS YOU MAY HAVE AND ANY RECOMMENDATIONS FOR ADDITIONAL REVIEW.

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

PART THREE. SUMMARY EVALUATION SHEET

Instructions and Key

ASSESSOR: Upon completing the entire assessment instrument, please record answers to the main questions (Questions 1-46) below. Circle the relevant answer, according to the following key:

KEY

Y = Yes; YQ = yes, but response qualified;

N = No; NQ = no, but response qualified;

S = Satisfactory, CS = Conditionally satisfactory, US = Unsatisfactory;

UN = Unimportant, MI = Minor omission, MA = Major omission;

NA = Not applicable

DK = Don't know, or can't tell

I. CLINICAL APPLICABILITY

Y

S

UN

N

CS

MI

US

MA

1

Description of patient population

1.1. Quality of description

1.2. Omission of description

Y

S

UN

N

CS

MI

NA

US

MA

2

Discussion of complex clinical problems

2.1. Quality of discussion

2.2. Omission of discussion

Y

S

UN

N

CS

MI

US

MA

3.

Rationale for excluding patient populations

3.1. Quality of rationale

3.2. Omission of rationale

II. CLINICAL FLEXIBILITY

Y

S

UN

N

CS

MI

US

MA

4.

Information about acceptable clinical exceptions

4.1. Quality of information or statement

4.2. Omission of information or statement

Y

S

UN

N

CS

MI

US

MA

5.

Information about acceptable nonclinical exceptions

5.1. Quality of information or statement

5.2. Omission of information or statement

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Y

S UN

N

CS MI

US MA

6.

Discussion of patient preferences in the health care decisions

6.1. Quality of discussion 

6.2. Omission of discussion

Y

S UN

N

CS MI

US MA

7.

Discussion of patient preferences in guideline development

7.1. Quality of discussion 

7.2. Omission of discussion

III. RELIABILITY/REPRODUCIBILITY

Y

S

N CS

US

8.

Independent review by experts or outside panels 

8.1. Quality of discussion

Y

S UN

N CS MI

US MA

9.

Explanation of lack of independent review 

9.1. Quality of explanation 

9.2. Omission of explanation

Y

S

N CS

US

10.

Guidelines pretested in some manner 

10.1. Quality of discussion

Y

S UN

N CS MI

US MA

11.

Explanation of lack of pretesting 

11.1. Quality of explanation 

11.2. Omission of explanation

IV. VALIDITY

STRENGTH OF THE SCIENTIFIC EVIDENCE AND PROFESSIONAL CONSENSUS

Y

S UN

N

CS MI

US MA

12.

Method of collecting (identifying and retrieving) scientific evidence is specifically described

12.1. Quality of method 

12.2. Lack of method

Y

N

 

13.

Adequate references to sources of scientific evidence

Y

N

 

14.

General discussion of strength of scientific evidence

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

Y

S

UN

N

CS MI

US

MA

15.

Explicit rating of the strength of the scientific evidence

15.1. Quality of rating method 

15.2. Lack of general discussion of rating method

Y

S

UN

N

CS MI

NQ

US

MA

16.

If a formal method of synthesis is used, explicit description of the method

16.1. Quality of formal method 

16.2. Omission of description of formal method

Y

S

UN

N

CS

MI

US

MA

17.

If applicable, the results of a formal synthesis of scientific evidence are explicitly reported

17.1. Quality of results of the synthesis 

17.2. Omission of results of the synthesis

Y

N

NQ

18.

If applicable, the expert or group judgment techniques used for reaching professional consensus are explicitly described

S

UN

CS MI

US

MA

 

18.1. Quality of expert or group judgment techniques 

18.2. Omission of description of expert or group judgment techniques

Y

N

 

19

If applicable, the strength of professional consensus resulting from use of group judgment techniques is reported

S

UN

CS

MI

US

MA

 

19.1. Quality of information about strength of professional consensus 

19.2. Omission of explicit information about strength of professional consensus

HEALTH BENEFITS AND HARMS/RISKS: QUALITATIVE DESCRIPTION

Y

S

UN

N

CS MI

 

US

MA

20.

Qualitative description of health benefits 

20.1. Quality of qualitative description

20.2. Omission of qualitative description

Y

S

UN

N CS MI

US

MA

21.

Qualitative description of potential harms or risks 

21.1. Quality of qualitative description 

21.2. Omission of qualitative description

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

HEALTH BENEFITS AND HARMS/RISKS: QUANTITATIVE INFORMATION

Y

S

UN

N

CS

MI

US

MA

22.

Quantitative information or estimates of health benefits

22.1. Quality of quantitative information

22.2. Omission of quantitative information

Y

N

NA

23.

Health benefits projected in terms of life

expectancy or similar measures harms or risks

Y

S

UN

N

CS

MI

US

MA

24.

Quantitative information or estimates of potential

24.1. Quality of quantitative information

24.2. Omission of quantitative information

HEALTH COSTS: QUALITATIVE DESCRIPTION

Y

S

UN

N

CS

MI

US

MA

25.

Qualitative description of health costs or expenditures

25.1 Quality of qualitative description

25.2. Omission of qualitative description

HEALTH COSTS: QUANTITATIVE DESCRIPTION

Y

S

UN

N

CS

MI

US

MA

26.

Quantitative information or estimates of health costs or expenditures

26.1. Quality of quantitative information

26.2. Omission of quantitative information

Y

N

NA

27.

If health benefits projected in terms of life expectancy or similar measures, costs per unit

of each identified benefit also estimated

Y

YQ N

 

28.

Generally, estimates of benefits, harms, and costs are consistent with the strength of provided

evidence

Y

N

 

29.

Major recommendations made in the guideline

Y

N

 

30.

Discussion of strength of the scientific evidence for each major recommendation

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

S

UN

C S

MI

US

MA

 

30.1. Quality of discussion

30.2. Omission of discussion

Y

YQ

N

31.

Each major recommendation consistent with

strength of scientific evidence

POTENTIAL CONFLICT AMONG SIMILAR SETS OF GUIDELINES

Y

N

NA

32.

Other sets of guidelines identified

Y

S

UN

N

CS

MI

US

MA

33.

Possible conflicts among existing guidelines discussed

33.1. Quality of discussion

33.2. Omission of discussion

V. CLARITY

LANGUAGE AND TERMS

Y

N

 

34.

Language describing the health condition is unambiguous

Y

N

 

35.

Language describing the options for management is unambiguous

Y

N

NA

36.

Language for each major recommendation is unambiguous

LOGIC

Y

Y

N

N

 

37.

Recommendations are comprehensive and present when expected

37.1. Reasons given for lack of expected recommendations

Y

N

NA

38.

Recommendations are consistent

STRUCTURAL CLARITY

Y

N

 

39.

Guideline document uses clear headings, indexes, etc.

Y

N

 

40.

Guideline document has accurate summary or abstract

Y

N

NA

41.

Users can find recommendations easily

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×

VI. SCHEDULED REVIEW: DEFINITION AND EVALUATION QUESTIONS

Y

S

UN

N

CS MI

US MA

42.

Scheduled date for review or a procedure for arriving at such a date is provided

42.1. Quality of the scheduled review date or procedure for setting one

42.2. Lack of a scheduled review date or procedure for setting one

VII. MULTI-DISCIPLINARY PROCESS

Y

S

UN

N

CS

MI

DK

US

MA

43.

Participation of persons in appropriate clinical and methodologic disciplines

43.1. Quality of the multi-disciplinary approach

43.2. Lack of a multi-disciplinary process

Y

YQ

N

44.

Guideline document notes potential biases or conflicts of interest or indicates they were taken into account

Y

N

DK

45.

Balance of potential biases or conflicts of interest

Y

S

UN

N

CS

MI

 

US

MA

46.

Description of the methods used to solicit views of those not on the guidelines development

panel and to present those views to the panel

46.1. Quality of methods used

46.2. Lack of a description of methods used

SUMMARY JUDGMENT, OTHER COMMENTS, OR NEED FOR ADDITIONAL REVIEW:

Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 346
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 347
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 348
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 349
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 350
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 351
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 352
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 353
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 354
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 355
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 356
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 357
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 358
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 359
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 360
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 361
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 362
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 363
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 364
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 365
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 366
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 367
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 368
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 369
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 370
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 371
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 372
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 373
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 374
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 375
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 376
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 377
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 378
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 379
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 380
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 381
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 382
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 383
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 384
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 385
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 386
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 387
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 388
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 389
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 390
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 391
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 392
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 393
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 394
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 395
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 396
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 397
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 398
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 399
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 400
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 401
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 402
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 403
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 404
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 405
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 406
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 407
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 408
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 409
Suggested Citation:"B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES." Institute of Medicine. 1992. Guidelines for Clinical Practice: From Development to Use. Washington, DC: The National Academies Press. doi: 10.17226/1863.
×
Page 410
Next: C: ROSTERS »
Guidelines for Clinical Practice: From Development to Use Get This Book
×
Buy Paperback | $110.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF

Guidelines for the clinical practice of medicine have been proposed as the solution to the whole range of current health care problems. This new book presents the first balanced and highly practical view of guidelines—their strengths, their limitations, and how they can be used most effectively to benefit health care.

The volume offers:

  • Recommendations and a proposed framework for strengthening development and use of guidelines.
  • Numerous examples of guidelines.
  • A ready-to-use instrument for assessing the soundness of guidelines.
  • Six case studies exploring issues involved when practitioners use guidelines on a daily basis.

With a real-world outlook, the volume reviews efforts by agencies and organizations to disseminate guidelines and examines how well guidelines are functioning—exploring issues such as patient information, liability, costs, computerization, and the adaptation of national guidelines to local needs.

  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook, NAP.edu's online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    Switch between the Original Pages, where you can read the report as it appeared in print, and Text Pages for the web version, where you can highlight and search the text.

    « Back Next »
  6. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  7. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  8. ×

    View our suggested citation for this chapter.

    « Back Next »
  9. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!