Guidelines for Clinical Practice From Development to Use (1992) / Chapter Skim
Currently Skimming:
B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES
B: A PROVISIONAL INSTRUMENT FOR ASSESSING CLINICAL PRACTICE GUIDELINES
Pages 346-410
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 346... ...
of the National Academy of Sciences has been engaged since the beginning of 1990 in two projects relating to the development, implementation, and evaluation of clinical practice guidelines. One IOM committee defined practice guidelines as "systematically developed statements to assist practitioners and patients in choosing appropriate health care for specific clinical conditions." It also delineated several desirable attributes of guidelines that are intended to help users understand the elements of a sound guideline and to recognize good (or not-so-good)
|
From page 347... ...
PURPOSES OF THE ASSESSMENT INSTRUMENT The central purpose of the IOM's instrument for assessing clinical practice guidelines is to provide an explicit method for examining the soundness of such guidelines and to encourage their systematic development. By assessment is meant a prospective judgment of the soundness of both the process used in developing a guideline and the resulting guideline.
|
From page 348... ...
First, with the aid of an outside consultant, staff drafted a set of questions to operationalize the eight conceptual attributes of good practice guidelines identified in the IOM's 1990 report on guidelines.2 (Discussions of these attributes introduce sections of the assessment instrument.) Second, these questions were combined with background information and instructions for users and subjected to considerable internal and external review, as described below.
|
From page 349... ...
In addition, application of an instrument such as this is, in itself, a thought-provoking exercise for individuals active in developing clinical practice guidelines." By contrast, negative comments were of the following kind: · "The form seems excessively lengthy and not particularly user-friendly." · "The instructions are verbose and redundant, and almost legalistic. They are not user-friendly." · "This instrument was very confusing, almost 'impossible."' · "Instrument of intellectual torture.
|
From page 350... ...
The "provisional" form in this document reflects the reactions of this group of seven experts, and many of their comments have been incorporated into the discussion presented here. IMPORTANT FEATURES OF THE PROVISIONAL ASSESSMENT INSTRUMENT Attributes of Practice Guidelines Types of Attributes Four attributes identified in the first IOM report on practice guidelines concern the substance of the guidelines clinical applicability or scope, clinical flexibility, reliability/reproducibility, and validity.
|
From page 351... ...
For example, the discussion or description should be thorough and comprehensive; the guideline developers should have based their work on appropriate and correct information (e.g., from the literature review) ; and they should have used appropriate methods (e.g., for evaluating the strength of the scientific evidence or reaching professional peer consensus)
|
From page 352... ...
Omitting such a description or discussion is of minor importance if it seems likely to affect negatively the ability of a guideline user or of evaluators to apply the guideline effectively or independently to assess its quality. Finally, omitting such a description or discussion is a major omission if the absence of such information essentially prevents guideline users or evaluators from applying the guideline effectively or even making an independent evaluation about the soundness of the guidelines document itself (at least on that particular feature)
|
From page 353... ...
Thus, it does not propose any formal weighting or numerical scoring scheme for the main questions, nor does it suggest a particular threshold, cutpoint, or floor against which current guidelines might be judged acceptable or unacceptable. If most responses to the questions are "satisfactory" (or "unimportant omissions" however, one might reasonably conclude that such a guidelines document would be sufficient for most clinical situations.
|
From page 354... ...
This might provide a rough indicator of whether the guidelines document could be used effectively in clinical situations. One reviewer, for example, suggested that a report for busy practicing physicians might usefully include "a graphic summary of the degree to which each attribute was successfully achieved, e.g., a bar representing the percent of key items within each attribute that were deemed satisfactory .
|
From page 355... ...
Several reviewers noted that completing the summary evaluation sheet is essentially a clerical task, provided the main part of the assessment instrument has been legibly and fully completed by one or more experts (as discussed earlier)
|
From page 356... ...
are sufficiently schooled either in the methodologic issues inherent in guidelines development or in the clinical issues related to the main topic of the guidelines document that they are able to complete the bulk of the assessment instrument unaided. Second, questions about clinical topics or methods can be referred to appropriate experts when necessary and without undue delay.
|
From page 357... ...
Primary and secondary publications, reports, and records relating to the development of the guideline document being assessed should be assembled before the assessment exercise begins. This might include, for example, reviews and syntheses of 5 For example, the AHCPR Forum panel that has been working on the issue of managing depression in community-based settings started originally with 50,000 citations to the literature, reviewed between 4,000 and 6,000 articles, and based its guidelines document on about 400 relevant articles (J.
|
From page 358... ...
For instance, simply assessing whether guideline developers explain or document a certain piece of information does not allow one to discriminate a comprehensive disclosure from one of poor quality. Similarly, lack of disclosure may have a significant or only a trivial impact on the clinical usefulness and validity of a guideline.
|
From page 359... ...
THE PROVISIONAL ASSESSMENT INSTRUMENT The form reproduced in the last part of this appendix has three main parts. First is a general information sheet, with space for the following items to be briefly described: clinical diagnoses or conditions; health practices, services, or technologies; target populations; primary settings of care; primary types of clinicians targeted; stated purposes of the guideline; source, author, or developer of the guideline document; person to contact for further information about the guideline document; date of issue of the guideline document; and name/affiliation of assessors.
|
From page 360... ...
. Clinical Practice Guidelines: Directions for a New Program.
|
From page 361... ...
Clinical diagnoses or conditions 2. Main health practices, services, or technologies considered 3.
|
From page 363... ...
The meaning of these response categories is as follows: Yes and no. Most of the main questions concern the presence of a discussion or piece of information about a particular attribute.
|
From page 364... ...
The omission of a description or discussion about a specific attribute or dimension is of minor importance if it (1) is likely to have only a little negative impact on the ability of a guideline user to apply the guideline effectively in the clinical decision-making process and (2)
|
From page 365... ...
Third, when the clinical conditions or problems covered by the guideline are likely to be complex, or when the guideline recommendations may be contingent on complex patterns of clinical factors, those points should be explicitly covered in the guideline document. This attribute requires that two things be true about the guideline document.
|
From page 368... ...
, or all of these; if no such factors exist, the guideline document should say so. Second is the role of patient preferences for different possible outcomes of care, when the appropriateness of a clinical intervention involves a substantial element of personal choice or values on the part of the patient.
|
From page 369... ...
. omission THE GUIDELINE DOCUMENT PROVIDES SPECIFIC INFORMATION ABOUT NONCLINICAL SITUATIONS IN WHICH EXCEPTIONS MIGHT BE MADE IN APPLYING THE GUIDELINES.
|
From page 370... ...
THE GUIDELINE DOCUMENT DISCUSSES THE ROLE OF PATIENT PREFERENCES, AS THEY RELATE TO HEALTH CARE DECISIONS IN THE PARTICULAR CASE THAT THE GUIDELINES COVER. Yes (Go to Question 6.1' No (Go to Question 6.2)
|
From page 371... ...
>> GO TO III. RELIABILITY/REPRODUCIBILITY >> 7.2.0MISSION OF THE DISCUSSION OF PATIENT PREFERENCES IN DEVELOPING THE GUIDELINE IS: Comments: Unimportant Minor omission Major Omission
|
From page 372... ...
Reliability and reproducibility of a guideline document is not likely ever to be assessable empirically. To approach these concepts, therefore, this attribute requires either that guidelines be subjected to some form of explicit, independent review by a group (or groups)
|
From page 373... ...
>> GO TO QUESTION 10 9.2. OMISSION OF AN EXPLANATION OF THE LACK OF INDEPENDENT REVIEW IS: Comments: Unimportant Minor omission Major omission 10.
|
From page 374... ...
THE GUIDELINE DOCUMENT EXPLAINS THE LACK OF PRETESTING. APPENDIX B Yes (Go to Question 11.1)
|
From page 375... ...
First, the collection, synthesis, and interpretation of scientific evidence must be documented and of satisfactory quality; ideally, each major recommendation will be described as based on "excellent," "acceptable," or "weak" evidence, or with a similar set of descriptive terms. Second, both qualitative and quantitative statements about health benefits and harms/risks appear in the guideline document, and insofar as possible those estimates are tied to and justified by the evidence amassed as part of the literature review and analysis.
|
From page 376... ...
, if any, must be explicitly discussed. Strength of Scientific Evidence and Professional Consensus 12.
|
From page 377... ...
THE METHOD(S) OF COLLECTING SCIENTIFIC EVIDENCE IS: Satisfactory Comments or Other Factors: Conditionally Unsatisfactory satisfactory (Specify)
|
From page 378... ...
THE GUIDELINE DOCUMENT EXPLICITLY RATES THE STRENGTH OF THE SCIENTIFIC EVIDENCE. Yes (Go to Question 15.
|
From page 379... ...
The discussion in the document of possible threats to external validity and generalizability of studies included in the scientific evidence supporting the guidelines is: Now answer: Adequate Inadequate No discussion given 15.1. OVERALL, THE METHOD USED TO RATE OR WEIGHT THE SCIENTIFIC EVIDENCE IS: Satisfactory Conditionally satisfactory Comments or Other Factors: Unsatisfactory (Specify)
|
From page 380... ...
IF A FORMAL METHOD OF SYNTHESIS IS USED TO COMBINE THE SCIENTIFIC EVIDENCE QUANTITATIVELY OR OTHERWISE TO DEVELOP SUMMARY OUTCOME MEASURES THAT REFLECT THE STRENGTH OF THE SCIENTIFIC EVIDENCE, THEN THE GUIDELINE DOCUMENT EXPLICITLY DESCRIBES THE METHOD. Yes, method used and described (Go to Question 16.1)
|
From page 381... ...
GIVEN THAT A FORMAL METHOD OF SYNTHESIS IS USED TO COMBINE THE SCIENTIFIC EVIDENCE QUANTITATIVELY OR OTHERWISE TO DEVELOP SUMMARY OUTCOMES MEASURES, THE GUIDELINE DOCUMENT EXPLICITLY REPORTS THE RESULTS OF THAT SYNTHESIS. Yes, method used and results reported (Go to Question 17.1)
|
From page 382... ...
IF FORMAL EXPERT OR GROUP JUDGMENT TECHNIQUES ARE USED TO REACH PROFESSIONAL CONSENSUS, THEN THE GUIDELINE DOCUMENT EXPLICITLY DESCRIBES THE TECHNIQUES. YeS, teChniqUeS USed and described (GO to QUeStiOn 18.1)
|
From page 383... ...
ARE USED TO REACH PROFESSIONAL CONSENSUS, THE GUIDELINE DOCUMENT EXPLICITLY GIVES INFORMATION ABOUT THE STRENGTH OF PROFESSIONAL CONSENSUS. Yes, techniques used and information given (Go to Question 19.1)
|
From page 384... ...
THE GUIDELINE DOCUMENT PROVIDES A QUALITATIVE DESCRIPTION OF THE HEALTH BENEFITS THAT ARE EXPECTED FROM A SPECIFIC HEALTH PRACTICE. Yes (Go to Question 20.1 )
|
From page 385... ...
THE GUIDELINE DOCUMENT PROVIDES A QUALITATIVE DESCRIPTION OF THE POTENTIAL HARMS OR RISKS THAT MAY OCCUR AS A RESULT OF A SPECIFIC HEALTH PRACTICE. Yes (Go to Question 21.1)
|
From page 386... ...
THE GUIDELINE DOCUMENT PROJECTS HEALTH BENEFITS OR OUTCOMES IN TERMS OF ADDITIONAL LIFE EXPECTANCY OR SIMILAR MEASURES, SUCH AS QUALITY-ADJUSTED LIFE YEARS. Comments: Yes No Not applicable/not necessary
|
From page 387... ...
THE GUIDELINE DOCUMENT PROVIDES QUANTITATIVE INFORMATION OR ESTIMATES ABOUT THE POTENTIAL HARMS OR RISKS OCCURRING AS A RESULT OF A SPECIFIC HEALTH PRACTICE. Comments: Yes (Go to Question 20.1)
|
From page 388... ...
THE GUIDELINE DOCUMENT PROVIDES QUANTITATIVE INFORMATION OR ESTIMATES ABOUT THE HEALTH COSTS OR EXPENDITURES THAT ARE EXPECTED AS A RESULT OF A SPECIFIC HEALTH PRACTICE. Yes (Go to Question 26.1)
|
From page 389... ...
Yes No 26e. The quantitative methodist used to estimate costs is: Now answer: Appropriate Inappropriate 26.1 THE QUANTITATIVE INFORMATION ABOUT EXPECTED HEALTH COSTS OR EXPENDITURES IS: Satisfactory (e.g., one or more estimates of costs, including accurate summary or composite measures, with ranges of uncertainty)
|
From page 390... ...
IF HEALTH BENEFITS ARE PROJECTED IN TERMS OF ADDITIONAL LIFE EXPECTANCY OR SIMILAR MEASURES, SUCH AS QUALITY-ADJUSTED LIFE YEARS, THEN THE COST PER UNIT OF EACH IDENTIFIED BENEFIT IS ESTIMATED. Yes, benefits projected in such terms and cost per unit estimated No, benefits projected in such terms but cost per unit not estimated Not applicable, benefits not so projected and cost per unit not estimated
|
From page 391... ...
THE GUIDELINE DOCUMENT EXPLICITLY DISCUSSES THE STRENGTH OF THE SCIENTIFIC EVIDENCE ON WHICH EACH MAJOR RECOMMENDATION IS BASED. Yes (Go to Question 30.1)
|
From page 392... ...
OMISSION OF A DISCUSSION OF THE STRENGTH OF THE SCIENTIFIC EVIDENCE FOR EACH MAJOR RECOMMENDATION IS: Comments: Unimportant Minor omission Major Omission 31.
|
From page 393... ...
THE GUIDELINE DOCUMENT IDENTIFIES POSSIBLE CONFLICTS AMONG EXISTING GUIDELINES AND THE REASONS FOR THEM. Yes (Go to Question 33.1)
|
From page 394... ...
Vague terms are avoided when describing the patient populations, health conditions, the health interventions, and the recommendations. For example, expressions such as "severe bleeding" are avoided in favor of (or at least qualified by)
|
From page 395... ...
IF THE GUIDELINES GIVE MAJOR RECOMMENDATIONS, EACH IS WRITTEN IN UNAMBIGUOUS TERMS. ASSESSOR: Refer to the list you developed for Question 29 in an swering this question.
|
From page 396... ...
37.1 IF EXPECTED RECOMMENDATIONS SEEM TO BE MISSING, THE GUIDELINE DOCUMENT DISCUSSES WHY. Comments: Yes No 38.
|
From page 397... ...
A USER OF THE GUIDELINE DOCUMENT CAN EASILY FIND EACH MAJOR RECOMMENDATION. ASSESSOR: Refer to the list developed for Question 29 in answering this question.
|
From page 398... ...
42.1. ASSESSOR: Respond to Items 42a-42d, below, to determine whether the scheduled review date information is satisfactory, then answer Question 42.1 below, using your best judgment as to the overall rating for this attribute of scheduled review.
|
From page 399... ...
Now answer: Yes No 42.1. THE SCHEDULED REVIEW DATE OR PROCEDURE FOR SETTING IT IS: Satisfactory Conditionally satisfactory Comments or Other Factors: Unsatisfactory (Specify)
|
From page 400... ...
First, some combination of individuals directly responsible for guidelines and those who have otherwise contributed to their development collectively represents all the key groups likely to affect or to be affected by the guidelines. Second, the guideline document describes the parties involved (including their credentials and potential biases)
|
From page 401... ...
Yes No 43c. An explanation, discussion, or rationale for selecting other individuals directly responsible for the guideline document (such as consultants)
|
From page 402... ...
THE GUIDELINE DOCUMENT EXPLICITLY NOTES ANY POTENTIAL BIASES AND/OR CONFLICTS OF INTERESTS OF THE PANEL MEMBERS, OR STATES THAT BIASES AND CONFLICTS OF INTEREST WERE DISCUSSED AMONG PANEL MEMBERS OR OTHERWISE TAKEN INTO ACCOUNT. YeS, Potential biases and/Or COnDiCtS Of interest are noted YeS, a Statement that biases and/Or Conflicts Of interest Were discussed is given NO, no note Or Statement about biases and/Or COnDiCtS Of interest .
|
From page 403... ...
OVERALL, POTENTIAL BIASES AND/OR CONFLICTS OF INTEREST APPEAR TO BE ADEQUATELY BALANCED OR OTHERWISE ACCOUNTED FOR IN THE GUIDELINE DEVELOPMENT PROCESS. Comments: Yes No (Specify)
|
From page 405... ...
Omission of discussion Y N 3. Rationale for excluding patient populations S CS US 3.1.
|
From page 406... ...
Omission of explanation STRENGTH OF THE SCIENTIFIC EVIDENCE AND PROFESSIONAL CONSENSUS Y N 12. Method of collecting (identifying and retrieving)
|
From page 407... ...
Quality of information about strength of professional consensus UN MI MA 19.2. Omission of explicit information about strength of professional consensus HEALTH BENEFITS AND HARMSIRISKS: QUALITATIVE DESCRIPTION Y N 20.
|
From page 408... ...
Quality of quantitative information 24.2. Omission of quantitative information HEALTH COSTS: QUALITATIVE DESCRIPTION Y N 25.
|
From page 409... ...
Omission of discussion Y YQ N 31. Each major recommendation consistent with strength of scientific evidence POTENTIAL CONFLICT AMONG SIMILAR SETS OF GUIDELINES Y N NA 409 Other sets of guidelines identified Y N 33.
|
From page 410... ...
Quality of the scheduled review date or procedure for setting one UN MI MA 42.2. Lack of a scheduled review date or procedure for setting one Vll.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.