National Academies Press: OpenBook

Clinical Practice Guidelines: Directions for a New Program (1990)


« Previous: Independent Review
Suggested Citation:"RELIABILITY/REPRODUCIBILITY." Institute of Medicine. 1990. Clinical Practice Guidelines: Directions for a New Program. Washington, DC: The National Academies Press. doi: 10.17226/1626.
Page 67

Below is the uncorrected machine-read text of this chapter, intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text of each book. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

ATTRIBUTES OF GOOD PRACTICE GUIDELINES 67 Pretesting Pretesting a set of guidelines on members of the intended user group (for example, practitioners or patients) using a real organization or a set of prototypical cases is desirable. (See also the discussions of reliability/reproducibility and clarity, below.) Description of methods, settings, and results of any pretests of the guidelines should be described. The Forum has been given authority to pretest guidelines, and the committee believes it should exercise that authority. RELIABILITY/REPRODUCIBILITY As conventionally used in a research context, reliability is linked to the measuring, diagnosing, or scoring of some phenomenon such as intelligence or bacterial infection.5 In the context of guidelines, the committee uses the concept to refer to the ability of some method or process to produce consistent results across time or across users, or both. In strictly technical terms, levels of reliability dictate possible (achievable) levels of validity; that is, qualitative and quantitative instruments and tools cannot be valid if they are not reliable. One kind of reliability is methodological. Ideally, if another group of qualified individuals using the same evidence, assumptions, and methods for guidelines development (for example, the same rules for literature review) were to develop guidelines, then they should produce essentially the same statements. In practice, such replications are almost unknown given the expense of the process,6 but discussion of previous trials of the methodology (for different conditions) and any resulting revisions may be useful. Likewise, review of the guidelines by an outside panel can help in assessing reliability. (Recall that independent review is also important to an assessment of validity, a fact that underscores the link between reliability and validity.) 5 The committee discussed how two common methodological concepts, sensitivity and specificity, applied to practice guidelines. For medical review and other evaluation criteria, these two related terms are fairly straightforward. Sensitivity and specificity refer, respectively, to a high "true positive rate" in detecting deficient or inappropriate care and a high "true negative rate" in passing over cases of adequate care. The concepts can be operationalized by requiring some evidence, drawn, for example, from pretesting of the review criteria on "prototype" cases or through pilot-testing in a specific organization. As described in Chapter 10 of the Medicare quality report, case-finding screens have often been found to be deficient on these two attributes. The committee concluded that these attributes need to be considered for evaluation instruments but do not add anything to the assessment of practice guidelines. 6 One effort at replication has been undertaken by those involved with the RAND Corporation's work to develop appropriateness indicators (Chassin, 1990).

Clinical Practice Guidelines: Directions for a New Program Get This Book
Buy Paperback | $50.00
MyNAP members save 10% online.
Login or Register to save!
Download Free PDF
  1. ×

    Welcome to OpenBook!

    You're looking at OpenBook,'s online reading room since 1999. Based on feedback from you, our users, we've made some improvements that make it easier than ever to read thousands of publications on our website.

    Do you want to take a quick tour of the OpenBook's features?

    No Thanks Take a Tour »
  2. ×

    Show this book's table of contents, where you can jump to any chapter by name.

    « Back Next »
  3. ×

    ...or use these buttons to go back to the previous chapter or skip to the next one.

    « Back Next »
  4. ×

    Jump up to the previous page or down to the next one. Also, you can type in a page number and press Enter to go directly to that page in the book.

    « Back Next »
  5. ×

    To search the entire text of this book, type in your search term here and press Enter.

    « Back Next »
  6. ×

    Share a link to this book page on your preferred social network or via email.

    « Back Next »
  7. ×

    View our suggested citation for this chapter.

    « Back Next »
  8. ×

    Ready to take your reading offline? Click here to buy this book in print or download it as a free PDF, if available.

    « Back Next »
Stay Connected!