Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter.
Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.
Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.
OCR for page 231
Appendix D
Systems for Rating the Strength
of Evidence and Clinical
Recommendations
231
OCR for page 231
232 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Selected Approaches to Rating Strength of Evidence
and Clinical Recommendations
Focus/ Systems for Rating
System Audience Evidence Quality
International Approaches
Grading of Focus: Grades of evidence
Recommendations Diagnosis and Randomized trial: High
Assessment, therapy Observational study: Low
Development, and Any other evidence: Very low
Evaluation (GRADE) Audience:
Working Group Decrease grade if limitations
Guideline
(2009) developers in study quality, important
inconsistency of results,
A voluntary, uncertainty about the directness
international, of the evidence, imprecise or
collaboration sparse data, and high risk of
reporting bias.
Increase grade if a very strong
association, evidence of a dose–
response gradient, presence of all
plausible residual confounding
would have reduced the observed
effect.
OCR for page 231
233
APPENDIX D
System for Rating Clinical Recommendations’ Strength
Strong: Desirable effects clearly outweigh the undesirable effects, or clearly
do not. Quality of evidence is high and other considerations support a strong
recommendation.
Weak: Trade-offs are less certain—either because of low-quality evidence or
because evidence suggests that desirable and undesirable effects are closely
balanced. The quality of evidence is high and other considerations support a
weak recommendation.
Based on:
• Quality of evidence.
• ncertainty about the balance between desirable and undesirable effects.
U
• ncertainty or variability in values or preferences.
U
• ncertainty about whether the intervention represents a wise use of
U
resources.
NOTE: Many organizations claim to use GRADE, but modify the system
in the application of translating evidence into clinical recommendations or
guidelines.
continued
OCR for page 231
234 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
Centre for Focus: CEBM is currently working on
Evidence-Based Prevention, updating its level of evidence
Medicine (CEBM) diagnosis, rankings and providing further
(2009) prognosis, rationale for them, tentatively due
therapy, to become available in January 2010.
One of several UK differential
centers with the aim of diagnosis/ This approach has different evidence
promoting evidence- symptom rating system depending on the
based health care prevalence, and type of healthcare intervention. For
economic and example, the following rating
decision system is used for therapy
analyses interventions:
Audience: Level 1a: Systematic review (SR) of
Doctors, randomized controlled trials (RCTs)
with homogeneity.a
clinicians,
teachers, and
Level 1b: Individual RCT with
others
narrow confidence interval.
Level 1c: All or none case series.b
Level 2a: SR with homogeneity of
cohort studies.
Level 2b: Individual cohort studies
(including quality RCT; e.g., <80%
follow-up).
Level 2c: Outcomes research,
ecological studies.c
Level 3a: SR with homogeneity of
case control studies.
Level 3b: Individual case control
study.
Level 4: Case series (and poor-
quality cohort and case control
studiesd).
Level 5: Expert opinion without
explicitly critical appraisal, or based
on physiology, bench research, or
“first principles.”
OCR for page 231
235
APPENDIX D
System for Rating Clinical Recommendations’ Strength
A: Consistent level 1 studies.
B: Consistent level 2 or 3 studies or extrapolationse from level 1 studies.
C: Level 4 studies or extrapolations from level 2 or 3 studies.
D: Level 5 evidence or troublingly inconsistent or inconclusive studies of
any level.
continued
OCR for page 231
236 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
Scottish Levels of evidence
Focus: All
Intercollegiate 1++ High-quality meta-analyses,
healthcare
Guidelines Network systematic reviews of RCTs, or
interventions
(SIGN) (2009) RCTs with a very low risk of bias.
Audience: 1+ Well-conducted meta-analyses,
systematic reviews, or RCTs with a
National Health
low risk of bias.
Service in
1– Meta-analyses, systematic reviews,
Scotland
or RCTs with a high risk of bias.
2++ High-quality systematic reviews
of case control or cohort studies.
___ High-quality case control or
cohort studies with a very low risk
of confounding or bias and a high
probability that the relationship is
causal.
2+ Well-conducted case control or
cohort studies with a low risk of
confounding or bias and a moderate
probability that the relationship is
causal.
2– Case control or cohort studies
with a high risk of confounding or
bias and a significant risk that the
relationship is not causal.
3 Non-analytic studies, such as
case reports, case series.
4 Expert opinion.
New Zealand Focus: The body of evidence is the sum
Guidelines Group Screening, of the evidence of all the individual
(NZGG) (2007) diagnosis, studies and the quality ratings of
prognosis, and each study.
Independent, therapy
Good evidence: From studies of
not-for-profit
Audience: strong design for answering the
Clinical question addressed.
practitioners,
Fair evidence: Reasonable
policy makers,
and consumers evidence, but there may be minimal
inconsistency, or uncertainty.
Expert opinion: For some outcomes,
trials or studies cannot be or have
not been performed and practice is
informed only by expert opinion.
OCR for page 231
237
APPENDIX D
System for Rating Clinical Recommendations’ Strength
Guidelines are developed based on judgment on the consistency, clinical
relevance, and external validity of the whole body of evidence.
A: At least one meta-analysis, systematic review, or RCT rated as 1++,
and directly applicable to the target population; or a body of evidence
consisting principally of studies rated as 1+, directly applicable to the target
population, and demonstrating overall consistency of results.
B: A body of evidence including studies rated as 2++, directly applicable to
the target population, and demonstrating overall consistency of results; or
extrapolated evidence from studies rated as 1++ or 1+.
C: A body of evidence including studies rated as 2+, directly applicable to
the target population and demonstrating overall consistency of results; or
extrapolated evidence from studies rated as 2++.
D: Evidence level 3 or 4; or extrapolated evidence from studies rated as 2+.
Good practice points: Occasionally, guideline development groups find
that there is an important practical point that they wish to emphasize, but
for which there is not, nor is there likely to be, any research evidence. This
typically will be where some aspect of treatment is regarded as such sound
clinical practice that nobody is likely to question it. These are shown in the
guideline as Good Practice Points, and are marked with a green check.
The grade of the recommendation is based on consideration of
• The design and quality of individual studies that have been identified.
• uantity, consistency, applicability, and clinical impact of the body of
Q
evidence that is applicable to the guidelines question.
• The consensus of a guideline development team.
A: The recommendation is supported by GOOD evidence.
B: The recommendation is supported by FAIR.
C: The recommendation is supported by EXPERT opinion (published) only.
I: Evidence to make a recommendation is INSUFFICIENT.
continued
OCR for page 231
238 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
The Canadian Focus: Uses flow charts to assess the
Hypertension evidence according to study
Diagnosis and
Education Program methodology:
therapy related
(2007) to hypertension
A: RCT with blinded assessment of
Audience:
A Canadian outcomes, intention-to-treat
volunteer, non-profit Canadian analysis, adequate follow-up, and
organization Diabetes sufficient sample size to detect a
Association, clinically important difference with
Canadian power >80%.
Society of
B: Adequate subgroup analysis:
Nephrology,
Canadian Analysis was a priori, performed
Coalition for within an adequate RCT and one
High Blood of only a few tested, and there
Pressure was sufficient sample size within
Prevention the examined subgroup to detect a
and Control, clinically important difference.
The College
C: Systematic review or meta-
of Family
Physicians of analysis: Comparison arms are
Canada, Heart derived from head-to-head
and Stroke comparisons within the same RCT.
Foundation of
D: Observational study or
Canada, and
Public Health systematic review in which the
Agency of comparison arms are derived from
Canada different placebo-controlled RCTs
and then extrapolations are made
across RCTs.
OCR for page 231
239
APPENDIX D
System for Rating Clinical Recommendations’ Strength
A: The recommendation is supported by a-, b-, or c-level evidence. Clinically
important outcomes and the study population is representative of the
population in the recommendation.
B: The recommendation is supported by a-, b-, or c-level evidence. Clinically
important or validated surrogate outcomes.
C: The recommendation is supported by a-, b-, c-, or d-level evidence.
For levels a, b, and c evidence, the outcome is an unvalidated surrogate
for clinically important outcomes. For level d evidence, there must be a
clinically important outcome and study population representative of the
recommendation population, or an outcome-validated surrogate, or results
that are extrapolated from study population to real population.
D: Outcome is an unvalidated surrogate for clinically important population,
or the applicability of the study is irrelevant.
continued
OCR for page 231
240 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
U.S. Approaches
Institute for Clinical Focus: Primary reports of new data
Systems Improvement collection:
Prevention,
(ICSI) (2003) A: RCT.
diagnosis, or
management
B: Cohort study.
Collaborative of 57 of a given
medical groups in symptom,
C: Nonrandomized trial with
Minnesota disease, or
condition for concurrent or historical controls, case
individual control study, study of sensitivity
patients and specificity of a diagnostic test,
under normal population-based descriptive study.
circumstances
D: Cross-sectional study, case series,
Audience: or case report.
Minnesota
healthcare
providers and
payers
OCR for page 231
241
APPENDIX D
System for Rating Clinical Recommendations’ Strength
Grade I: Good evidence
The evidence consists of results from studies of strong design for answering
the question addressed. The results are both clinically important and
consistent with minor exceptions at most. The results are free of any
significant doubts about generalizability, bias, and flaws in research design.
Studies with negative results have sufficiently large samples to have
adequate statistical power.
Grade II: Fair evidence
The evidence consists of results from studies of strong design for answering
the question addressed, but there is some uncertainty attached to the
conclusion because of inconsistencies among the results from the studies or
because of minor doubts about generalizability, bias, research design flaws,
or adequacy of sample size. Alternatively, the evidence consists solely of
results from weaker designs for the question addressed, but the results have
been confirmed in separate studies and are consistent with minor exceptions
at most.
Grade III: Limited evidence
The evidence consists of results from studies of strong design for answering
the question addressed, but there is substantial uncertainty attached to
the conclusion because of inconsistencies among the results from different
studies or because of serious doubts about generalizability, bias, research
design flaws, or adequacy of sample size. Alternatively, the evidence
consists solely of results from a limited number of studies of weak design
for answering the question addressed.
Grade not assignable: No evidence is available that directly supports or
refutes the conclusion.
continued
OCR for page 231
244 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
U.S. Preventive Focus: High: The available evidence usually
Services Task Force Prevention includes consistent results from well-
(USPSTF) designed, well-conducted studies
(2008) Audience: in representative primary care
Guideline populations. These studies assess
developers and the effects of the preventive service
users on health outcomes. This conclusion
is therefore unlikely to be strongly
affected by the results of future
studies.
Moderate: The available evidence
is sufficient to determine the effects
of the preventive service on health
outcomes, but confidence in the esti-
mate is constrained by factors such as
• he number, size, or quality of
T
individual studies.
• nconsistency of findings across
I
individual studies.
• imited generalizability of findings
L
to routine primary care practice.
• ack of coherence in the chain of
L
evidence.
As more information becomes
available, the magnitude or direction
of the observed effect could change,
and this change may be large enough
to alter the conclusion.
Low: The available evidence is
insufficient to assess effects on
health outcomes. Evidence is
insufficient because of
• he limited number or size of
T
studies.
• mportant flaws in study design
I
or methods.
• nconsistency of findings across
I
individual studies.
• aps in the chain of evidence.
G
• indings not generalizable to
F
routine primary care practice.
• ack of information on important
L
health outcomes.
More information may allow
estimation of effects on health
outcomes.
OCR for page 231
245
APPENDIX D
System for Rating Clinical Recommendations’ Strength
A: The USPSTF recommends the service. There is high certainty that the net
benefit is substantial. Offer or provide this service.
B: The USPSTF recommends the service. There is high certainty that the
net benefit is moderate or there is moderate certainty that the net benefit is
moderate to substantial. Offer or provide this service.
C: The USPSTF recommends against routinely providing the service. There
may be considerations that support providing the service in an individual
patient. There is at least moderate certainty that the net benefit is small.
Offer or provide this service only if other considerations support the
offering or providing the service in an individual patient.
D: The USPSTF recommends against the service. There is moderate or high
certainty that the service has no net benefit or that the harms outweigh the
benefits. Discourage the use of this service.
I statement: The USPSTF concludes that the current evidence is insufficient
to assess the balance of benefits and harms of the service. Evidence is
lacking, of poor quality, or conflicting, and the balance of benefits and harms
cannot be determined. Read the clinical considerations section of USPSTF
Recommendation Statement. If the service is offered, patients should
understand the uncertainty about the balance of benefits and harms.
continued
OCR for page 231
246 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
Professional Societies
American College Focus: A: Data derived from multiple
of Cardiology Prevention, randomized clinical trials or meta-
Foundation/American diagnosis, or analyses.
Heart Association management of
(ACCF/AHA) (2009) B: Data derived from a single
heart diseases or
conditions randomized trial, or non-
randomized studies.
Audience:
C: Consensus opinion of experts,
Healthcare
providers case studies, or standard of care.
American Focus: Pediatric A: Well-designed, randomized
Academy of guidelines for controlled trials or diagnostic
Pediatrics (AAP) all healthcare studies on relevant populations.
(2004) interventions
B: RCTs or diagnostics studies with
Audience: minor limitations; overwhelmingly
Guideline consistent evidence from
developers, observational studies.
implementers,
C: Observational studies (case
and users
control and cohort design).
D: Expert opinion, case reports,
reasoning from principles.
X: Exceptional situations where
validating studies cannot be
performed and there is a clear
preponderance of benefit or harm.
OCR for page 231
247
APPENDIX D
System for Rating Clinical Recommendations’ Strength
Any combination of classification of recommendation and level of evidence
is possible. A recommendation can be Class I, based entirely on expert
opinion (level C), or Class IIB, with level A evidence if based on multiple
RCTs with divergent conclusions.
Class I: Conditions for which there is evidence and/or general agreement
that a given procedure or treatment is useful and effective. Class 1
statements may read: should, is recommended, is indicated, or is useful/
effective/beneficial.
Class II: Conditions for which there is conflicting evidence and/or a
divergence of opinion about the usefulness/efficacy of a procedure or
treatment.
Class IIa: Weight of evidence/opinion is in favor of usefulness/efficacy.
Class IIa statements may read: is reasonable, can be useful/effective/
beneficial, is probably recommended, is probably indicated.
Class IIb: Usefulness/efficacy is less well established by evidence/
opinion. Class IIb statements may read: may/might be considered, may/
might be reasonable, usefulness/effectiveness is unknown/unclear/
uncertain/not well established.
Class III: Conditions for which there is evidence and/or general agreement
that the procedure/treatment is not useful/effective and in some cases
may be harmful. Class III statements may read: is not recommended, is not
indicated, should not, is not useful/effective/beneficial, may be harmful.
Strong recommendation: The benefits of the recommended approach clearly
exceed the harms (or in the case of a negative recommendation, the harms
clearly exceed the benefits) and the quality of the evidence is either excellent
or impossible to obtain (A, sometimes B, or X).
Recommendation: The benefits exceed the harms or vice versa, but the
quality of evidence is not as strong (sometimes B, C, or X).
Option: The evidence quality that exists is suspect or not that well-designed;
well-conducted studies have demonstrated little clear advantage of one
approach versus another (A, B, C, or D).
No recommendation: There is both lack of pertinent evidence and an
unclear balance between benefits and harms (D).
continued
OCR for page 231
248 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
American Academy Focus: Screening, Similar ratings systems exist for
of Neurology (AAN) diagnosis, diagnostic, prognostic, and screening
(2004) prognosis, and interventions. Therapeutic
therapy of interventions is one example:
neurologic
Class I: Prospective, RCT with
disorders
masked outcome assessment, in
Audiences: a representative population. The
Neurologists, following are required: (a) primary
patients, payers, outcome(s) clearly defined, (b)
federal agencies, exclusion/inclusion criteria clearly
other healthcare defined, (c) adequate accounting
providers, and for dropouts and crossovers with
clinical numbers sufficiently low to have
researchers minimal potential for bias, (d) relevant
baseline characteristics are presented
and substantially equivalent among
treatment groups or there is
appropriate statistical adjustment for
differences.
Class II: Prospective matched group
cohort study in a representative
population with masked outcome
assessment that meets a through d
above or an RCT in a representative
population that lacks one criteria in a
through d.
Class III: All other controlled trials
(including well-defined natural
history controls or patients serving
as own controls) in a representative
population, where outcome
is independently assessed, or
independently derived by objective
outcome measurement.
Class IV: Evidence from uncontrolled
studies, case series, case reports, or
expert opinion.
OCR for page 231
249
APPENDIX D
System for Rating Clinical Recommendations’ Strength
A: Established as effective, ineffective, or harmful (or established as useful/
predictive or not useful/predictive) for the given condition in the specified
population.
Recommendation: Should be done or should not be done.
Translation of evidence to recommendation: Requires at least two consistent Class
I studies.
B: Probably effective, ineffective, or harmful (or probably useful/predictive or
not useful/predictive) for the given condition in the specified population.
Recommendation: Should be considered or should not be considered.
Translation of evidence to recommendation: Requires at least one Class I study or
two consistent Class II studies.
C: Possibly effective, ineffective, or harmful (or possibly useful/predictive or
not useful/predictive) for the given condition in the specified population.
Recommendation: May be considered or may not be considered.
Translation of evidence to recommendation: Level C rating requires at least one
Class II study or two consistent Class III studies.
B: Data inadequate or conflicting. Given current knowledge, treatment (test,
predictor) is unproven.
Recommendation: None.
Translation of evidence to recommendation: Studies not meeting criteria for Class
I–Class III.
continued
OCR for page 231
250 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
American College of Focus: High: RCTs without important
Chest Diagnosis and limitations or overwhelming evidence
Physicians (ACCP) management of from observational studies.
(2009) chest disease
Moderate: RCTs with important
Audience: Chest limitations (inconsistent results,
physicians methodologic flaws, indirect, or
imprecise) or exceptionally strong
evidence from observational studies.
Low: Observational studies or case
series.
National Focus: High: High-powered randomized
Comprehensive Prevention, clinical trials or meta-analysis.
Cancer Network diagnosis, and
(NCCN) (2008) Lower: Runs the gamut from phase II
therapy related
to cancer to large cohort studies to case series to
individual practitioner experience.
Audience:
Oncologists and
other healthcare
providers
OCR for page 231
251
APPENDIX D
System for Rating Clinical Recommendations’ Strength
1A: Strong recommendation. High level of evidence. Benefits outweigh the
risks/burdens, or the risks/burdens outweigh the benefits.
1B: Strong recommendation. Moderate evidence. Benefits outweigh the
risks/burdens, or the risks/burdens outweigh the benefits.
1C: Strong recommendation. Low or very low evidence. Benefits outweigh
the risks/burdens, or the risks/burdens outweigh the benefits.
2A: Weak recommendation. High evidence, and the risks/burdens are evenly
balanced with the benefits.
2B: Weak recommendation. Moderate evidence, and the risks/burdens are
evenly balanced with the benefits.
2C: Weak recommendation. Low or very low evidence, and the risks/
burdens are evenly balanced with the benefits. Or the balance of benefits to
risks and burdens is uncertain.
Category 1: The recommendation is based on high-level evidence (e.g.,
randomized controlled trials), and there is uniform NCCN consensus.
Category 2A: The recommendation is based on lower level evidence and
there is uniform NCCN consensus.
Category 2B: The recommendation is based on lower level evidence and
there is non-uniform NCCN consensus (but no major disagreement).
Category 3: The recommendation is based on any level of evidence, but
reflects major disagreement.
continued
OCR for page 231
252 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
TABLE D-1 Continued
Focus/ Systems for Rating
System Audience Evidence Quality
Infectious Diseases Focus: I: Evidence from >1 properly
Society of America Healthcare randomized, controlled trial.
(2001) interventions
II: Evidence from >1 well-designed
for infectious
diseases clinical trial, without randomization;
Audience: from cohort or case-controlled
Infectious analytic studies (preferably from >1
disease center); from multiple time-series; or
clinicians from dramatic results
from uncontrolled experiments.
III: Evidence from opinions of
respected authorities, based on clinical
experience, descriptive studies, or
reports of expert committees.
a Homogeneity refers to an SR that is free of worrisome variations (heterogeneity) in
the directions and degrees of results between individual studies.
b Met when all patients died before the Rx became available, but some now survive
on it, or when some patients died before the Rx became available, but none now die
on it.
cA member of CEBM stated that this ranking requires further analysis, as well as more
detailed explanation of what is meant by ecological and outcomes research.
d Poor-quality prognostic cohort study refers to one in which sampling is biased in fa-
vor of patients who already had the target outcome, or the measurement of outcomes
is accomplished in < 80 percent of study patients, or outcomes were determined in an
unblinded, non-objective way, or there is no correction for confounding errors.
e Extrapolations are where data are used in a situation that has potentially clinically
important differences than the original study situation.
OCR for page 231
253
APPENDIX D
System for Rating Clinical Recommendations’ Strength
A: Good evidence to support a recommendation for use.
B: Moderate evidence to support a recommendation for use.
C: Poor evidence to support a recommendation for use.
f Clinicaldecision rules (CDRs) are tools designed to help clinicians make bedside di -
agnostic and therapeutic decisions. The development of a CDR involves three stages:
derivation, validation, and implementation.
g Patient-oriented evidence measures outcomes that matter to patients: morbidity,
mortality, symptom improvement, cost reduction, and quality of life. Disease-oriented
evidence measures intermediate, physiologic, or surrogate end points that may or
may not reflect improvements in patient outcomes (e.g., blood pressure, blood chem -
istry, physiologic function, pathologic findings).
SOURCES: AAN (2004); ACCF/AHA (2009); ACCP (2009); CEBM (2009); Ebell et al.
(2004); GRADE Working Group (2009); ICSI (2003); Kish (2001); NCCN (2008); NZGG
(2007); SIGN (2009); Steering Committee on Quality Improvement Management
(2004); Tobe et al. (2007); USPSTF (2008).
OCR for page 231
254 CLINICAL PRACTICE GUIDELINES WE CAN TRUST
REFERENCES
AAN (American Academy of Neurology). 2004. Clinical practice guidelines process
manual. http://www.aan.com/globals/axon/assets/3749.pdf (accessed July 28,
2009).
ACCF/AHA (American College of Cardiology Foundation/American Heart Associa-
tion). 2009. Methodology manual for ACCF/AHA guideline writing committees.
http://www.americanheart.org/downloadable/heart/12378388766452009Met
hodologyManualACCF_AHAGuidelineWritingCommittees.pdf (accessed July
29, 2009).
ACCP (American College of Chest Physicians). 2009. The ACCP grading system for
guideline recommendations. http://www.chestnet.org/education/hsp/grading
System.php (accessed July 28, 2009).
CEBM (Centre for Evidence-Based Medicine). 2009. Oxford Centre for Evidence-
based Medicine—Levels of Evidence (March 2009). http://www.cebm.net/
index.aspx?o=1025 (accessed July 28, 2009).
Ebell, M. H., J. Siwek, B. D. Weiss, S. H. Woolf, J. Susman, B. Ewigman, and M. Bow -
man. 2004. Strength of recommendation taxonomy (SORT): A patient-centered
approach to grading evidence in medical literature. American Family Physician
69(3):548–556.
GRADE Working Group (Grading of Recommendations Assessment, Development,
and Evaluation Working Group). 2009. Grading the quality of evidence and the
strength of recommendations http://www.gradeworkinggroup.org/intro.htm
(accessed July 20, 2009).
ICSI (Institute for Clinical Systems Improvement). 2003. Evidence grading system.
http://www.icsi.org/evidence_grading_system_6/evidence_grading_system__
pdf_.html (accessed September 8, 2009).
Kish, M. A. 2001. Guide to development of practice guidelines. Clinical Infectious
Diseases 32(6):851–854.
NCCN (National Comprehensive Cancer Network). 2008. About the NCCN clinical
practice guidelines in oncology. http://www.nccn.org/professionals/physician_
gls/about.asp (accessed September 8, 2009).
NZGG (New Zealand Guidelines Group). 2007. Handbook for the preparation of
explicit evidence-based clinical practice guidelines. http://www.nzgg.org.nz/
download/files/nzgg_guideline_handbook.pdf (accessed September 4, 2009).
SIGN (Scottish Intercollegiate Guidelines Network). 2009. SIGN 50: A guideline de -
veloper’s handbook http://www.sign.ac.uk/guidelines/fulltext/50/index.html
(accessed July 20, 2009).
Steering Committee on Quality Improvement Management. 2004. Classifying recom -
mendations for clinical practice guidelines. Pediatrics 114(3):874–877.
Tobe, S. W., R. M. Touyz, and N. R. C. Campbell. 2007. The Canadian Hypertension
Education Program—a unique Canadian knowledge translation program. Cana-
dian Journal of Cardiology 23(7):551–555.
USPSTF (U.S. Preventive Services Task Force). 2008. Grade definitions. http://www.
ahrq.gov/clinic/uspstf/grades.htm (accessed July 28, 2009).