Guiding Principles for Developing Dietary Reference Intakes Based on Chronic Disease (2017) / Chapter Skim
Currently Skimming:
6 Evidence Review: Judging the Evidence for Causal Relationships
6 Evidence Review: Judging the Evidence for Causal Relationships
Pages 129-194
The Chapter Skim interface presents what we've algorithmically identified as the most significant single chunk of text within every page in the chapter.
Select key terms on the right to highlight them within pages of the chapter.
From page 129... ...
Although the current DRI process considers all three indicators, this committee has been asked to provide recommendations and guiding principles for developing chronic disease DRIs, specifically. As mentioned in Chapter 1, a key activity relevant to chronic disease endpoints within the purview of DRI committees is evaluating the certainty of the evidence with regard to two main questions: (1)
|
From page 130... ...
. Systematic reviews of studies may also be particularly affected by reporting bias, where a biased subset of all the relevant data is available.
|
From page 131... ...
In the DRI process, a DRI committee is equivalent to the guideline panel in the Clinical Practice Guideline process. Heterogeneity: The variation in study outcomes within the body of evidence for a particular outcome.
|
From page 132... ...
Obser vational studies have a greater risk of selection bias and ascertainment bias than do experimental studies. Cross-sectional studies, cohort studies, and case-control studies are types of observational studies.
|
From page 133... ...
It includes assessing the systematic review evidence profiles, quantitatively characterizing the intake-response, considering relation ships with various chronic diseases and potential overlapping benefits and harms, determining the need for, and appropriateness of, extrapolation to other popula tions, and reviewing other relevant evidence. Synthesis of evidence: Evaluating the body of evidence collected in a systematic manner and using quantitative and qualitative synthesis strategies.
|
From page 134... ...
It is further assumed that the initial and any subsequent systematic reviews and protocols are conducted independently of the DRI committee. The first use of this type of systematic review and evidence review process for DRI purposes was reflected in the development of the 2011 update of the DRIs for calcium and vitamin D, which included consideration of chronic disease endpoints (Brannon et al., 2014; IOM, 2011a)
|
From page 135... ...
Scoping may also identify systematic reviews, including meta-analyses already published on nutrition-disease topics. These previously published systematic reviews can help target relevant literature and also can be incorporated as part of the evidence review.
|
From page 136... ...
136 PRINCIPLES FOR DEVELOPING DRIs BASED ON CHRONIC DISEASE Initiating Finding and Assessing 1 a Systematic Review 2 Individual Studies Establish a team with appropriate Conduct a comprehensive 1.1 expertise and experience to 2.1 systematic search conduct the systematic review for evidence Manage bias and conflict of Take action to address 1.2 interest of the team conducting 2.2 potentially biased reporting the systematic review of research results Ensure user and stakeholder input 1.3 as the review is designed and 2.3 Screen and select studies conducted Manage bias and conflict of 1.4 interest for individuals providing 2.4 Document the search input into the systematic review Formulate the topic 1.5 for the systematic review 2.5 Manage data collection Develop a systematic 1.6 review protocol 2.6 Critically appraise each study Submit the protocol for 1.7 peer review Make the final protocol publicly 1.8 available, and add amendments to the protocol in a timely fashion FIGURE 6-1 Standards for systematic reviews.
|
From page 137... ...
JUDGING THE EVIDENCE FOR CAUSAL RELATIONSHIPS 137 Synthesizing the Body Reporting Systematic 3 of Evidence 4 Reviews Use a pre-specified method to Prepare the final report 3.1 evaluate the body of evidence 4.1 using a structured format 3.2 Conduct a qualitative synthesis 4.2 Peer review the draft report Decide if, in addition to a Publish the final report in a qualitative analysis, the systematic 3.3 review will include a quantitative 4.3 manner that ensures free public access analysis (meta-analysis) If conducting a meta-analysis, 3.4 conduct it according to specified required elements FIGURE 6-1 Continued
|
From page 138... ...
in order to inform the eventual guideline. The questions in Box 6-2, also drawn from systematic reviews to support WHO guideline development, illustrate more complex NOFS-chronic disease questions using the general PICO approach (Brouwer, 2016; Mensink, 2016)
|
From page 139... ...
hat is the effect in the population of reduced percentage of total energy W intake from trans fatty acids (TFAs) relative to higher intake for reduction in risk of noncommunicable diseases (NCDs)
|
From page 140... ...
of saturated or trans fat and substitution or replacement effects that are relevant to food manufacturing. Systematic Review Registration and Report Format Once the systematic review team is assembled and as the systematic review process begins, the committee strongly suggests registering the review in advance with the international prospective register of systematic reviews (PROSPERO)
|
From page 141... ...
Forming the Systematic Review Team, Technical Expert Panel, and DRI Committee Developing chronic disease DRIs depends on the contributions of many stakeholders, including the federal agencies, groups of experts, and the public. Within the current process, three groups of experts -- the systematic review team, the technical expert panel, and the DRI committee -- are involved in different capacities and time frames.
|
From page 142... ...
• echanistic study findings can help direct research agendas related to M human observational and intervention studies to gather the necessary data to move population policies forward. experienced with the evidence review approach, biostatisticians, and persons with experience in guideline construction and policy applications.
|
From page 143... ...
SYSTEMS OF EVIDENCE SYNTHESIS AND THEIR APPLICATIONS The committee concluded that addressing diet-chronic disease relationships requires a credible, established, structured system for systematic review and review of the totality of the evidence, for all important topics that may be translated into population policy, and that this applies where "diet" refers to specific NOFSs, as is the focus of this DRI-oriented report, as well as when applied to foods or entire dietary patterns. The guiding principles discussed throughout the chapter reflect the fact that systematic reviews and evidence reviews in general are well-developed scientific activities for informing policy and practice.
|
From page 144... ...
THE "GRADE" (GRADING OF RECOMMENDATIONS, ASSESSMENT, DEVELOPMENT AND EVALUATION) SYSTEM: DEFINITION AND JUSTIFICATION After reviewing the systems in Annex 6-2 and based on the knowledge and experience of committee members about what would meet the needs of DRI committees and users of DRI reports, the committee adopted the GRADE system as the reference point for the evidence reviews relating to NOFS-chronic disease considerations.
|
From page 145... ...
Meets Criteria for an Appropriate Evidence Review Tool The GRADE system meets the requirements for an established system of guidance for evidence synthesis. It has been used extensively in many scientific domains, and comprehensive documentation and precedent exist to address many of the issues relevant to NOFS-chronic disease questions.
|
From page 146... ...
is applied to evidence review in general, with particular suggestions and examples for application to the development of NOFS-chronic disease DRIs. In particular, this section addresses four critical and challenging issues in conducting an evidence review and determining the certainty of the evidence using the GRADE system: (1)
|
From page 147... ...
For example, as Figure 6-2 illustrates and as described earlier in this chapter, in the GRADE approach, a systematic review of the relevant scientific literature by the systematic review team precedes the work of the DRI committees to evaluate the totality of the evidence. The following discussion assumes that the systematic review and subsequent evidence reviews by the DRI committee are largely independent and that these processes precede and are separate from the development of guidelines, as described in Chapter 1 and above.7 Selecting and Ranking Health and Disease Outcomes GRADE emphasizes the importance of identifying all outcomes that are meaningful to the population to whom a recommendation or guideline will be applied.
|
From page 148... ...
Introduction -- GRADE evidence profiles and summary of findings tables. J Clin Epidemiol 64(4)
|
From page 149... ...
. For example, one could ask in one question "what is the effect of replacing saturated fat with monounsaturated fatty acids or various types of polyunsaturated fatty acids on cardiovascular disease outcomes" but extract data to support examination of each type of replacement separately.
|
From page 150... ...
As mentioned above, sufficient interaction between the DRI committee and the systematic review team and technical expert panel will ensure a more efficient process because any differences in judgments would have been discussed before the DRI committee receiving the final systematic review. For binary or other categorical outcomes, evidence profiles include, at a minimum, a list of all outcomes, the numbers of participants and studies addressing these outcomes, the fundamental study designs (RCTs or observational studies)
|
From page 151... ...
However, as discussed in Chapter 3, many NOFS-chronic disease questions will not have been addressed through RCTs for reasons of feasibility or cost. Also, the trials that do exist might have limited value for several reasons related to the form or dose of the
|
From page 152... ...
Characteristics of included prospective cohort studies of saturated fatty acids and health outcomes. (FA=fatty acids; DM=diabetes mellitus; CVD=cardiovascular disease; BMI=body mass index; MI=myocardial infarction; CHD=coronary heart disease; BP=blood pressure; ECG=electrocardiogram)
|
From page 153... ...
Observational studies also provide information that is used in assessing intake-response relationship and in risk management. However, as noted in Chapter 3, observational studies that have a retrospective casecontrol or cross-sectional design may be excluded from eligibility for inclusion in systematic reviews because of their lower suitability for supporting judgments about causal relationships, in comparison to prospective studies.
|
From page 154... ...
One of the factors considered in assessing the certainty of a body of evidence is the risk of bias of the individual studies. Systematic reviews are expected to critically appraise individual studies and evaluate the totality of evidence for each outcome in terms of the risk of bias (i.e., not serious, serious, or very serious)
|
From page 156... ...
As discussed below, the field of nutrition has no well-accepted risk-of-bias tools. There are no tools that prompt for evaluation of certain methodological features that are encountered with respect to assessing causal associations of NOFSs or other aspects of diet with chronic diseases (Chung, 2017)
|
From page 157... ...
.12 This tool is based on Cochrane risk-of-bias domains (Selection Bias, Performance Bias, Detection Bias, Attrition Bias) and it is tailored by study design, with different sets of questions applying to RCTs (14 questions)
|
From page 158... ...
. There were no cohort studies in adults; however, data from 5 out of 5 studies of other study design in adults included in the systematic review detected a statistically significant positive relationshzip between dietary free sugars and levels of dental caries.
|
From page 159... ...
If it would, then one rates down the evidence for imprecision. For instance, in Figure 6-4, evidence from cohort studies of total trans fatty acid on ischemic stroke was rated down for imprecision because of 95 percent confidence intervals that suggested benefit at the lower bound and exceeded the threshold for harm at the upper bound (de Souza et al., 2015, appendix 6, footnote 27)
|
From page 160... ...
This is particularly the case when the body of evidence is from observational studies because assessors might not know if the published observational studies are representative of the studies conducted due to the lack of published or registered protocols. With this in mind, the terms GRADE suggests using are "undetected" and "strongly suspected" publication bias.
|
From page 161... ...
Although large effects are rarely seen with any nutrition intervention unless individuals are deficient in a particular nutrient, the committee concluded that uprating the confidence in the body of evidence of observational studies requires a large effect in the health outcome. This requirement provides higher assurance that the association could not be due to residual confounding.
|
From page 162... ...
If such a relationship exists, then the DRI committee will characterize the certainty of the intake-response relationships, consider benefits and harms and their relative importance, and recommend chronic disease DRIs levels or ranges when appropriate, as explained in Chapter 7. Determining causation between exposures and outcomes has been a longstanding philosophical as well as practical challenge in population
|
From page 163... ...
This level of confidence likely requires at least some evidence from high-quality RCTs in which the measured outcome is a chronic disease event or qualified surrogate disease marker. Option 2: Use level B evidence • his option also includes level B evidence as a basis for DRI decisions T about causation.
|
From page 164... ...
Lesser levels of confidence in causation should not lead to actionable recommendations or, in the case here, the development of chronic disease DRIs. Presumably, to move forward with developing chronic disease DRIs, determination of an intake-response in data from observational studies as part of the evidence rating also would be needed.
|
From page 165... ...
3. consultation with the technical expert panel, systematic reviews In should be sufficiently inclusive of all study designs that potentially con tribute to evaluation of the causal NOFS-chronic disease relationship of interest and identification of associated intake-response relationships.
|
From page 166... ...
and specific questions and will generally include nutrition science, scientific study design and analysis, public health, biostatistics, nutrition and chronic disease epidemiology, disease patho genesis, and evidence review conduct.
|
From page 167... ...
Presented at the Workshop of the Committee on Development of Guiding Principles for the Inclusion of Chronic Disease Endpoints in Future Dietary Reference Intakes, January 9, 2017, Washington, DC. Chung, M., S
|
From page 168... ...
2015. Intake of saturated and trans unsaturated fatty acids and risk of all cause mortality, cardiovascular disease, and type 2 diabetes: Systematic review and meta-analysis of observational studies.
|
From page 169... ...
2015. Nutrition Evidence Library -- 2015 Dietary Guide lines Advisory Committee systematic reviews.
|
From page 170... ...
2009. AMSTAR is a reliable and valid measurement tool to assess the methodological quality of systematic reviews.
|
From page 171... ...
2016. ROBIS: A new tool to assess risk of bias in systematic reviews was developed.
|
From page 172... ...
What is the effect of restricting intake of free sugars to below 10 percent of total energy intake? 15 Adults Population Apparently healthy adults in low-, middle-, and high-income countries • n each, consider population characteristics, I such as age, gender, ethnicity, country/ region (urban/rural)
|
From page 173... ...
, and impact Consider whether artificial sweeteners/milk/ other foods are used as control Intermediates • ake into account effect of energy density T Outcome • ody weight or fatness gain measured by: B - weight change, BMI - incidence of obesity and overweight - body fatness16 and distribution assessed in a variety of ways • ental caries (not erosion) D Time frame • or controlled feeding studies where a high F proportion of food is directly provided and there is no caloric restriction, outcomes are change in weight or body fatness within a minimum study duration of 8 weeks • or studies where the intervention is F advisory or shopping type, without caloric restriction, outcomes are obesity incidence, change in weight or body fatness with a minimum study duration of 6 months (26 weeks)
|
From page 174... ...
• ercent of total energy intake from sugars P • onsumption of sugar-sweetened beverages C • ruit juices F Control Comparison of levels Continuous or categorical Adherence to recommendations Confounders/effect • aseline level of all categories of sugars B modifiers/intermediates intake • nergy intake E • nergy expenditure, fitness, and physical E activity • onsider other interventions in design, C dietary and non-dietary (protocol to specify) • onsider influence of other aspects of diet/ C dietary patterns In cohort studies: unadjusted and adjusted estimates; what adjusted for, how (protocol to specify)
|
From page 175... ...
2003. Diet, nutrition and the prevention of chronic diseases: Report of a joint WHO/FAO expert consultation.
|
From page 177... ...
Purpose/Who Is Using It Risk of Bias Domains Grading Method Notes Nutrition Evidence To assess the risk of bias • Selection bias The NEL BAT is tailored Adapted from the Library Bias Assessment of each individual study • Performance bias by study design, with Cochrane Bias Methods Tool (NEL BAT) a (any design)
|
From page 178... ...
This is achieved by assigning a judgement of Used in Cochrane Low risk of bias, High systematic reviews risk of bias, or Unclear risk of bias.
|
From page 179... ...
in meta-analyses groups Each domain has one Supplemented with and systematic reviews A 3. scertainment of to four questions to nutrition-specific items either the exposure or answer.
|
From page 180... ...
Office of of individual studies • Confounding bias three questions, depending using established Health Assessment and (human and non- • Performance bias on study type. guidelines for Translation (OHAT)
|
From page 181... ...
. e See https://ntp.niehs.nih.gov/pubhealth/hat/noms/index-2.html (accessed July 24, 2017)
|
From page 182... ...
The true effect may be substantially GRADE approaches" Used to evaluate R • eporting bias different from the estimate of the effect that differ from the the evidence in Very low quality: We have very little confidence in approach described systematic reviews Studies may be upgraded based on: the effect estimate. The true effect is likely to be by the GRADE M • agnitude of treatment effect substantially different from the estimate of effect Working Group.
|
From page 183... ...
Low (4-5.99 total points) : There is low confidence but has modified other tools E • ffect size (only for meta-analyses of in the effect estimate; further research will provide the classification cohort studies)
|
From page 184... ...
not a grading for "strength of the recommendations" because AHRQ systematic reviews do not make clinical recommendations. The clinical recommendation process is a totally independent process thus not covered in ARHQ methods guide.
|
From page 185... ...
• K ey question • N umber of studies C: The USPSTF recommends selectively offering or • S ummary of findings providing this service to individual patients based • C onsistency/precision on professional judgment and patient preferences. • R eporting bias There is at least moderate certainty that the net • O verall study quality benefit is small.
|
From page 186... ...
Grading Method Notes National Toxicology Framework for Seven-step process Level of confidence in the body of evidence: Confidence rating is Program (NTP) systematic review 1.
|
From page 187... ...
. • I nadequate evidence: There is insufficient evidence available to assess whether the exposure to the substance is associated with the health outcome(s)
|
From page 188... ...
Grading Method Notes World Cancer Methodology used • S tudy type Criteria for grading evidence: Judgments were Research Fund/ by the WCRF/ • H eterogeneity within or between study Convincing = Evidence is strong enough to support based on an American Institute AICR to ascertain types or in different populations a judgment of a convincing causal relationship, assessment of the for Cancer Research causal relationships • R andom or systematic error (including which justifies goals and recommendations designed evidence available (WCRF/AICR) g between food, confounding, measurement error, and to reduce the incidence of cancer.
|
From page 189... ...
* Any grading tool for strength of the body of evidence can be applied to grade the body of evidence of any study design that is defined by the inclusion criteria of the systematic reviews.
|
From page 190... ...
Using It Checklist Grading Method Notes A Measurement Tool To develop and AMSTAR Checklist (Answer yes/ A well-done systematic review AMSTAR can be to Assess Systematic evaluate the no/can't answer/not applicable) : is one that has addressed all used for systematic Reviews (AMSTAR)
|
From page 191... ...
Revised Assessment of To quantify the Added quantification to the For each of the 11 questions Not developed by the Multiple Systematic quality of systematic AMSTAR checklist in the AMSTAR checklist, a AMSTAR group Reviews reviews set of criteria is given and a (R-AMSTAR) b number value is assigned based on how many criteria are met.
|
From page 192... ...
AMSTAR is primarily a rating of reporting quality and cannot assess the other potential biases in a systematic review. AMSTAR is generally used as a quality assessment tool for complex systematic reviews that want to include existing systematic reviews to determine whether a systematic review has sufficient quality to be included.
|
From page 193... ...
2016. ROBIS: A new tool to assess risk of bias in systematic reviews was developed.
|
Key Terms
This material may be derived from roughly machine-read images, and so is provided only to facilitate research.
More
information on Chapter Skim is available.