In addition to determining the primary and secondary literature that would be used to draw conclusions, the committee considered other characteristics of the studies related to the methods used by researchers in their design and conduct.
Bias refers to systematic, or nonrandom, error. Bias causes an observed value to deviate from the true value, and could weaken an association or generate a spurious association. Because all studies are susceptible to bias, a goal of the research design is to minimize bias or to adjust the observed value of an association by correcting for bias. There are different types of bias, such as selection bias and information bias.
Selection bias occurs when systematic error in obtaining study participants results in a potential distortion of the true association between exposure and outcome. Information bias results from the manner in which data are collected and can result from measurement errors, imprecise measurement, and misdiagnosis. Those types of errors might be uniform in an entire study population or might affect some parts of the population more than others. Information bias might result from misclassification of study subjects with respect to the outcome variable or from misclassification of exposure. Other common sources of information bias are the inability of study subjects to recall the circumstances of their exposure accurately (recall bias) and the likelihood that one group more frequently reports what it remembers than another group (reporting bias). Information bias is especially harmful in interpreting study results when it affects one comparison group more than another.
Confounding occurs when a variable or characteristic otherwise known to be predictive of an effect and associated with the exposure (and not on the causal pathway) can account for part or all of an apparent association. A confounding variable is an uncontrolled variable that influences the outcome of a study to an unknown extent, and makes precise evaluation of its effects impossible. Examples of confounders are age, sex, smoking, and pre-existing illness. Carefully applied statistical adjustments can often control for or reduce the influence of a confounder.
A false positive (type-one error) occurs when routine statistical variation leads to an apparent association between an exposure to a stressor and a health effect when no association is present. This happens when the observed result of a study falls in the tail of the probability distribution hypothesized to describe the process being studied. Standard statistical methods, such as p-values and confidence intervals, allow one to assess the likelihood that random error due to sampling is responsible for positive findings. Replication of a positive finding in additional studies demonstrates that it is not simply a false positive, but does not guard against the same biases and confounders distorting the results if the studies’ designs are the same. Consistent results in multiple studies with different designs, and hence vulnerabilities to different confounders and sources of bias, increases confidence that the observed relationship is real and