THE FRAMEWORK FOR SCIENTIFIC ASSESSMENT

Causality

The Immunization Safety Review Committee has adopted the framework for assessing causality developed by its predecessors (convened by the IOM in 1991 and 1994) to address questions of immunization safety. The categories of causal conclusions used by the committee are as follows:

  1. No evidence

  2. Evidence is inadequate to accept or reject a causal relationship

  3. Evidence favors rejection of a causal relationship

  4. Evidence favors acceptance of a causal relationship

  5. Evidence establishes a causal relationship.

Assessments begin from a position of neutrality regarding the specific vaccine safety hypothesis under review. That is, there is no presumption that a specific vaccine (or vaccine component) does or does not cause the adverse event in question. The weight of the available clinical and epidemiological evidence determines whether it is possible to shift from that neutral position to a finding for causality (“the evidence favors acceptance of a causal relationship”) or away from causality (“the evidence favors rejection of a causal relationship”). The committee does not conclude that the evidence favors rejecting causality merely if the evidence is inadequate to support causality. Instead, it maintains a neutral position, concluding that the “evidence is inadequate to accept or reject a causal relationship”. For some relationships that fall into this category, data are plentiful but the results are conflicting or not strongly convincing. For other relationships that fall into this category, the data specifically addressing the causal relationship are scarce. Some authors of similar assessments use phrases such as “the evidence does not presently support a causal association.” The committee believes however that such language does not make the important distinction between evidence that a relationship does not exist (category 3) and evidence that is indeterminate with regard to causality (category 2).

Although there are no firm rules for an amount of or quality of evidence required to support a specific category of causality conclusion, standard epidemiological criteria are used to guide the decision. The strongest category is “establishes causality,” which is reserved for those relationships where the causal link is unequivocal, such as with OPV and vaccine-associated paralytic polio, or certain anaphylactic reactions to vaccine administration. The next category is “favors acceptance” of a causal relationship. This is evidence that is strong and generally convincing, although it is not firm enough to be described as unequivocal or established. “Favors rejection” is the strongest category in the negative direction. There is no “establishes no causal relationship” category since it is virtually impossible to prove the absence of a relationship with the same certainty that one can establish the presence of one. Finally, if the evidence



The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 27
Immunization Safety Review: Multiple Immunizations and Immune Dysfunction THE FRAMEWORK FOR SCIENTIFIC ASSESSMENT Causality The Immunization Safety Review Committee has adopted the framework for assessing causality developed by its predecessors (convened by the IOM in 1991 and 1994) to address questions of immunization safety. The categories of causal conclusions used by the committee are as follows: No evidence Evidence is inadequate to accept or reject a causal relationship Evidence favors rejection of a causal relationship Evidence favors acceptance of a causal relationship Evidence establishes a causal relationship. Assessments begin from a position of neutrality regarding the specific vaccine safety hypothesis under review. That is, there is no presumption that a specific vaccine (or vaccine component) does or does not cause the adverse event in question. The weight of the available clinical and epidemiological evidence determines whether it is possible to shift from that neutral position to a finding for causality (“the evidence favors acceptance of a causal relationship”) or away from causality (“the evidence favors rejection of a causal relationship”). The committee does not conclude that the evidence favors rejecting causality merely if the evidence is inadequate to support causality. Instead, it maintains a neutral position, concluding that the “evidence is inadequate to accept or reject a causal relationship”. For some relationships that fall into this category, data are plentiful but the results are conflicting or not strongly convincing. For other relationships that fall into this category, the data specifically addressing the causal relationship are scarce. Some authors of similar assessments use phrases such as “the evidence does not presently support a causal association.” The committee believes however that such language does not make the important distinction between evidence that a relationship does not exist (category 3) and evidence that is indeterminate with regard to causality (category 2). Although there are no firm rules for an amount of or quality of evidence required to support a specific category of causality conclusion, standard epidemiological criteria are used to guide the decision. The strongest category is “establishes causality,” which is reserved for those relationships where the causal link is unequivocal, such as with OPV and vaccine-associated paralytic polio, or certain anaphylactic reactions to vaccine administration. The next category is “favors acceptance” of a causal relationship. This is evidence that is strong and generally convincing, although it is not firm enough to be described as unequivocal or established. “Favors rejection” is the strongest category in the negative direction. There is no “establishes no causal relationship” category since it is virtually impossible to prove the absence of a relationship with the same certainty that one can establish the presence of one. Finally, if the evidence

OCR for page 27
Immunization Safety Review: Multiple Immunizations and Immune Dysfunction is not reasonably convincing in either the causal or non-causal direction, it is placed in the category “inadequate to accept or reject a causal relationship.” Evidence that is sparse, conflicting, of weak quality, or just suggestive falls into this category. The sources of evidence considered by the committee in its scientific assessment of causality include epidemiological and clinical studies directly addressing the question at hand. That is, the data relate to the effects of the vaccine(s) under review and the specific adverse health outcome(s) under review— in the case of this report, the effects of multiple immunizations on developing immune system function. Epidemiological studies carry the most weight in a causality assessment; these studies measure health-related exposures or outcomes in a defined sample of subjects and make inferences about the nature and strength of associations between exposures and outcomes in the overall population from which the study sample was drawn. Epidemiological studies can be categorized as observational or experimental (clinical trial), and as uncontrolled (descriptive) or controlled (analytic). Among these various study designs, experimental studies generally have the advantage of random assignment to exposures and therefore carry the most weight in assessing causality. Uncontrolled observational studies are important but are generally considered less definitive than controlled studies. In uncontrolled observational studies where observations are made over time, confounding (e.g., changing case definitions and improving case detection) may influence the incidence and prevalence of the adverse outcomes studied. Case reports and case series are generally inadequate by themselves to establish causality. Despite the limitations of case reports, the causality argument for at least one vaccine-related adverse event (the relationship between vaccines containing tetanus toxoid and Guillain-Barré syndrome) was strengthened most by a single, well-documented case report on recurrence of the adverse event following re-administration of the vaccine, a situation referred to as a “rechallenge” (IOM, 1994). Biological Mechanisms Evidence considered in the scientific assessment of biological mechanisms includes human, animal, and in vitro studies related to biological or pathophysiological processes by which immunizations could cause immune system dysfunction. This kind of review has been referred to in previous reports of this committee (IOM, 2001a, 2001b) and others (IOM, 1991, 1994) as an assessment of the “biological plausibility” of a causal relationship. The committee has previously described biological plausibility as existing on a spectrum, ranging from not plausible to established. An agreed upon hierarchy of evidence required for assessments of biological plausibility does not exist, nor does an associated terminology (Weed and Hursting, 1998).

OCR for page 27
Immunization Safety Review: Multiple Immunizations and Immune Dysfunction The committee has noted, moreover, that the term biological plausibility is a source of confusion on at least two fronts. First, it is associated with guidelines (sometimes referred to as the Bradford Hill criteria) for causal inference from epidemiological evidence (Hill, 1965). In that context, an assessment of the biological plausibility of an association demonstrated by epidemiological analysis is meant to ensure that such an association is consistent with current biological knowledge. Evidence regarding biological plausibility can never prove causality. Therefore, it is also meant to guard against attributions of causality to biologically implausible statistical associations that might result from studies that have not adequately accounted for important variables. For example, although a strong statistical relationship might exist between a woman’s risk of breast cancer and the number of bathrooms in her home, there is no mechanism based on knowledge of cancer biology that could indicate the relationship is causal. Rather, the number of bathrooms is associated with socioeconomic status, which is associated with such factors as diet that can be linked mechanistically to cancer biology. The biological implausibility of an association between the number of bathrooms in a house and the risk of breast cancer weakens the argument for a causal relationship. In other cases, a review of the biological plausibility of an association might add reassurance that the epidemiological findings point toward or reflect causality. Occasionally an epidemiological observation has been explained by a reasonable biological mechanism that, on further investigation, appeared not to be relevant for the pathophysiology. This committee, however, is often faced with a set of circumstances in which the epidemiological evidence is judged inadequate to accept or reject a causal association between a vaccine exposure and an adverse event of concern. It is then left with the task of examining proposed or conceivable biological mechanisms that might be operating if an epidemiologically sound association could be shown between vaccine exposure and an adverse event. Identification of sound mechanisms could influence the development of an appropriate research agenda and give support for policymakers, as decisions frequently must be made in situations of incomplete information regarding causality. Finally, there is often value in understanding and pursuing possible biological mechanisms even if the epidemiological evidence suggests a lack of a causal association. New epidemiological studies could question that existing causality assessment and the biological data would gain prominence in the new assessments. Also, a review of biological data could give support to the negative causality assessment or could cause one to reconsider or pursue the epidemiological findings further. Second, the committee understands that some readers of its reports are confused by what are perceived as contradictory findings. Although the committee has previously stated that biological plausibility can range across a spectrum, readers sometimes regard the term with a degree of certainty or precision the committee never intended. When other evidence of causality is available, bio-

OCR for page 27
Immunization Safety Review: Multiple Immunizations and Immune Dysfunction logical plausibility adds an additional piece of supportive evidence. However, in the absence of other evidence pointing to a causal relationship, use of the term biological plausibility, as ingrained in the language of causal inference, seems to add confusion. Thus the committee finds that for the purpose of its reports, the lack of clarity in the phrase “biological plausibility” warrants the adoption of new terminology and a new approach to its discussion of biological data. The committee will review evidence regarding “biological mechanisms” that might be consistent with the proposed relationship between a vaccine exposure and given adverse events. This biological assessment section of the report is written distinct from any argument regarding the causality of such relationships. This is not meant to imply that current understanding of biological processes does not shape or guide assessments of causality. In fact, the current thinking of a possible biological explanation for a relationship between immunization and an adverse event will influence some of the important controls used in a good epidemiological analysis. The important consideration of “confounders” in epidemiological studies comes from understanding biological phenomena that could underlie or explain the observed statistical relationship. Only when important confounders are considered can the statistical observation be considered for evidence of causality. However, absent evidence of a statistical association, or convincing clinical evidence, biological mechanisms cannot be invoked to prove causality. There are three general categories of evidence on biological mechanisms: Theoretical only: A reasonable mechanism can be hypothesized that is commensurate with scientific knowledge and that does not contradict known physical and biological principles, but it has not been demonstrated in humans or animal models. Experimental evidence: The evidence can be derived under highly contrived conditions. For example, the results require extensive manipulation of the genetics of an animal system or extreme vaccine antigen exposures in vivo or in vitro in terms of dose, route, or duration. Other experimental evidence is derived under less contrived conditions. For example, a compelling animal or in vitro model exists whereby administration of a vaccine antigen under conditions similar to human use results in a pathological process analogous to a human disease pathology. Experimental evidence often describes effects on just one or a few of the steps in the pathological process required for expression of disease. As more components of the theoretical pathways are shown to operate in reasonable experimental models, the more confident one is that the mechanisms could possibly result in disease in humans. Evidence that the mechanism results in known disease in humans: For example, a wild-type infection causes the adverse health outcome, or another vaccine has been demonstrated to cause the same adverse outcome by the same or similar mechanism. Data from population-based studies of the effects of the vaccine

OCR for page 27
Immunization Safety Review: Multiple Immunizations and Immune Dysfunction administration on the occurrence of the adverse outcomes under review contribute not to the biological mechanisms argument but to the causality argument. Beginning with this report, the committee will summarize the biological mechanisms as theoretical only, or as having derived from either experimental evidence or mechanism-related evidence in humans. If there is evidence in experimental models or humans for a mechanism, we will designate it as weak, moderate, or strong. Though the committee tends to judge evidence in humans to be “stronger” than experimental evidence from animals or in vitro systems, the strength of the evidence also depends on other factors, such as the experimental design and sample size. Obviously, the conclusions drawn from this review will depend both on evidence and scientific judgment. To ensure that its own summary judgment is defensible, the committee intends to be as explicit as possible regarding the strengths and limitations of the biological data. Published and Unpublished Data Published reports that have been subjected to a rigorous peer review process carry the most weight in the committee’s assessment. Unpublished data and other reports that have not undergone peer review do have value, and they are often considered by the committee; they could be used, for example, in support of a body of published literature with similar findings. If the committee concluded that the unpublished data were well described, had been obtained using sound methodology, and presented very clear results, the committee could report, with sufficient caveats in the discussion, how those data fit with the entire body of published literature. But only in extraordinary circumstances could an unpublished study refute a body of published literature. In general, the committee cannot rely heavily on unpublished data in making its scientific assessments (regarding either causality or biological mechanisms) because they have not been subjected to a rigorous peer review process, and therefore must be interpreted with caution. The committee acknowledges that its approach differs from the state of the art for evidence-based reviews of clinical practices in medicine, which does not include consideration of unpublished or non-peer-reviewed information or of studies with flawed experimental designs (U.S. Preventive Services Task Force, 1996). However, the Immunization Safety Review Committee was convened specifically to assess topics that are often of immediate and intense concern. In some cases, the committee’s review will take place as data are only beginning to emerge. Thus, given the unique nature of this project, the committee thought it was important to review and consider as much information as possible, including unpublished information. The committee did not perform primary or secondary analyses of unpublished data, however. In reviewing unpublished material, the committee applied generally accepted standards for assessing the quality of scientific evidence, as described above. (All unpublished data reviewed by the