The current immunization schedule recommended by the Advisory Committee on Immunization Practices (ACIP) was developed after consideration of the safety and effectiveness of the component vaccines and the burden of the infectious diseases on the population targeted by each vaccine. The Food and Drug Administration’s (FDA’s) current protocol for approval of new vaccines requires an evaluation of the effect of administration of a new vaccine along with other vaccines within the preexisting schedule. Therefore, the burden of disease and evidence of adequate immunogenicity when vaccines are administered together with existing recommended vaccines are established at the time of FDA approval and development of a recommendation by the ACIP. Although the committee’s review of the available scientific evidence revealed that no potential adverse health outcomes that may occur after immunization with the recommended immunization schedule rose to a level of concern or biological plausibility sufficient to justify a strong recommendation for immediate study, the committee was asked to recommend methodological approaches that could be implemented should the need arise.
To fulfill its appointed charge, the committee deliberated on five distinct topics to meet the requirements of its statement of task: (1) factors that should be used to determine that new research is needed; (2) major stakeholder concerns that the committee identified; (3) epidemiological evidence on the health effects of the current schedule; (4) major stakeholder concerns and available epidemiological evidence recast into testable research questions; and (5) possible research approaches to address priority research questions.
As discussed in Chapter 5, the committee noted that limited published data do not provide evidence that the recommended immunization schedule is associated with safety or health risks. Indeed, the available epidemiological data repeatedly indicate the health benefits associated with the recommended schedule (e.g., reduced infections and hospitalizations).
To undertake new studies on the immunization schedule beyond analyses with existing data from surveillance systems, researchers will need to carefully consider the current evidence, both epidemiological and biological, that supports the plausibility of their hypotheses. The decision to initiate further studies should depend on the results of an evaluation of three considerations that the committee identified through its review of stakeholder concerns and scientific findings:
- epidemiological evidence of potential adverse health outcomes associated with elements of the immunization schedule (such as postmarketing signals or indications of an elevated risk from observational or experimental studies);
- biological plausibility supporting hypotheses linking specific aspects of the immunization schedule with particular adverse health outcomes; and
- expressed concerns from some stakeholders about the immunization schedule’s safety, which should support efforts to evaluate the previous two considerations.
Currently, the U.S. Department of Health and Human Services (HHS) considers these criteria before initiating new studies through the Vaccine Safety Datalink (VSD). As discussed in Chapter 3, the Vaccine Adverse Event Reporting System (VAERS) allows parents and providers to report suspected adverse events after immunization. If an association is suspected on the basis of these signals, medical experts in the Clinical Immunization Safety Assessment (CISA) Network evaluate the pathophysiological basis of the suspected event. Researchers may also conduct, using VSD, population-based epidemiological studies on the basis of signals reported through VAERS and conclusions about biological plausibility reported by the CISA Network.
The committee concluded that stakeholder concerns have a role in guiding the research priorities of the Centers for Disease Control and Prevention (CDC), FDA, the National Institutes of Health, and the National Vaccine Program Office because they may point to potential research questions that need to be validated from their epidemiological signals and the plausibility
of the suggested biological pathways. Given the safeguards already in place, stakeholder concerns alone are not sufficient reason to embark on costly clinical research, such as new randomized controlled trials (RCTs) or prospective cohort studies, without the existence of supporting signals or evidence of biological plausibility.
Recommendation 6-1: The committee recommends that the Department of Health and Human Services incorporate study of the safety of the overall childhood immunization schedule into its processes for setting priorities for research, recognizing stakeholder concerns, and establishing the priorities on the basis of epidemiological evidence, biological plausibility, and feasibility.
Animal models play a critical role in preclinical studies during development of all medications, including vaccines (Kanesa-thasan et al., 2011). For example, rats and mice are used for investigations into fundamental basic science issues to establish ranges of dosing, to explore immunogenicity, and even to provide perspectives on some clinical outcomes. Studies of acute toxicity, tolerability, and causes of fever have been performed in guinea pigs and rabbits (Kanesa-thasan et al., 2011). Subsequent studies of safety may be carried out in rats or primates, as appropriate (Kanesa-thasan et al., 2011). Animal models may also be useful for studies exploring novel vaccines, the extent of interference with vaccine immunogenicity by concurrently administered vaccines, and the bactericidal qualities of antibodies. In its review of the existing evidence of the immunization schedule and safety, the committee did not explicitly review mechanistic evidence for any health outcomes, such as case studies or existing animal models, and instead points to the excellent work of previous committees in their reviews of individual vaccines (IOM, 2002, 2012). However, various stakeholders expressed interest in the potential use of animal models, and the committee therefore also considered the potential of studies with animal models of disease to advance knowledge of the biological mechanisms by which the childhood immunization schedule might be associated with adverse events.
To use animal models for the biological study of the recommended immunization schedule, however, many challenges must be overcome and limitations must be appreciated. For example, if one is interested in events that are purported to occur long after vaccine administration, such as asthma or food allergy, one must establish the generalizability of animal models of those diseases to the human context. Furthermore, spontaneously occurring models of diseases in animals would have to be developed before
To the committee’s knowledge, realistic animal models that could provide information on the potential of long-term health outcomes of the full immunization schedule in humans are not available. Furthermore, an assessment of the long-term effects of multiple immunizations in, for example, rats 3 months after they receive those immunizations would not be applicable to humans because the onset of such chronic diseases takes years to arise in humans.
An example of an animal model is the model of allergic hypersensitivity to dust mites and Ascaris in monkeys, which has resulted in studies of asthma (Hogan et al., 1994). However, few such primate colonies with relevance to asthma in humans exist. Furthermore, the cost to establish and maintain a primate colony is extremely high, and the availability of allergic monkeys is therefore extremely limited.
In the absence of animal models of the spontaneous onset of chronic diseases such as Guillain-Barré syndrome, studies of the effects of multiple vaccinations on aspects of airway hyperreactivity in mice or monkeys could be performed, but such studies would be limited in their ability to answer questions about the aggregate immunization schedule.
The key limitation to the use of animal models for evaluation of the immunization schedule therefore is not the availability of science or resources but the limited ability of models to produce results generalizable to the human experience. Given the committee’s recognition of the complexity of the immunization schedule, the importance of family history, the role of individual immunologic factors, and the complex interaction of immunization with the health care system, the committee determined that it would be more appropriate to focus future research efforts on human research rather than research involving animal models.
In summary, it is not possible to recommend studies with animals to inform the notion that the aggregate childhood immunization schedule results in the onset of chronic diseases. The committee also recognized the role of animal models in understanding neurological diseases, which have made important contributions to the understanding of disease processes that affect the brain in terms of structural or motor changes, such as seizures. In addition to the limitations described above in relation to chronic diseases, the study of neurological diseases such as autism has limited use for animal models since “no animal embodies the repertoire of behaviors seen in the human, and in particular, no animal has language equivalent to that of the human” (IOM, 2012, p. 86). Thus, there are sizable barriers in using animal models to assess such neurological outcomes following administration of the childhood immunization schedule.
The complexity of the current immunization schedule, which includes variables such as the number of doses, the age of administration, and the time between doses, permits the examination of a large number of potential research questions. Nevertheless, the committee noted a general lack of consistent and integrated theories of biological mechanisms or pathways that link specific elements of the immunization schedule to specific health conditions in the vaccinated child.
Perhaps the most compelling hypothesis is that introduction of an excess of immune-stimulating agents into an immature or dysregulated immune system might result in a cascade of adverse immunological processes culminating in asthma, allergies, autoimmune disorders, and the like. Nevertheless, the biological evidence to support this line of reasoning was examined by an Institute of Medicine committee in 2002 as part of the Immunization Safety Review series, and that examination found no more than weak justification for such a hypothesis (IOM, 2002).
Likewise, the committee’s review of existing epidemiological studies of the immunization schedule was complicated by the effectively infinite number of variations for delivery of the recommended childhood immunization schedule that could be investigated. The literature summarized in Chapter 5 reflects the range of approaches that have been used to characterize departures from the recommended schedule, and no single approach prevailed across multiple investigations.
The committee struggled in its efforts to identify research questions that could be posed to evaluate the health outcomes after immunization with the recommended childhood immunization schedule because of a lack of well-defined exposures and biologically plausible outcomes. Thus, the primary research questions of interest that the committee identified and that are listed below are broad and most likely too general to be readily translated into new research studies, unless biologically plausible hypotheses emerge.
Among the many questions about the current immunization schedule that could be posed, the committee identified what it viewed to be the leading research questions of interest on the basis of a review of stakeholder concerns. The committee parsed the phrase “this question” in Part 2 of the statement of task into four broad research questions. These questions are listed in Box 6-1.
The committee identified other potential gaps in research on the larger health care delivery system and policy-setting procedures that influence parents’ knowledge of and decisions about their immunization choices for their children. For example, several stakeholders identified the need for additional research on effective provider-patient communications on the risks and benefits of vaccinations. Others suggested the value of additional re-search
- How do child health outcomes compare between those who receive no vaccinations and those who receive the full currently recommended immunization schedule?
- How do child health outcomes compare between (a) those who receive the full currently recommended immunization schedule and (b) those who omit specific vaccines?
- For children who receive the currently recommended immunization schedule, do short- or long-term health outcomes differ for those who receive fewer immunizations per visit (e.g., when immunizations are spread out over multiple occasions), or for those who receive their immunizations at later ages but still within the recommended ranges?
- Do potentially susceptible subpopulations—for example, children from families with a history of allergies or autoimmune diseases— who may experience adverse health consequences in association with immunization with the currently recommended immunization schedule exist?
on patient barriers to obtaining vaccinations. Although the committee acknowledges that these subjects are of interest and indeed are merely two examples of a large number of potential questions about the system of delivery of the immunization schedule that research could evaluate (see Chapter 4), they are beyond the scope of this committee’s task. Therefore, the committee makes no recommendations regarding further research aimed at addressing such concerns; however, the committee encourages HHS to make continued efforts to identify populations facing barriers to immunization and consider stakeholder concerns on the safety, efficacy, and delivery of the immunization schedule and communication about the immunization schedule, as detailed in Recommendation 4-1.
This chapter focuses on potential health benefits or concerns about the recommended schedule at the individual level (e.g., the vaccinated child) and population-level considerations, including monitoring of community immunity (also called “herd immunity,” which is the indirect protection afforded to unimmunized individuals, e.g., infants too young to be vaccinated against pertussis when a sufficient fraction of the population is vaccinated), that are necessary for study of the recommended immunization schedule.
The next section focuses on research questions that directly address the individual health benefits and risks of the recommended immunization schedule for the vaccinated child and describes a number of research approaches that could be pursued. The chapter then highlights the critical point that the consequences of individual vaccination choices can be considered only in light of the level of immunization in the larger population, to which the individual is invariably linked.
The committee recognized the vital importance of considering the population health impacts of any studies of the childhood immunization schedule. As the immunization schedule exists within a complex system consisting of individual-level protection and community immunity, studies that require any variations to the immunization schedule may have a profound impact on broader population health. After the discussion of methods to study individual health outcomes, the committee describes methods to monitor and maintain community immunity.
Each of the primary research questions of interest to stakeholders concerned about the safety of the immunization schedule described in Box 6-1 could be investigated by a range of study methods that vary considerably according to their cost, feasibility, and ethical propriety. At the one end are secondary analyses of existing data sets that could be initiated immediately; at the other end are primary research efforts involving the collection of new data, most notably, large, new RCTs.
This section describes the range of research approaches that could be pursued to investigate the leading questions of interest, with attention given to each approach’s potential according to cost, feasibility, and anticipated scientific yield and utility. The research strategies broadly include
- initiation of new RCTs,
- initiation of new observational studies, and
- secondary analyses of data from current vaccine safety surveillance systems in the United States (such as VSD) and comparable international systems.
Each of these approaches has some potential to advance knowledge of the four primary research questions identified. The following sections discuss the strengths, limitations, cost, and feasibility of each approach.
It is widely acknowledged that when it is possible to randomize study participants, the RCT is the preferred design for evaluating the effectiveness and safety of health interventions. Data obtained from RCTs are often touted as the “gold standard” for clinical evidence, and results from a properly conducted clinical trial are considered to be of superior quality and reliability to evidence from most observational studies. The committee deliberately considered the form that an RCT of the immunization schedule could take and explored whether such a design would be both ethical and practical.
The critical advantage of the RCT is its ability to randomly assign participants to follow one of two or more different immunization schedules. Such a design would enable researchers to be reasonably certain that any observed difference in outcomes would be free of bias that could result from unequal allocation to treatment groups and would create reasonably comparable groups. The outcomes observed in a well-conducted RCT thus should accurately reflect an actual causal effect of treatment rather than results that could arise from population differences (Friedman et al., 2010).
Although it is well established that vaccines prevent a vast burden of disease among immunized as well as unimmunized or underimmunized people via community immunity, data suggest that some children continue to receive no vaccinations. One could argue that it would be ethical to recruit this population to an RCT comparing a group that receives the standard vaccination schedule with a group that receives no immunization. Because participants would be randomly placed in one of these study arms, at least half of the participating children, who otherwise would receive no vaccination, would receive all or part of the recommended immunization schedule. The other half would receive no benefit, except for a possible improvement in community immunity that would increase their chances of avoiding vaccine-preventable diseases. They would also avoid any hypothetical risk of receiving immunizations according to the ACIP-recommended schedule.
The committee considered and rejected this logic on the basis that any child, even the child of a parent who staunchly rejects vaccination, who is randomized to a no-vaccination arm is essentially consigned to an elevated risk of severe illness and even possible death should the child contract a vaccine-preventable disease. Moreover, should a child in the no-vaccination arm contract a preventable disease, the risk to other unprotected people in the community would increase. Randomization of such a child would also place the child’s pediatrician in the position of having to go against professional medical guidelines. Likewise, parents of intentionally unvaccinated children are unlikely to allow their children to be randomized to receive vaccines. Similarly, the committee believes that any study stipulating that
some children receive less than the recommended immunization schedule would not be ethical. The ethics of human experimentation always trump scientific and other considerations, and no study that needlessly endangers children is acceptable. As the committee did not find evidence to suggest that the current schedule is unsafe, the committee concludes that any RCT comparing the current schedule with an alternative schedule that does not provide full and timely coverage of all the currently recommended vaccines would offer an unacceptable risk of vaccine-preventable diseases in individuals and in the population.
The committee believes that it may be ethical to use the RCT design to evaluate the third research question, which seeks to determine how health outcomes differ for those who receive the full recommended schedule in unconventional ways. A potential schedule that might be feasible as a comparative intervention is one that would disperse the vaccinations within the recommended window so that children are visiting their health care providers more often but receiving fewer doses at each visit. An example of such a study would be one that compares the health of infants who receive their five immunizations at the 4-month visit during one encounter with a health care provider with the health of infants who receive the same immunizations after age 4 months over the course of five separate visits. Because such a dispersed vaccination schedule would require an increased number of visits, often in rapid succession over a period of a few weeks, such a study would add substantial costs to both parents and providers and, moreover, may be unacceptable to insurers if its effectiveness—measured as a decreased rate of adverse outcomes—is negligible. Although it is unobjectionable ethically, the committee considered the time and financial strains resulting from immunization on a dispersed schedule to be too prohibitively costly to recommend pursuing this line of research and, thus, does not endorse this method as a feasible option for studying the recommended immunization schedule.
Certain segments of the population, including premature infants, children born into families with histories of autoimmune disease, and children with genetic traits not yet identified that confer an increased chance of developing diseases having autoimmune features, could be vulnerable both to putative harmful effects of vaccination and, conversely, to the absence of protection from vaccine-preventable diseases should they not be vaccinated. The benefits of immunization to such possibly vulnerable populations could surpass those to children in nonvulnerable groups, allowing them to avoid vaccine-preventable diseases that, although mild for others, could be severe for them. One might hypothesize, however, that the risk of a severe adverse effect of immunization is elevated in this group if, for example, administration of several vaccines causes an immune overload that precipitates the onset of an immunological disease.
If observational data suggest that a particular element of the schedule is associated with a particular adverse outcome in an identifiable subgroup, it could be ethical to conduct a randomized trial of the schedule with such a population, if such a trial does not require some children to receive a reduced schedule that would put them at risk for vaccine-preventable disease. However, as both the potential risks and the benefits are elevated and, moreover, the research community does not currently have a sound idea of the magnitudes of those risks and benefits, it is premature to propose RCTs to evaluate differences in outcomes between these hypothesized groups.
General Feasibility Issues
As detailed in Chapter 3, RCTs to evaluate the introduction of individual vaccinations are conducted within the context of the currently recommended childhood immunization schedule. The committee found no evidence that a trial has ever been conducted to evaluate the entire immunization schedule, for example, to compare administration of the recommended schedule of vaccines with administration of an alternative schedule. To conduct such a trial would require careful consideration of multiple factors. For instance, it has been established that some vaccines are associated with fevers, febrile convulsions, anaphylaxis, and other syndromes, which in some cases are similar to the symptoms of the diseases that they are intended to prevent. These adverse reactions are mostly rare. For example, febrile seizures occur for only 1 of every 3,000 measles, mumps, and rubella (MMR) vaccine doses (IOM, 2012), but a sufficiently large study of the safety of a schedule that omits or delays MMR would likely show an increased risk of seizures in the group receiving the regular doses of MMR. Unless researchers somehow accounted for the occurrence of the more serious preventable diseases, it may appear that nonvaccination is “safer” in this respect. To further complicate matters, the rare unvaccinated child in an otherwise heavily vaccinated area will benefit from community immunity and may thus appear to have done better than his or her peers, some of whom will develop adverse effects, such as fever.
Because vaccination in the United States essentially begins at birth, an RCT of the immunization schedule would have to randomize children either before birth or shortly thereafter. In addition to the many practical difficulties that this raises, randomization before birth means that the trial cannot be conducted solely through interactions with child health care providers, as pregnant women will typically be seeing a pregnancy care provider in the months preceding delivery. Such a trial would also require parents to adhere to their child’s assigned schedule for at least 6 years and to avoid catch-up immunizations in the years that follow to evaluate hypothesized long-term health outcomes, all of which would likely add up to
Clinical trials commonly mask participants and evaluators to the identity of the randomized treatments to prevent bias in the evaluation of treatment effects. In an RCT comparing the recommended schedule with an alternative schedule, masking of subjects would involve administration of placebo injections at the recommended vaccination times (for the alternative arm) and at the alternative times (for the recommended arm). Such a scheme would be cumbersome and difficult to implement, potentially causing errors in treatment administration and discouraging good compliance. It would also be unacceptable to parents, who would object to their children being repeatedly injected.
One key limitation of RCTs, which was discussed in Chapter 2 in the context of RCTs already performed to evaluate vaccine safety, is that they generally require large sample sizes to have adequate power. The power critically depends on the incidence rate of the adverse outcome in question. For example, a 90 percent power to detect a halving of the rate of an adverse event that occurs in 8 percent of children would require a relatively small sample size, likely no more than 2,000 participants. With disorders that are less common, for example, those that occur in only 1 percent of a population, one would need about 15,000 subjects to achieve a 90 percent power of detection. For events that occur very rarely, for example, in 0.25 percent of children, a trial would need upward of 50,000 participants to have the same level of power. Given the weak biological justification for the association of the immunization schedule with any adverse outcome, an RCT would have to include tens or hundreds of thousands of participants to be powered to look for a range of outcomes simultaneously, including those that are very rare (see Appendix D).
Only if observational studies suggest specific hypotheses to address could researchers use smaller sample sizes in follow-on RCTs. Given the large number of participants that would be required, the cost of such trials would also be prohibitive. Tens of millions of dollars would likely be required to adequately study the identified hypotheses. A federal investment in an RCT of the immunization schedule would therefore be infeasible, and unless further epidemiological evidence of safety problems from observational studies reveals a safety problem, such an investment could be considered wasteful.
Overall, the committee recognizes the value of the RCT in providing definitive data on the potential effects of the immunization schedule on adverse outcomes and asserts that the RCT should have a role in the overall research program on the safety of the schedule. Even though RCTs on individual and combination vaccines are part of the federal research
infrastructure, in the absence of data to suggest that the current schedule is unsafe, the committee must reject on ethical grounds any RCT design that compares the current schedule with an alternative that does not involve full vaccination within the permitted time windows. The committee believes that if clearly defined, biologically plausible hypotheses emerge from observational studies—either studies based on current resources, such as VSD, or studies with newly recruited cohorts—then these could serve as the basis for further research by the use of studies with the RCT design. Before HHS initiates further research on the entire immunization schedule, a thorough review of the biological plausibility of the association of a particular outcome with an aspect of the schedule should be conducted.
Recommendation 6-2: The Department of Health and Human Services should refrain from initiating randomized controlled trials of the childhood immunization schedule that compare safety outcomes in fully vaccinated children with those in unvaccinated children or those vaccinated by use of an alternative schedule.
New Observational Studies
Observational studies are the cornerstone of epidemiological science and are often used to evaluate associations between exposures and outcomes in situations in which randomization to a treatment arm would be unethical or in which it would not be feasible, either because of costs or other factors, to directly assign and monitor an intervention in the study population. Observational studies can involve either primary data, in which new data are obtained by the investigators to examine study hypotheses, or secondary data analysis, in which instances investigators analyze data that have been previously collected. In its consideration of the use of observational methods to address the four research questions of interest to stakeholders concerned about the safety of the immunization schedule identified in Box 6-1, the committee discussed potential options and challenges for studies with both primary data (in this section), and secondary data (in the section that follows).
Prospective Cohort Studies
Prospective cohort studies, which monitor—forward in time— populations selected on the basis of their exposure status, would be the most ambitious options involving primary data collection to address research questions, such as comparison of the health outcomes between children who receive no vaccinations and those who receive the full, currently recommended immunization schedule. As was mentioned earlier in
this report, a small percentage of the U.S. population receives no recommended childhood immunizations for reasons ranging from religious or philosophical beliefs, such as followers of Christian Science and some in U.S. Amish communities, to health reasons, such as children with certain conditions, to personal convictions about the safety of vaccines. Given the above-average proportion of unimmunized children in these populations, ranging from 4 to 16 percent in surveys of different communities (Smith et al., 2004; Wenger et al., 2011), it has been suggested that such a population could serve as a naturally occurring unimmunized group in designing a new prospective cohort study. However, such a study would have limited utility to accurately assess differences in health outcomes between unimmunized and fully immunized children. First, there are questions regarding the potential size and resulting statistical power for such a study. As with RCTs, sufficiently large numbers of participants would need to be recruited for each study arm—those who are unimmunized and those who are fully immunized. Because some Amish communities and other potential naturally occurring unimmunized populations have relatively so few unvaccinated children, the sample population of unimmunized children who could be recruited would likely be too small to provide adequate statistical power, particularly for very rare outcomes (see Appendix D).
Furthermore, the study would need to account for the many confounding variables that distinguish distinct subgroups of naturally occurring unimmunized populations from the rest of the U.S. population, including lifestyle factors and known genetic variables that may play a role in the development of allergies, asthma, and other conditions. For example, data from the National Immunization Survey have shown that unimmunized children are characteristically different from children who are underimmunized or fully immunized on the basis of race, gender, socioeconomic status, and parental concerns (Smith et al., 2004). For all these reasons, the committee does not recommend the pursuit of prospective cohort studies with distinct subgroups of naturally occurring unimmunized populations (such as those who decline immunizations due to membership in specific religion or cultural groups).
One option warranting additional investigation would involve embedding a new prospective cohort study of nonvaccinated and fully vaccinated families within the VSD surveillance system. If adequate numbers of fully unvaccinated children were included within VSD, it might be possible to identify comparable, well-matched, fully vaccinated children and actively monitor both groups over time with direct assessments of health functioning. In contrast to a study of, for instance, Amish families only, this study would likely include a more diverse and less highly-selective group of unimmunized children (with reduced potential for confounding) and with a larger sample size.
Further investigation of the number and characteristics of fully unvaccinated children or children vaccinated by use of alternative schedules within VSD appears warranted. It would be important to ensure an adequately comparable comparison group of fully vaccinated children. The committee raised some concerns that differences between the comparison groups of interest might constrain the utility of such a study, for reasons discussed below in regards to secondary analyses.
Furthermore, to be of sufficient scientific quality, such a study would require considerable effort to retain study participants. Additional consideration should be given to the feasibility of assessing long-term health outcomes for participants in VSD and the cost of doing so. This information would be essential to adequately assess the feasibility and cost of initiating a new prospective cohort study nested within VSD.
In addition to studies focused on existing unimmunized populations, the committee recognized that other longitudinal cohort studies of infants and children could be informative for evaluating long-term health outcomes after immunization, if a large sample size was available and accurate recording of immunization coverage was possible. One such opportunity is the National Children’s Study (NCS), which is funded by both the U.S. Congress and the National Institutes of Health through the Children’s Health Act of 2000 and which received total funding of $744.6 million from fiscal years 2007 to 2011. The budgetary request for fiscal year 2013 is $165 million, which will fund the continuation of the pilot study and introduction of data collection for the main study (National Children’s Study, 2012).
The main NCS will be a multicenter effort that will examine the effect of a child’s environment—including variables such as air and water quality, diet, family dynamics, and cultural influences—on his or her general health and well-being from birth through age 21 years. With a target population of 100,000 children, the NCS will be adequately powered to evaluate rare health outcomes and will aim to prioritize the investigation of environmental determinants of neurodevelopmental disorders and asthma, among other outcomes. Once begun, the main study will actively collect immunization histories. NCS therefore affords an opportunity to study potential health outcomes among children with a range of immunization histories, and the committee encourages such efforts through NCS and other similar cohorts to create a rich set of data for continued research.
Given the opportunity available through NCS, the limits of studying distinct subgroups of naturally occurring unimmunized populations, and the high cost of pursuing prospective data collection, the committee does not consider the initiation of new prospective cohort studies to be the most feasible or fruitful approach to studying the recommended immunization schedule at this time.
Although they are less demanding in time and cost than a cohort study, the committee concluded that studies with case-control designs are unlikely to advance knowledge and provide answers to the four primary research questions of interest to concerned stakeholders presented in Box 6-1. The main reasons for this conclusion are that (1) the major variations in immunization history of interest are relatively uncommon, necessitating the enrollment of a large number of affected cases and unaffected study participants, and (2) it is not clear how accurately investigators would be able to retrospectively reconstruct details of the child’s vaccination history. In addition, case-control studies can be used only if the adverse event of interest is known (see Appendix D for further discussion). Additional methodological work designed to determine the accuracy of retrospective ascertainment of vaccine histories and known adverse events may well be warranted.
Secondary Analyses of Existing Databases
Unlike prospective observational studies, which require the collection of new data, secondary analyses of accumulated data, such as retrospective cohort or case-control studies, are traditionally less resource intensive because they generally rely largely on information previously or routinely collected in existing databases. Given the comprehensive state of immunization data systems in the United States, the committee considered secondary analyses with data from existing data sets to be the most feasible option for the study of the safety of the childhood immunization schedule. In particular, a number of questions about variations in the current immunization schedule could be further investigated by the use of VSD.
VSD is the premier electronic health record (EHR)-based vaccine safety data system in the United States (Baggs et al., 2011; Chen et al., 1997; DeStefano, 2001). As noted in Chapter 3, VSD is a collaboration between the CDC and nine health plans that serve about 9.5 million members and that have an annual birth cohort of more than 100,000. In recent years, funding for VSD has totaled approximately $9 million per year, with additional funding being provided for special projects, making VSD a relatively low-cost and effective data system for investigating immunization safety (Frank DeStefano, CDC, personal communication, September 25, 2012).
VSD could be valuable for answering the research questions that the committee identified in Box 6-1 because it includes information on the immunization histories of participants that can be used to identify
- individuals vaccinated according to some alternative immunization schedules;
- variations in immunization schedules because of different immunization policies in the participating health plans, variations in clinical practice, vaccine shortages, problems with access, or parental decisions to delay vaccinations;
- multiple outcomes, including adverse events, diagnoses, and procedures as well as mortality;
- covariates, including race, age, gender, and zip code–level demographics; and
- global indices of shorter-term child health and service utilization, including numbers of days hospitalized, numbers of emergency room visits, and so forth.
Accordingly, secondary analyses of the data in VSD databases would add to current knowledge and help answer the four primary research questions listed in Box 6-1. For example, in a review of alternative immunization schedules in the Kaiser Permanente Colorado system, VSD researchers initiated a retrospective matched cohort study to examine patterns and trends for children defined as undervaccinated at ages 2 to 24 months and compared the health care utilization rates between undervaccinated children and children vaccinated at the appropriate age.
Eight sites in the VSD participated in this study. Of 323,247 children born (within the participating managed care organization sites) between 2004 and 2008, 48.71 percent were considered undervaccinated for at least 1 day before age 24 months. The prevalence and specific patterns of undervaccination significantly increased across the study duration. In a matched cohort analysis, undervaccinated children had a significantly lower outpatient visit rate (11 percent) than did children who were vaccinated in an age-appropriate manner. In contrast, undervaccinated children had significantly greater (25 percent more) inpatient hospital admission rates than did children vaccinated at the appropriate age.
In a second matched cohort analysis, children who were undervaccinated because of parental choice had fewer outpatient visits and emergency room encounters than did children vaccinated at the appropriate age. In this second matched cohort analysis, no significant detectable difference in inpatient visit rates was detected between the two groups. Among children considered undervaccinated for any reason, 1,399 instances of undervaccination (variations in immunization history that could indicate alternative schedules) were detected. Among children undervaccinated because of parental choice, 756 distinct instances of undervaccination were detected (Glanz et al., 2013). More study will clearly be needed to draw conclusions
As already mentioned, families electing different immunization schedules presumably differ in meaningful ways (e.g., according to their access to health care providers, attitudes toward vaccines, health care utilization, and sociodemographic factors). Although these differences may not affect reported incidence of adverse events or the presence of disease, they could be related to individual beliefs or to access to health care. Although confounding can ultimately be reduced by explicit adjustment for covariates, it cannot be fully addressed through analysis of existing study variables.
Moreover, the VSD system has limitations, including a population limited to children in private health care plans and therefore not representative of the entire U.S. population, loss of children to follow-up when families move or switch insurers, and an occasional need for additional data not routinely collected by VSD. These limitations may be addressed by the collection of supplementary data, including through patient interviews or medical record reviews.
To address the adequacy of long-term follow-up data, the magnitude of patient attrition from VSD would need to be fully investigated. For example, preliminary evidence suggests that among children born in 2001, over half continue to be included in the VSD database (Frank DeStefano, CDC, personal communication, August 28, 2012).
Collection of Additional Data on VSD Participants
One potential enhancement to VSD would be to collect additional demographic and, possibly, family history data for current participants. Basic information on vaccination history, child gender, race/ethnicity, and birth status (e.g., gestational age or birth weight) could be systematically collected for all participants. New approaches to the collection of additional data on a family history of allergies, autoimmune disorders, neurological disorders, and the like should be considered. These data would permit analyses of the fourth research question (about potentially susceptible subpopulations) that cannot be readily conducted at this time.
Collection and banking of blood samples with appropriate informed consent from VSD participants would support subsequent analyses of subpopulations that are potentially susceptible to adverse events according to genetic and epigenetic characteristics.
A more costly enhancement to the current system would be to attempt to capture additional data on child health, possibly including additional data on participants’ use of health care services that are not already in the database.
Finally, it might be conceivable to conduct direct assessments of subgroups of interest (e.g., those who receive no vaccinations and a comparable group that receives the full immunization schedule). This option is discussed further below, but it is more feasible to study children who have had incomplete immunizations by a specified age than to identify children considered vaccine refusals because the population which falls into the latter category is generally very small.
Extending the Length of Follow-Up of VSD Patients
A limitation of VSD is that it includes data only from individuals in the nine participating health plans. Families with young children may move and switch health plans, resulting in limited follow-up information after their immunizations. This shortcoming is largely overcome in comparable systems in Scandinavia and the United Kingdom because of their universal health care systems and patient registries that contain information on medical services received from primary care providers. The use of strategies to collect health care utilization data through EHRs or provider reports after a participant has left the original health plan may warrant consideration.
Increasing the Number and Variety of VSD Participants
With an annual birth cohort of more than 100,000 participants, the total number of children monitored through VSD is substantial. However, national estimates derived from a representative sample of all U.S. children, including those in public health plans, suggest that less than 1 percent of children receive no vaccines. Data from VSD (Jason Glanz, University of Colorado–Denver, personal communication) suggest that the number of unvaccinated children within VSD is generally consistent with national values. Approximately 1.23 percent of children participating in VSD had no vaccinations recorded by age 1 year, and 1 percent of children had no vaccinations recorded by age 2 years. These estimates are limited to children who were born between 2004 and 2008 and who had a minimum period of enrollment in VSD of 12 months and a maximum enrollment of 36 months. It is not clear how commonly other variations of the recommended immunization schedule occur among the children in VSD.
In addition, the diversity of the participants represented in VSD is limited by the fact that managed care organizations in the Southwest and rural South are not currently among the managed care organizations participating in VSD. Furthermore, because VSD does not now include any public insurance plans, its population has fewer low-income and minority individuals than the number in the U.S. population as a whole. Options to broaden the diversity of VSD participants would enhance the utility of this
Further discussion would be required to assess the feasibility and cost of such efforts. The committee noted that although VSD represents the most promising system for investigating outcomes after immunization with the recommended childhood immunization schedule, other resources discussed in Chapter 3, such as VAERS, the National Immunization Survey, and immunization information systems, are highly valued resources for monitoring vaccine safety and coverage as well. The Post-Licensure Rapid Immunization Safety Monitoring (PRISM) program, which has been used to evaluate vaccine safety in a larger cohort than the VSD, may have the capability to monitor rare adverse events potentially associated with the childhood immunization schedule. However, the data are not yet well-characterized.
Analyses of comparable international immunization surveillance systems in countries including Denmark, the United Kingdom, and Canada have historically been better suited for these purposes for the reasons described below. Although consideration of international immunization surveillance systems was not central to the committee’s task, analyses in Denmark, the United Kingdom, Canada, and other countries also hold considerable promise for advancing knowledge about the health outcomes associated with the immunization schedule. First, as discussed in Chapter 3, these countries often collect and maintain full immunization histories for the entire population, greatly increasing the total sample size and the number of children immunized with less common combinations of vaccines (including no vaccines). Second, many of these countries have comprehensive health and educational registries permitting linkage to longer-term and less severe child outcomes. Third, these systems include a richer set of variables on sociodemographic characteristics and family history, permitting analyses of potentially susceptible subpopulations.
The committee considered but does not recommend cross-national comparisons because of the potential bias and lack of generalizability from results that must account for different environments, vaccine antigens, or immunization schedules. The U.S. population differs from the populations in other countries in important ways, including on the basis of genetics and health care history. Even vaccine efficacy can vary among populations, as has been demonstrated in separate studies of a Haemophilus influenzae type b conjugate vaccine in two different populations (Eskola et al., 1990; Ward et al., 1990). A cross-national comparison to study child health outcomes related to recommended childhood immunization schedules would require careful and extensive consideration of the possible covariates, many of which may not be known at this time. Ecological comparisons may be useful for monitoring disease trends and detecting epidemiological signals; however, the information gathered from such studies could not be
The major limitations of U.S. surveillance systems to address the primary research questions identified in this report are (1) the potentially limited number of families included in these systems who will have used the major alternative immunization schedules of interest; (2) potentially high rates of migration from the participating health care organizations, resulting in varying and often short-term follow-up after vaccination; (3) limits on how much information on less severe health outcomes is collected from participating children; and (4) limited ancillary information routinely collected about participating children, such as premature birth or a family history of allergies.
Despite these limitations, VSD is currently the best available system for the study of the safety of the immunization schedule in the United States and holds tremendous promise for advancement, including the potential for future prospective cohort studies. Furthermore, continuing to move toward the increased use of EHRs (as encouraged by federal funding), which are what allow VSD to capture and link large amounts of immunization and health data on children, will help the United States establish richer data sets that are more comparable to those in other high-income countries.
To further enhance the data collected by VSD, the system should strive to obtain complete demographic information to strengthen its functions and generalizability to the whole U.S. population. Secondary analyses with data from other existing databases similar to VSD would be feasible, ethical, and a lower-cost approach to investigating the research questions that the committee identified, including research on alternative immunization schedules. To date, the data obtained from VSD have already been used to study health outcomes of children with incomplete immunizations or who may follow alternative schedules, as described above. In addition, the VSD system has a large enough proportion of unvaccinated children to investigate differences in health outcomes of unvaccinated and vaccinated children. Increased efforts to collect information on individual medical histories could lead to a fruitful source of data for studying which populations are potentially susceptible to vaccine adverse events. The committee recognizes that the currently funded managed care organizations’ commitment to VSD studies needs to remain high to continue and build upon existing efforts. Additionally, VSD’s utility will be expanded with the addition of more detailed demographic data and family medical histories.
Recommendation 6-3: The committee recommends that the Department of Health and Human Services (HHS) and its partners continue to
fund and support the Vaccine Safety Datalink project to study the safety of the recommended immunization schedule. Furthermore, HHS should consider expanding the collaboration with new health plan members and enhancing the data to improve its utility and generalizability.
If large numbers of children avoided immunization, community immunity would be eroded and this protective effect would disappear for those who are not or who cannot be fully vaccinated. Thus, any analysis of vaccine safety data needs to consider the community immunity aspect of the milieu in which the study is conducted. Such complications would affect both clinical trials and observational studies.
Consideration of Population Impacts of Alternative Schedules
Attempts to quantify the relative safety of contrasting immunization schedules need to take into account at least two separate health outcomes: (1) adverse events related to the administration of specific vaccines and the overall immunization schedule, and (2) the respective impacts of alternative schedules on the circulation of vaccine-preventable diseases and the consequent adverse outcomes associated with infection. Secondary effects (such as longer waiting times and the greater cost of care if more visits are needed for immunization) and potential medical errors in provider offices accustomed to the routine schedule would also have to be measured.
Previously, high-profile analyses have focused on calculation of the number of serious reactions either per vaccine or over the immunization schedule compared with the per child risk of hospitalization associated with vaccine-preventable diseases (Sears, 2011). Although such analyses are intuitively appealing, they overlook the intimate association between immunization and age-specific disease incidence. Specifically, any shifts in the immunization schedule that lead to a net increase in the time spent vulnerable to these diseases will inevitably increase the circulation of these pathogens. The population-level impacts of such an outcome will be a simultaneous rise in the incidence of the affected infectious diseases and a reduction in the age at which they are contracted. Thus, not only is the risk of exposure to vaccine-preventable diseases increased but so is the likely severity of infection, which may be most acute in younger children (Heiniger et al., 1997).
A clear manifestation of the dual impact of immunization on the incidence and age distribution of vaccine-preventable diseases has been documented in Sweden, where the pertussis vaccine was removed from the
national pediatric immunization schedule in 1979 because of concerns over the reactogenicity of the whole-cell vaccine (Gangarosa et al., 1998; Romanus et al., 1987). After a 17-year hiatus, the acellular pertussis vaccine was added to the immunization schedule in 1996 (Carlsson and Trollfors, 2009). Analyses of age-stratified incidence reports highlighted both a sharp decline in the incidence and a marked increase in the age distribution of pertussis cases as a result of the resumption of immunization against pertussis (Rohani et al., 2010). Importantly, Swedish data also illustrate the concept of community immunity.
A pattern similar to that seen in Sweden has been observed in England and Wales, where declines in the uptake of MMR after controversy instigated by a subsequently retracted paper questioning the vaccine’s safety were associated with a rise in measles notifications and a shift in the incidence of measles toward younger age groups (Jansen et al., 2003).
Predicting Changes to Community Immunity
As outlined in the commissioned paper (see Appendix D), a variety of designs may be used to compare the safety of alternative schedules. It is, unfortunately, difficult predict the long-term population-level consequences of disease transmission as a result of changes to the immunization schedule. It is possible, however, to use mathematical and computational models to predict the impacts of changes in the administration of any one specific vaccine on the incidence of the infectious disease affected by that vaccine. This process involves three distinct steps: model formulation, parameterization, and model validation. These and other elements of the models are described below.
The development of a disease-specific transmission model begins with determination of the model structure and key processes, which are informed by the known immunology and epidemiology of the system. For instance, a loss of immunity may be a necessary ingredient for a model of pertussis transmission, whereas a latent carrier stage may be appropriate for varicella (Anderson and May, 1992; Keeling and Rohani, 2008). The model also needs to explicitly consider age-dependent heterogeneities in contact rates, susceptibility to complications, and reporting.
A number of age-specific models have been proposed for many of the key childhood infections, including measles (Anderson and May, 1992; Schenzle, 1984), pertussis (Hethcote, 1998; Rohani et al., 2010), Streptococcus pneumoniae infection (Cobey and Lipsitch, 2012), rubella (Metcalf et al., 2011), and chickenpox (Ferguson et al., 1996).
The usefulness of any model and the reliability of its predictions depend on its veracity. Thus, models need to be carefully based on ground truths, a process that is made particularly challenging for high-dimensional age-structured models because a fundamental challenge to the effective parameterization of age-specific models is determination of the appropriate patterns of contact by age. It is fortunate that recent studies have addressed this problem, and detailed information on the typically age-stratified patterns of contact in the United States (Del Valle et al., 2007) and a number of European countries (Mossong et al., 2008) is now available. Synthesis of this information together with historical incidence data to formulate validated transmission models is made possible by the use of modern inference techniques, including sequential Monte Carlo methods for hypothesis testing (Ionides et al., 2006). An example is the age-structured pertussis model developed by Rohani et al. (2010) and parameterized with data from incidence reports from Sweden.
The production of fully validated transmission models requires access to age-specific incidence reports. This is often a critical bottleneck in such an endeavor, as public health agencies (e.g., CDC) do not routinely provide such complete data via, for instance, the National Notifiable Diseases Surveillance System (Goldwyn and Rohani, 2012). When detailed incidence reports, stratified by age, county, and immunization status (e.g., through the Supplementary Pertussis Surveillance System), do become available, requests for access to such data are not always granted in a timely manner, and may be answered with the provision of data that was not obtained using the best-available methods (Thacker et al., 2012).
Quantifying Uncertainty and Sensitivity
The predictions of any formal modeling analyses need to be evaluated within the context of their inherent variability and should be subject to extensive sensitivity analyses (Blower, 2000). Uncertainty in predictions can be quantified by use of a wide array of rigorous probabilistic approaches to model execution, whereby the system of equations is translated into a Markov chain process (Gibson and Bruck, 2000; Gillespie, 1977; Keeling and Rohani, 2008). Such an approach would permit a detailed situational analysis, whereby the model could provide policy makers with information on the most likely (i.e., the median) outcome, for example, the size of the focal vaccine-preventable disease outbreak given a specific change in the
immunization schedule. This approach would also provide information about extreme outcomes or the 95th percentile of predicted outbreak sizes (Park et al., 2009; Rohani et al., 2009). Examination of sensitivity involves extensive repetition of the model simulation as a critical parameter of interest (e.g., the efficacy of the first dose of diphtheria and tetanus toxoids and acellular pertussis vaccine adsorbed administered at 3 months of age) is systematically varied.
The development, appropriate parameterization, and scrutiny of mechanistic transmission models have been adopted by a number of governmental agencies, and this process has been influential for determination of the implementation of specific immunization practices in countries such as the United Kingdom. In 2002, for example, Edmunds et al. used an approach similar to that outlined here to examine the potential cost-effectiveness of introduction of an acellular pertussis booster vaccine to the schedule in England and Wales (Edmunds et al., 2002). Similarly, Jit et al. (2008) carried out extensive analyses of detailed transmission models to inform the policy decision of the government of the United Kingdom on the effectiveness of routine vaccination of 12-year-old schoolgirls against human papillomavirus. Other examples include identification of the optimal targeting of age groups to contain the influenza pandemic (Medlock and Galvani, 2009), as well as pinpointing the most effective immunization schedule for meningococcal serogroup C (Trotter and Edmunds, 2006).
The committee deliberated on many potential research approaches and worked to determine which were feasible, ethical, and cost-effective. The commissioned paper in Appendix D helped identify methods that could be considered. Many questions can be answered by use of the methods described above, although they are not currently well integrated.
Chapter 7 summarizes the committee’s judgment on its statement of task. Setting of priorities for research will be challenging. For example, the committee does not recommend a study comparing the recommended immunization schedule and no immunization at this time because a high-quality randomized trial is not ethical and a prospective observational study could be complex, lengthy, and expensive and would potentially provide inconclusive results about key health outcomes after immunization. Thus, the committee proposes establishment of a process for setting priorities incorporating epidemiological and other evidence (on the basis of formal systematic reviews), biological plausibility, feasibility, and stakeholder concerns.
Anderson, R.M., and R.M. May. 1992. Infectious diseases of humans: Dynamics and control, New York: Oxford University Press.
Baggs, J., J. Gee, E. Lewis, G. Fowler, P. Benson, T. Lieu, A. Naleway, N.P. Klein, R. Baxter, E. Belongia, J. Glanz, S.J. Hambidge, S.J. Jacobsen, L. Jackson, J. Nordin, and E. Weintraub. 2011. The Vaccine Safety Datalink: A model for monitoring immunization safety. Pediatrics 127(Suppl 1):S45-S53.
Blower, S.M. 2000. A tale of two futures: HIV and antiretroviral therapy in San Francisco. Science 287(5453):650-654.
Carlsson, R.M., and B. Trollfors. 2009. Control of pertussis—lessons learnt from a 10-year surveillance programme in Sweden. Vaccine 27(42):5709-5718.
Chen, R.T., J.W. Glasser, P.H. Rhodes, R.L. Davis, W.E. Barlow, R.S. Thompson, J.P. Mullooly, S.B. Black, H.R. Shinefield, and C.M. Vadheim. 1997. Vaccine safety datalink project: A new tool for improving vaccine safety monitoring in the United States. Pediatrics 99(6):765-773.
Cobey, S., and M. Lipsitch. 2012. Niche and neutral effects of acquired immunity permit coexistence of pneumococcal serotypes. Science 335(6074):1376-1380.
Del Valle, S.Y., J. Hyman, H.W. Hethcote, and S.G. Eubank. 2007. Mixing patterns between age groups in social networks. Social Networks 29(4):539-554.
DeStefano, F. 2001. The Vaccine Safety Datalink project. Pharmacoepidemiology and Drug Safety 10(5):403-406.
Edmunds, W.J., M. Brisson, A. Melegaro, and N.J. Gay. 2002. The potential cost-effectiveness of acellular pertussis booster vaccination in England and Wales. Vaccine 20(9):1316-1330.
Ferguson, N.M., R.M. Anderson, and G.P. Garnett. 1996. Mass vaccination to control chickenpox: The influence of zoster. Proceedings of the National Academy of Sciences of the United States of America 93(14):7231-7235.
Friedman, L.M., C.D. Furberg, and D.L. DeMets. 2010. Fundamentals of clinical trials. New York, NY: Springer.
Gangarosa, E.J., A. Galazka, C. Wolfe, L. Phillips, R. Gangarosa, E. Miller, and R. Chen. 1998. Impact of anti-vaccine movements on pertussis control: The untold story. Lancet 351(9099):356-361.
Gibson, M.A., and J. Bruck. 2000. Efficient exact stochastic simulation of chemical systems with many species and many channels. Journal of Physical Chemistry A 104(9): 1876-1889.
Gillespie, D.T. 1977. Exact stochastic simulation of coupled chemical reactions. Journal of Chemical Physics 81(25):2340-2361.
Glanz, J.M., S.R. Newcomer, K.J. Narwaney, S.J. Hambidge, M.F. Daley, N.M. Wagner, D.L. McClure, S. Xu, A. Rowhani-Rahbar, G.M. Lee, J.C. Nelson, J.G. Donahue, A.L. Naleway, J.D. Nordin, M.M. Lugg, and E.S. Weintraub. 2013. A population-based cohort study of undervaccination in 8 managed care organizations across the United States. JAMA Pediatrics:1-8.
Goldwyn, E.E., and P. Rohani. 2012. Bias in pertussis incidence data and its implications for public health epidemiology. Journal of Public Health Management and Practice. Epub ahead of press.
Gust, D.A., S. Campbell, A. Kennedy, I. Shui, L. Barker, and B. Schwartz. 2006. Parental concerns and medical-seeking behavior after immunization. American Journal of Preventive Medicine 31(1):32-35.
Gust, D.A., D. Weber, E. Weintraub, A. Kennedy, F. Soud, and A. Burns. 2008. Physicians who do and do not recommend children get all vaccinations. Journal of Health Communication 13(6):573-582.
Hethcote, H. 1998. Oscillations in an endemic model for pertussis. Canadian Applied Mathematics Quarterly 6:61-88.
Hogan, M.B., K.E. Harris, and R. Patterson. 1994. A naturally occurring model of immunoglobulin E antibody-mediated hypersensitivity in laboratory animals. Journal of Laboratory and Clinical Medicine 123(6):899-905.
IOM (Institute of Medicine). 2002. Immunization safety review: Multiple immunizations and immune dysfunction. Washington, DC: National Academy Press.
IOM. 2012. Adverse effects of vaccines. Washington, DC: The National Academies Press.
Ionides, E., C. Bretó, and A. King. 2006. Inference for nonlinear dynamical systems. Proceedings of the National Academy of Sciences of the United States of America 103(49): 18438-18443.
Jansen, V.A.A., N. Stollenwerk, H.J. Jensen, M. Ramsay, W. Edmunds, and C. Rhodes. 2003. Measles outbreaks in a population with declining vaccine uptake. Science 301(5634):804.
Jit, M., Y.H. Choi, and W.J. Edmunds. 2008. Economic evaluation of human papillomavirus vaccination in the United Kingdom. British Medical Journal 337:a769.
Kanesa-thasan, N., A. Shaw, J.J. Stoddard, and T.M. Vernon. 2011. Ensuring the optimal safety of licensed vaccines: A perspective of the vaccine research, development, and manufacturing companies. Pediatrics 127(Suppl 1):S16-S22.
Keeling, M.J., and P. Rohani. 2008. Modeling Infectious Diseases in Humans and Animals. Princeton, NJ: Princeton University Press.
Levi, B.H. 2007. Addressing parents’ concerns about childhood immunizations: A tutorial for primary care providers. Pediatrics 120(1):18-26.
Medlock, J., and A.P. Galvani. 2009. Optimizing influenza vaccine distribution. Science 325(5948):1705-1708.
Metcalf, C., C. Munayco, G. Chowell, B. Grenfell, and O. Bjørnstad. 2011. Rubella meta-population dynamics and importance of spatial coupling to the risk of congenital rubella syndrome in Peru. Journal of the Royal Society Interface 8(56):369-376.
Mossong, J., N. Hens, M. Jit, P. Beutels, K. Auranen, R. Mikolajczyk, M. Massari, S. Salmaso, G.S. Tomba, and J. Wallinga. 2008. Social contacts and mixing patterns relevant to the spread of infectious diseases. PLoS Medicine 5(3):e74.
National Children’s Study. 2012. About the study. Bethesda, MD: National Children’s Study. http://www.nationalchildrensstudy.gov/about/Pages/default.aspx (accessed October 5, 2012).
Park, A.W., J.M. Daly, N.S. Lewis, D.J. Smith, J.L.N. Wood, and B.T. Grenfell. 2009. Quantifying the impact of immune escape on transmission dynamics of influenza. Science 326(5953):726-728.
Rohani, P., R. Breban, D.E. Stallknecht, and J.M. Drake. 2009. Environmental transmission of low pathogenicity avian influenza viruses and its implications for pathogen invasion. Proceedings of the National Academy of Sciences of the United States of America 106(25):10365-10369.
Rohani, P., X. Zhong, and A.A. King. 2010. Contact network structure explains the changing epidemiology of pertussis. Science 330(6006):982-985.
Romanus, V., R. Jonsell, and S.O. Bergquist. 1987. Pertussis in Sweden after the cessation of general immunization in 1979. Pediatric Infectious Disease Journal 6(4):364-371.
Schenzle, D. 1984. An age-structured model of pre- and post-vaccination measles transmission. Mathematical Medicine and Biology 1(2):169-191.
Sears, R.W. 2011. The vaccine book: Making the right decision for your child. New York, NY: Little, Brown and Company.
Trotter, C.L., and W.J. Edmunds. 2006. Reassessing the cost-effectiveness of meningococcal serogroup C conjugate (MCC) vaccines using a transmission dynamic model. Medical Decision Making 26(1):38-47.
Wenger, O.K., M.D. McManus, J.R. Bower, and D.L. Langkamp. 2011. Underimmunization in Ohio; Amish: Parental fears are a greater obstacle than access to care. Pediatrics 128(1):79-85.