Click for next page ( 162


The National Academies | 500 Fifth St. N.W. | Washington, D.C. 20001
Copyright © National Academy of Sciences. All rights reserved.
Terms of Use and Privacy Statement



Below are the first 10 and last 10 pages of uncorrected machine-read text (when available) of this chapter, followed by the top 30 algorithmically extracted key phrases from the chapter as a whole.
Intended to provide our own search engines and external engines with highly rich, chapter-representative searchable text on the opening pages of each chapter. Because it is UNCORRECTED material, please consider the following text as a useful but insufficient proxy for the authoritative book pages.

Do not use for reproduction, copying, pasting, or reading; exclusively for search engines.

OCR for page 161
Appendix D Study Designs for the Safety Evaluation of Different Childhood Immunization Schedules Martin Kulldorff1 SUMMARY To date, there have been few comparative studies evaluating the safety of different vaccine schedules. A few of the existing studies have shown that there are cases in which the risk of adverse events can depend on the vac- cination schedule used. Hence, it is both a feasible and an important area of study. As a relatively new field of investigation, the big question is what types of study designs will be most fruitful for evaluating different child- hood vaccine schedules. A number of possible study designs are presented in this review to evaluate different features or components of the vaccine schedule. These include the timing of individual vaccines, the timing be- tween doses of the same vaccine, the interaction effect between vaccines and concurrent health conditions or pharmaceutical medications, the interaction effects of different vaccines given on the same day, the ordering of different vaccines, and the effect of cumulative summary metrics such as the total number of vaccines or the total amount of some vaccine ingredient. Study designs for the comparative evaluation of one or more complete schedules are also considered. Methods are presented both for adverse events with an early onset, which are the easiest to study, and for adverse events with a late onset, including serious chronic conditions. It is concluded that a wide variety of different vaccine schedule components can be studied. 1  Department of Population Medicine, Harvard Medical School and Harvard Pilgrim Health Care Institute. 161

OCR for page 161
162 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY INTRODUCTION Before approval by the Food and Drug Administration (FDA), vaccines are evaluated for efficacy and safety using large Phase III randomized con- trolled trials. For childhood vaccines, the number of children enrolled in these trials is typically in the thousands. That is sufficient to detect common but not rare adverse events. For the latter, there exist several postmarketing vaccine safety surveillance systems using observational data on children who receive the vaccines as part of their general care. In the United States, these include the Vaccine Adverse Event Reporting System (VAERS), the Vaccine Safety Datalink (VSD), and the Clinical Immunization Safety As- sessment Network, all sponsored by the Centers for Disease Control and Prevention (CDC), as well as the Post-Licensure Rapid Immunization Safety Monitoring System (PRISM), which is part of the FDA-sponsored Mini- Sentinel Initiative. Internationally, there are other important vaccine safety surveillance systems such as the Epidemiology Vaccine Research Program at the National Institute for Health Data and Disease Control in Denmark; the Vaccine Adverse Event Surveillance and Communication (VAESCO) Network, coordinated by the European Center for Disease Control; the World Health Organization (WHO) Collaborating Centre for International Drug Monitoring; and the Immunization Division at the Communicable Disease Surveillance Centre in England. All these vaccine safety systems have proven to be very useful and important. They have detected unsus- pected adverse events leading to revisions in vaccine recommendations and, in other cases, established the safety of vaccines for which important safety concerns existed. Throughout their existence, there has been continuous and rapid development with respect to the types of questions studied and the epidemiological and statistical methods used. For example, for every new childhood vaccine approved by the FDA, VSD now conducts near real- time safety surveillance using weekly data feeds from electronic health re- cords (Lieu et al., 2007; Yih et al., 2011). The credit for these continuously improved vaccine safety surveillance systems goes to the devoted scientists that are building the systems and using them for many important studies, to the government agencies supporting this work, and to the vaccine safety advocacy groups that are the key public voice for improved and expanded vaccine safety surveillance. Most postmarketing studies evaluate the general question as to whether or not a vaccine causes an adverse event. Very few postmarketing studies have evaluated whether the risk of adverse events depends on the schedul- ing of the vaccines. For example, few postmarketing studies have evaluated whether the risk of adverse events depends on the age at which a vaccine is given, on the relative timing of two different vaccines, or on a combined cumulative effect generated by the timing of dozens of different vaccines.

OCR for page 161
APPENDIX D 163 These are all different components of the vaccine schedule, and any one of these could potentially be related to the number and severity of adverse events. When evaluating the safety of different vaccine schedules, it is hence important to study the whole range of issues, from the timing of a single vaccine to summary metrics based on the timing of dozens of vaccines. The paper presented in this appendix was commissioned by the Insti- tute of Medicine Committee on the Assessment of Studies of Health Out- comes Related to the Recommended Childhood Immunization Schedule. The paper considers different types of potential questions and concerns about the safety of vaccine schedules and describes different epidemiologi- cal study designs and statistical methods that can be used to answer such questions in a scientifically rigorous manner. The core of this paper is a set of proposals for the type of study designs and methods that would be ap- propriate for the comparative evaluation of vaccine adverse events under different vaccine schedules, and the paper is written in the context of the many difficulties raised by the speakers at the committee meetings held in February and March 2012. Note, though, that it is not a synthesis, an evaluation, or a review of the many excellent presentations made at those meetings. Instead, it should be viewed as complementary information. Note also that the paper does not say anything about the advantages or disad- vantages about specific vaccines or vaccine schedules. Rather, the focus is on potential study designs and methods and their ability, or inability, to answer such questions. DEFINITIONS OF KEY TERMS Component of the vaccine schedule: some specific feature of the vaccine schedule, such as the age at which one of the vaccines is given or the total amount of immune-stimulating content received from all vaccines in the schedule. Not to be confused with different components of a single vaccine. Early onset: an adverse event that manifests itself and can be detected within a few weeks after vaccination. Late onset: an adverse event that does not manifest itself and/or cannot be detected until a few months or years after vaccination. Potential adverse event: a health event under evaluation in a vaccine safety study, in order to determine if it is caused by the vaccine(s) or not. VACCINE SCHEDULES, ADVERSE EVENTS, AND DATA SETS Vaccine Schedules and Their Components To study the safety of different childhood vaccine schedules is an im- portant but complex task. With dozens of vaccines, many of which have

OCR for page 161
164 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY multiple doses, there are an almost infinite number of possible vaccine schedules that can be used. To scientifically evaluate the safety of differ- ent vaccine schedules, it is necessary to look at specific components of the schedule. Some such components are as follows: Timing of Specific Vaccines • The age at which a specific vaccine is given, such as the age at the first dose of the hepatitis B vaccine. • The relative timing of different doses of the same vaccine, such as the number of months between the first and second doses of the 7-valent pneumococcal conjugate vaccine (PCV7). • The interaction between the timing of a specific vaccine and time- varying health events or health status, such as a vaccination given to a child taking a temporary or seasonal medication. Relative Timing of Two or More Different Vaccines • The interaction among different vaccines given on the same day, such as the effect of giving the measles, mumps, and rubella (MMR) vaccine and varicella vaccines at the same health care visit or dif- ferent health care visits. • The order in which different vaccines are given, such as whether measles vaccine is given a few months before or after the ­ iphtheria- d tetanus-pertussis (DTP) vaccine. Summary Metrics of a Vaccine Schedule • The total number of vaccinations given to the child before a certain age, such as the 6th birthday. • The average age at which the vaccines were given. • The cumulative amount of immune-stimulating content present in all vaccines received. In addition to specific components of the vaccine schedule, one can also try to compare complete vaccine schedules. Comparison of Complete Vaccine Schedules • Whether or not the child has approximately followed the CDC- recommended vaccine schedule. • The comparative safety of a specific alternative vaccine schedule, such as Dr. Bob’s (Sears, 2007), versus the one recommended by CDC.

OCR for page 161
APPENDIX D 165 The study design and statistical methods used depend on which vac- cine schedule component is being evaluated. As they are quite different, each component of Components (a) to (e) is dealt with in separate sections of this paper. For cumulative summary metrics, the methods are similar irrespective of what component of the vaccine schedule the metric is de- signed to measure. Components (f) to (h) are treated together. Methods for comparing different complete vaccine schedules are discussed, and one vaccine schedule with completely unvaccinated children is evaluated. More general methodological issues and financial and ethical considerations are also discussed. The different types of studies should not be done in isolation from each other. If it is found that one complete vaccine schedule has an excess number of adverse events compared to another, we do not know which component of the schedule caused the difference. Hence, it is not recom- mended that studies comparing complete schedules be conducted without also evaluating specific components of those schedules. Likewise, when a specific component is studied, results may be confounded by other compo- nents of the vaccine schedule. For example, a child receiving vaccine A at an early age may be more likely to also receive vaccine B at an early age, and the timing of vaccine A will then be correlated with the number of ad- verse events even if it is the timing of vaccine B that is the culprit. It could also be that there are two different vaccine schedule components that cause adverse events but that they cancel each other out when one looks at the difference between two complete schedules, making it impossible to detect the problem if only the complete schedules are studied. Another reason for studying specific components of the vaccine sched- ule is that, if a problem is found, we need to know how to revise the schedule in order to reduce the number of adverse events. Just because one complete vaccine schedule is found to cause more adverse events than an- other, we do not necessarily have to revise all components of that schedule. Adverse Events with Early Versus Late Onset In vaccine safety studies, the goal is to evaluate if there is a causal relationship between the vaccine(s) and some health event of interest. The latter is denoted as a potential adverse event, as it may or may not be an actual adverse event caused by the vaccine(s). The type of health event un- der study determines the appropriate methodological methods for vaccine safety studies. This paper considers two main types. The first type consists of potential adverse events with an early onset that can be detected soon after the onset. The event itself could be either acute and of a passing na- ture without any permanent damage, such as a febrile seizure, or chronic, lasting many years, such as a stroke. The second type consists of potential adverse events with a late onset several months or years after vaccination

OCR for page 161
166 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY and events with an early or a gradual onset that cannot be detected until long after vaccination. For simplicity, all of these are denoted ”late onset.” These potential adverse events can also be either acute or chronic in nature. The most suitable study designs and analysis methods are greatly de- pendent on whether the potential adverse event has an early or late onset, and in the description below, separate methods are proposed for the two outcome types. This is a little bit of a simplification, since there are, of course, also potential adverse events that fall somewhere in between on this spectrum. It should also be pointed out that an early-onset chronic condi- tion can be studied by use of either of the methods described for early or late onset, but the early-onset methods are in most cases preferable. Another key issue is whether there is a clear time at which the potential adverse event happened, as with, for example, a seizure, or whether the dis- ease evolves more gradually, without a single clearly defined day of onset, as with, for example, narcolepsy or autism. This does not affect the study design as much as the time of onset, but it is an important consideration when defining and collecting the data. For most potential adverse events, we are interested only in incident diagnoses, that is, the first time that a particular diagnosis has been made. For example, if a child is diagnosed with asthma at age 2 years and then has a follow-up visit for his/her asthma at age 4 years, we do not want to attribute the asthma to a vaccination given at age 3 years. Depending on the potential adverse event under study, one can define an incident diagnosis as a diagnosis that has not occurred during the previous D days. The value of D will depend on the adverse event, but a typical value is about 1 year. The potential adverse event studied either can be very specific, such as febrile seizures or autism, or can be more general, such as all-cause outpa- tient physician visits, emergency department visits, or hospitalizations. The latter set of events may seem more desirable, as it includes the combined effect of the vaccine schedule on all important health events, but the op- posite is true. Such general definitions are more prone to biases, and they are therefore more difficult to study. This is because people that follow the CDC-recommended vaccine schedule may be different from those that do not, in terms of their health care–seeking behavior. For example, parents that are more prone to take their children to the doctor when the child is sick may also be more prone to take their children to the well care visits during which most vaccines are given. Data Sets for Postmarketing Vaccine Safety Studies To facilitate the understanding of the study designs and methods de- scribed in subsequent sections, a brief background is first given concerning

OCR for page 161
APPENDIX D 167 some of the data sets that are available and currently used for postmarket- ing vaccine safety studies. Premarketing Randomized Trials Phase III randomized trials are primarily designed to evaluate the ef- ficacy of vaccines. They are also able to find common adverse events, but their sample size is typically not large enough to evaluate rare but seri- ous adverse events. Their primary use for postmarketing vaccine safety surveillance is to generate study hypotheses. For example, a single case of Kawasaki disease in the vaccine arm of a Phase III randomized trial is not evidence that the vaccine causes Kawasaki disease, since it could be pure coincidence, but it may warrant a postmarketing safety evaluation. Spontaneous Reporting Systems Most countries in the world have a vaccine safety surveillance system based on spontaneous reports. These are linked together through the WHO Collaborating Centre for International Drug Monitoring in Uppsala, Swe- den, so that it is possible to combine data from multiple countries. In the United States, CDC and FDA are joint sponsors of VAERS. These systems contain spontaneous reports of suspected vaccine ad- verse events sent in by physicians, nurses, patients, parents, manufacturers, and others. The gender and age of the vaccinated person are some of the variables collected. There is often information about multiple vaccines given on the same day. Analyses are done by the use of proportional reporting ratios (Evans et al., 2001) and similar methods. For example, if 1.5 percent of all vaccine-related adverse event reports are for seizures and there are 1,000 reports for vaccine A, then we would expect 15 seizure reports for vaccine A. If, in reality, there are 45 such reports, the proportional reporting ratio is 3. That is more than what one would expect, and it may indicate that there is an excess risk of seizures after vaccination. Actual analyses are more complex, since it is necessary to adjust for age and other variables. There are also other more sophisticated methods used (Bate et al., 1998; DuMouchel, 1999; Rothman, 2011). The major advantage of VAERS is that it receives reports from the whole country. The two major disadvantages are that there is underreport- ing and that there are no reliable denominator data. That is, while we have information about a number of vaccinated children with the potential adverse event of interest, we do not know the total number of children that were vaccinated, how many unvaccinated children had the same type of event, or how many vaccinated children had the event without it being reported.

OCR for page 161
168 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY Some reports to VAERS are studied further in the Clinical Immuniza- tion Safety Assessment Network. Among other things, this network aims to “improve the scientific understanding of vaccine safety at the individual patient level” by obtaining and evaluating detailed genetic and other infor- mation from each patient (LaRussa et al., 2011). Electronic Medical Records For 2011, it is estimated that 57 percent of office-based physicians used electronic medical records (EMRs), up from 24 percent in 2005 (Hsiao et al., 2011). The EMRs most useful for medical research are the ones from large health plans, as they contain medical records for a well-defined mem- ber population, including both inpatient and outpatient encounters. The VSD project is the premier EMR-based vaccine safety system in the United States (Baggs et al., 2011; Chen et al., 1997; DeStefano and the Vaccine Safety Datalink Research Group, 2001). Led by CDC, it is a collaboration with 10 health plans: Group Health in the State of Washington; Harvard Pilgrim/Atrius Health in Massachusetts; HealthPartners in Minnesota; Kaiser Permanente in Colorado, Georgia, Hawaii, Northern California, Oregon, and Southern California; and Marshfield Clinic in Wisconsin. Together, these health plans have about 9.5 million members and an an- nual birth cohort of more than 100,000. The VSD system is used both for retrospective studies and for near-real-time vaccine safety surveillance with weekly analyses of newly approved vaccines. Similar systems exist in a few other countries, including the Epidemiology Vaccine Research Program at the National Institute for Health Data and Disease Control (Statens Serum Institut) in Denmark. The major advantage with EMR systems is that denominator data are available, as all vaccinated children can be identified. It is then possible to compare the number of adverse events in vaccinated and unvaccinated children or vaccine-exposed and unexposed time periods within the same child. A disadvantage is that the data are registered for purposes other than research, and there is sometimes miscoding of health events. Depending on the health outcome, manual chart review is therefore sometimes warranted. Health Insurance Claims Data Health insurance companies have medical information for millions of insured members and their families, which they receive when doctors and hospitals file their financial reimbursement claims. One such system in the United States is the PRISM program, run by FDA as part of its Mini-­ Sentinel project (Nguyen et al., 2012). Claims data are more limited than EMRs but can be used in much the same way for postmarketing vaccine

OCR for page 161
APPENDIX D 169 safety studies. The major advantage is the large sample size that can be achieved. The major disadvantage is that some health conditions are not captured. Depending on the potential adverse event under study and the confounders that need to be adjusted for, this may or may not be a problem. Because of their similarities, EMRs and health insurance claims data will be treated as the same type of data in this appendix under the name “health plan data.” Study-Specific Data Collection Sometimes, new data are collected specifically for vaccine safety studies, such as a self-controlled case series, a case-control study, a cohort study, or a postmarketing randomized trial. An intermediate option is to obtain some of the data from health plans, disease registries, and/or vaccine registries, while the remaining data are collected from study-specific patient surveys or measurements. The available options are too many to provide a detailed description of each. TIMING OF SPECIFIC VACCINES In a randomized childhood vaccine trial, the age at which the vaccine is given is tightly controlled by the study design, to correspond to the future planned vaccine schedule. This is appropriate, but once a vaccine is on the market, it is also given at a wide variety of other ages, for a variety of rea- sons. There are two scenarios in which it is of great interest to evaluate the risk of a vaccine as a function of the age at which the vaccine was given. (i) If a vaccine safety study has shown that there is a statistically significant excess risk of an adverse event, we want to know if the excess risk varies by the age at which the vaccine was given. (ii) Even if a general safety study covering all age groups has not shown a statistically significant excess risk of the adverse event, there could still be an excess risk if the vaccine is given at certain ages outside the recommended schedule. Such a safety problem could be masked by the noneffect among the most populous age group, and a special study looking at age-specific risks would be warranted. Known Adverse Events with Early Onset Background Some vaccines have been shown to cause an acute adverse event within a few weeks after vaccination. Examples include intussusception 3 to 7 days after vaccination with rotavirus vaccine (RotaShield) (Kramarz et al., 2001; Murphy et al., 2001) and febrile seizure 7 to 10 days after vaccina-

OCR for page 161
170 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY tion with MMR and the measles, mumps, rubella, and varicella (MMRV) vaccine (Klein et al., 2010). There are also several such examples of less severe adverse events like fever and rash. The adverse event may be serious enough to warrant the withdrawal of the vaccine from the market, as with the rotavirus vaccine, or it may be mild enough to keep using the vaccine, as with MMR. A midlevel alternative option is to revise the vaccination schedule to minimize the number of adverse events or to contraindicate the vaccine in a certain age group. Knowledge of the relative and attributable risk of the adverse event as a function of age is one important component when deciding between these options, together with other important fac- tors, such as how the immunogenicity varies by age. This paper discusses only methods for obtaining knowledge about the former and not how to weight different sources of information to arrive at a final decision. Examples In two different studies, Gargiullo et al. (2006) and Rothman et al. (2006) evaluated the effect of age on the excess risk of intussusceptions after vaccination with the rotavirus vaccine (RotaShield). In a more recent study, Rowhani-Rahbar et al. (2012) evaluated the effect of age on the risk of febrile seizures after vaccinations with MMR and MMRV. All three stud- ies found that the risk of the adverse event varied greatly by age. Data EMRs from health plans and health insurance claims from health plans are ideally suited for studying this question. It is also possible to use data from a case-control study. VAERS data cannot easily be used since VAERS does not contain information about the age distribution of vaccinated chil- dren. Too few data are available from premarketing randomized because such trials are too small and typically do not include individuals over a wide enough range of ages. In light of existing observational data, specifically designed postmarketing randomized trials could be unethical, depending on the nature of the known adverse event. Methods The first key step is to determine the time between the vaccination and the adverse event as precisely as possible. Some children will, just by chance, have the adverse event soon after vaccination. To maximize the precision of our age estimates, we want to exclude as many of them as possible, by counting only the adverse events occurring in the true risk window. An efficient way to determine the appropriate risk window is to

OCR for page 161
APPENDIX D 171 use a temporal scan statistic. For a cohort of vaccinees with a subsequent event of interest, record the number of days from vaccination to the event. Ignore events that occur on the same day as the vaccination, as they may have a different background rate, as well as those that occur beyond an upper limit, such as 70 days after vaccination. If there is no relationship between the vaccine and the adverse event, we expect the adverse events to be uniformly distributed during the [1, 70]-day period. The temporal scan statistic scans the time period for any cluster of events, without any assump- tions about their location or length. The method determines the statistical significance of such clusters, adjusting for the multiple testing inherent in the hundreds of overlapping time periods evaluated. As an example, tem- poral scan statistics were used to determine that the excess risk of seizures after vaccination with MMRV is confined to the 7- to 10-day postvaccina- tion period (Klein et al., 2010). The second step is to evaluate the relationship between age at vaccina- tion and excess risk of the adverse event. The simplest and most common way to do this is to divide age into different groups, such as 6 to 12 months and 12 to 24 months, and compare the risk. It is unrealistic to assume that the risk suddenly jumps at a particular age, and for greater precision, it is better to model risk as a continuous function of age. This can be done by the use of either regression with first-, second-, and higher-degree polynomi- als or regression splines (Rothman et al., 2006). In these analyses, it is important to take the underlying natural age- related risk into account. For example, the incidence of intussusceptions is very low immediately after birth, after which it gradually increases until about 5 months of age, after which it gradually decreases (Eng et al., 2012). There are a number of possible ways to adjust for this, depending on the exact study design. In a cohort study of vaccinated individuals, one can use historical data to estimate the age curve, using a polynomial function, and then use that as an offset term in the regression model. An alternative approach is to use both a risk and control interval for each individual, in a self-controlled analysis, evaluating whether the relative risk in these intervals varies by age of vaccination. Note, though, that if the natural incidence rate for the adverse event varies greatly by age in weeks rather than years, it is still necessary to incorporate an offset term based on the natural age curve even when a self-controlled analysis is conducted. In a case-control study, matching by age ensures that the age-based incidence curve is adjusted for.

OCR for page 161
190 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY and that is correlated with the outcome of interest. The goal here is to get a larger variance in the health outcome being measured, thereby increasing the statistical power. It is important that the probability of selection be unrelated to any aspect of the vaccine schedule. After the study population has been selected, proceed as described for the first scenario. Vaccine-Specific Versus General Components of Vaccine Schedules The more general components of the vaccine schedule described above, as well as the comparison between complete schedules discussed earlier, are considerably more difficult to study than the more vaccine-specific compo- nents described above. There are several reasons for this. First and foremost, there are many alternative vaccine schedules, and slightly different schedules have to be lumped together in the same com- parison group. For the cumulative summary metrics, many different vac- cination schedules will have the same value, for example, the average age at vaccination. If one vaccine schedule is safer than an alternative vaccine schedule in terms of a specific outcome but they both have the same average age at vaccination, then the effect size will be attenuated and go undetected. If a statistically significant excess number of adverse events is found, a second problem with these designs is that it can be hard to know which aspect of the schedule caused the excess or reduced risk. Is it the timing of one specific vaccine, is it an interaction between two or more vaccines, or is it something else? Hence, any statistically significant findings will have to be followed up with studies concerning more specific vaccine schedule components. A third issue is confounding. While confounding is present in all obser- vational studies, it is likely to be a greater problem when complete vaccine schedules are studied. For example, children for whom most of the vaccines are delayed from the recommended schedule may be different in terms of both health care utilization and socioeconomic factors. This may bias the results, and the bias may exist whether the delayers are deliberately follow- ing an alternative schedule or not, and the bias may go in different direc- tions for these two groups. The same type of confounding can be present when one is looking at more specific components of the vaccine schedule, but it is likely to be less strong, as such individual components are likely to have more random and less systematic variability than a complete sched- ule. A way to intuitively see this is to note that whatever it is that causes a general parental tendency to delay vaccinations, that will likely be more correlated with the average timing of all vaccinations than with the timing of a single vaccination. To date, there have been few comparative studies evaluating the safety

OCR for page 161
APPENDIX D 191 of different vaccine schedules. For the above-mentioned reasons, it is sug- gested that, initially, the majority of such studies focus on the most vaccine- specific components of the vaccine schedule described earlier, as well as the content-defined components mentioned above. Information from such studies will greatly facilitate the design and understanding of subsequent studies evaluating the more general components discussed earlier as well as the comparison of complete vaccines schedules described above. Cross-National Comparisons Different countries have different recommended vaccine schedules, so it may seem natural to do cross-national studies to compare the safety of the schedules in an ecological study design. Unfortunately, this is very dif- ficult to do well and generally not recommended. The problem is that the incidence of most diseases varies by geographical region for reasons other than the vaccine schedule, such as genetics, diet, physical exercise, or other environmental factors. Any such cross-national study may hence be heavily biased. This does not mean that one cannot do studies that include data from multiple countries or regions, as long as each one has a range of dif- ferent exposures in each place. In such studies, the geographical region can easily be adjusted for in the analysis, in order to take the differential disease incidences into account. Time Trend Evaluations Another ecological study design is to take a particular country or region and compare time trends in disease incidence with temporal changes in the vaccination rate or vaccination schedule. This is also not recommended. In addition to vaccinations, there are many other reasons why the reported disease incidence may increase or decrease over time, including changes in environmental risk factors and changes in health care practice and diag- nosis. Hence, an apparent temporal correlation between increasing disease incidence and increasing vaccine usage could be completely spurious. It should be pointed out that the bias can also go in the other direction. Even if there is no temporal correlation between disease incidence and vaccina- tions, a true relationship can be hidden by a compensatory effect from an unknown confounder. Near-Real-Time Safety Surveillance In what is called “rapid-cycle analysis,” the VSD project has pioneered near-real-time vaccine safety surveillance (Lieu et al., 2007; Yih et al., 2011). For newly approved vaccines and selected adverse events, weekly

OCR for page 161
192 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY data feeds are received from the health plans and the data are analyzed by continuous sequential statistical methods (Kulldorff et al., 2011). If there are specific concerns regarding the newly revised vaccine schedule, such rapid-cycle analysis can also be implemented for many of the study designs described above. Data Mining Most vaccine safety studies evaluate a specific vaccine-event pair. For VAERS data, data mining methods are also used, where thousands of potential vaccine and adverse event pairs are evaluated simultaneously, without there being any prior hypothesis about their being an excess risk of the event. This is done to cast the net as wide as possible. Recently, data mining methods are also started to be used for health plan data. As vaccine safety data mining develops further, it may also be used to study questions regarding the vaccine schedule. Disease-Causing Complications Versus Adverse Events It should be noted that in these types of studies, it is not always clear what is an adverse event and what is not. For example, a child may have a febrile seizure that was caused by one or more of the vaccines or a febrile seizure caused by a disease, where the child got the disease because he or she was not immunized against it. Hence, an excess risk of seizures due to a particular vaccine schedule could be due either to vaccines given at a cer- tain time when the child is more sensitive to adverse events or to vaccines not given at a certain time when the child needed the vaccine protection. If the same type of health event is caused by the vaccine among one group of children as an adverse event and by the disease among the same or another group of children as a complication, then the vaccine may be found to cause an excess number of the adverse events in a vaccinated population, since the nonvaccinated children benefit from herd immunity. Hence, find- ings about the risk to individuals in a mostly vaccinated population cannot necessarily be generalized to the population level. Vaccine Efficacy and Effectiveness This paper covers only the study of potential adverse events after vacci- nation. If a study does not find an excess risk, all is fine and there is no need to worry about vaccine efficacy. On the other hand, if a true differential risk of adverse events is found with respect to some component of the vaccine schedule, vaccine efficacy and effectiveness must also be considered when contemplating a revised vaccine schedule. Some vaccines, such as MMR,

OCR for page 161
APPENDIX D 193 have a different immune response depending on the age of the child, and vaccine efficacy therefore depends on the vaccine schedule. The timing of a vaccination also influences the time period during which the child is pro- tected from the disease and the herd immunity of the population at large. Herd immunity can also be affected if a parent refuses future vaccinations after his or her child has had an adverse event vaccine that could have been avoided with a different schedule. While such an analysis is outside the scope of this paper, all these factors must be considered in a joint cost- benefit analysis before the recommended vaccine schedule is revised, if and when there is a finding of a vaccine schedule-dependent adverse event. FINANCIAL CONSIDERATIONS When deciding what to study and what study design to use, cost is an important consideration. The study designs mentioned in this paper range from very cheap to very expensive. For some designs, the cost depends on how common the potential adverse event is. While we cannot do any precise sample size calculations, we will for illustrative purposes consider three classes of health outcomes: common, moderately rare, and very rare. Common events are those that affect more than 1 out of every 100 children, such as allergies and some learning disorders. Moderately rare outcomes are those that affect more than 1 out of 100 but less than 1 in 10,000, such as intussusception. Very rare outcomes are those affecting less than 1 in 10,000, such as Guillain-Barré syndrome, The least expensive studies are those using VAERS data. Since those data are already collected, only the investigator’s time needs to be covered. Unfortunately, VAERS data are of limited use when one is studying vaccine schedules. The cost is independent of the adverse event. The second least expensive study designs are the ones based on fully automated health plan data. While they involve no new data collection, the extraction of data from large administrative databases is a complex activity involving detailed knowledge of the database structure and content, sophisticated computer programming, and thorough data quality control. To set up a new system from scratch is very costly, but the marginal cost of additional studies in existing systems is not. In most cases, the cost is independent of the potential adverse event under study. For common and medium-rare outcomes, a VSD-size study population of about 100,000 annual births should be enough for most study designs. For very rare out- comes, data from more and larger health plans may be needed in order to achieve sufficient power, resulting in additional expenses. Bigger datasets may also be needed for common events and moderately rare adverse events when complete vaccination schedules are evaluated, if only a small propor- tion of health plan members follow the particular schedule of interest. In

OCR for page 161
194 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY summary, the cost of these types of studies is similar to the cost of current vaccine safety studies conducted in VSD, PRISM, and similar systems. With most health plan data, vaccine information has a high positive predictive value, but that is usually not the case for disease outcomes. For such adverse events, it is often necessary to conduct chart reviews to con- firm whether or not a patient actually had the health event of interest, and that will increase the cost. For very rare adverse events, it is not a major additional cost, but for medium-rare and common adverse events, it can be. One way to reduce this cost is to first do a study on fully automated data and do chart review only when that study shows an excess risk of adverse events, to confirm or dismiss that finding. The next level of cost is incurred by study designs that combine health plan data with specially collected outcome data that are not available as part of the EMRs. The cost will depend on the type of data collected but will in most situations be very high. For medium-rare and very rare out- comes, a very large number of health plan enrollees will have to be enrolled, potentially making such studies prohibitively expensive. Randomized trials are the most expensive study design. For medium- rare and very rare adverse events, the study needs to have a very large sample size to detect a potential problem. For example, if a vaccine causes a specific adverse event in 1 of every 1,000 children, that is not something that can be detected in a randomized trial with 4,000 children in each arm, for a total of 8,000, even if the baseline rate of the event is very rare. To see this, suppose that there are four adverse events in the vaccinated arm and none in the control arm receiving the placebo. Under the null hypothesis, the probability of all four being in the vaccinated arm is (1/2)4 = 0.0625, which is not statistically significant, and hence, we cannot conclude that it was the vaccine that caused the adverse events. So, for medium-rare and very rare adverse events, we need data with tens or hundreds of thousands of vaccinated children, and for such sample sizes, randomized trials are prohibitively expensive. A cost advantage of randomized trials over case- control studies is that multiple potential adverse events can be evaluated within the same study. Irrespective of the design, studies evaluating late-onset events are more expensive than those evaluating early-onset events, since the individuals must be followed for a much longer time. With health plan data, this requires larger population sizes since many children will be lost to follow- up. When health plan data are augmented with specially collected data on health outcomes, children must be tracked and monitored for a longer time, which is costly. The same is true for randomized trials.

OCR for page 161
APPENDIX D 195 ETHICAL CONSIDERATIONS As with all medical research, ethical considerations are very important when one is designing vaccine safety studies. With observational studies with health plan data, the key ethical issue is patient confidentiality, which can be ensured through existing research practices. For randomized trials, ethical considerations play a much more impor- tant role. Depending on the vaccine component of interest, a randomized trial can sometimes be conducted in a way so that both arms fall within the recommended vaccination schedule, in which case there are no ethical concerns. An example of such a trial would be whether to give children two vaccines on the same day or a week apart. At the other extreme, it would be unethical to do a randomized trial where children in one arm are completely unvaccinated, since the scientist will then knowingly put some of the children at increased risk for vaccine-preventable diseases, some of which may result in death. Somewhere in between these two extremes there is a gray zone where randomized trials may or may not be ethical, depend- ing on the vaccine schedules being compared and on the available strength of the evidence regarding efficacy and potential adverse events. Experts on medical ethics should then be consulted. For more common adverse events, randomized trials have a potential role to play in postmarketing vaccine safety studies. There is little reason to use them to evaluate the general safety of a particular vaccine, since that evaluation is already covered by the Phase III trials. Questions for which randomized trials may be used include the order in which different vaccines are given, the exact timing between doses of the same vaccine, and whether two different vaccines are given on the same day or a week apart. If a randomized trial is conducted, it is important to consider the effect on herd immunity. If the two arms differ by delaying one or more vaccines by at most a few weeks, it is not a major issue. If vaccination in one arm is delayed for a much longer time period or not given at all, it may reduce herd immunity. This may put children that are not participating in the study at increased risk for the disease, and this can be especially serious for immune-compromised children for which a vaccination is contraindicated. To minimize the negative effect on herd immunity, such randomized trials should be spread out geographically, so that there are at most a few addi- tional unvaccinated children in any given location. In that way, nonpartici- pating children will not be at an increased risk of the disease, and equally important, those children randomized to the delayed vaccination will still have some protection against the disease from herd immunity.

OCR for page 161
196 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY CONCLUSIONS Randomized trials are the “gold standard” for scientific studies, and premarketing Phase III randomized trials play an important role in the evaluation of vaccine-related adverse events. Because of their limited sample size, rare adverse events may not be detected, though. For financial and ethical reasons, the utility of randomized trials is more limited for postmar- keting vaccine safety studies. On the basis of utility and cost, health plan–based study designs are the most promising for the safety evaluation of different vaccine schedules. This is definitely true for medium-rare and very rare adverse events that cannot be detected in Phase III randomized trials, but such data can also be used to study common adverse events. The key is to always consider potential problems with confounding, and it is often a good idea to use different study designs with different potential biases for the same research question. Hypotheses about potential adverse events may come from Phase III trials or from observational postmarketing studies with data from health plans or spontaneous reporting systems. The comparative safety evaluation of different vaccine schedules is a complex and multifaceted task, and all aspects of the vaccine schedule are currently understudied with regards to potential adverse events. A number of different study designs and methods can be used to evaluate different components of the schedule. For all known and most potential adverse events, it is recommended that a wide variety of vaccine schedule components be evaluated. Direct evaluation of complete vaccine schedules is more difficult and probably less fruitful, but it is not impossible. Such studies are most useful when conducted in parallel with studies of specific components of the schedule. This is especially important when there is a significant adverse event finding, since it is otherwise impossible to know which of the many features of the complete schedule are actually causing the adverse events. This paper should not be utilized as a cookbook where definite study designs and methods are obtained and used for different classes of prob- lems in a black box–type approach. Each study is unique, depending on the vaccine(s) under study, the potential adverse event(s) of interest, the data used, and the scientific research question. All those aspects need to guide the methodology. The goal of this paper is simply to show that a wide variety of study designs and methods are available to study the comparative safety of different vaccine schedules, and the hope is that some of the proposed methods can serve as a starting point when thinking about the most suitable designs and statistical methods to use for different studies. This paper does not present an exhaustive list of study designs and methods that can be used for the comparative evaluation of potential

OCR for page 161
APPENDIX D 197 adverse events due to different childhood vaccination schedules. As more such studies are performed, additional designs and methods will surely be developed and used. The paper should not be interpreted as a recommenda- tion to use all of the study designs and statistical methods mentioned. The scientific question should drive which designs and methods are used, and while some of them may become widely used, others may not be used at all. What the paper attempts to show is that the comparative safety evalu- ation of vaccine schedules is complex and multifaceted and that a wide variety of study designs and statistical methods are available to a scientist who wishes to conduct such studies. REFERENCES Aaby, P., H. Jensen, J. Gomes, M. Fernandes, and I. M. Lisse. 2004. The introduction of diphtheria-tetanus-pertussis vaccine and child mortality in rural guinea-bissau: An ob- servational study. International Journal of Epidemiology 33(2):374-380. Almenoff, J. S., W. DuMouchel, L. A. Kindman, X. Yang, and D. Fram. 2003. Disproportion- ality analysis using empirical Bayes data mining: A tool for the evaluation of drug inter- actions in the post-marketing setting. Pharmacoepidemiol Drug Safety 12(6):517-521. Baggs, J., J. Gee, E. Lewis, G. Fowler, P. Benson, T. Lieu, A. Naleway, N. P. Klein, R. Baxter, E. Belongia, J. Glanz, S. J. Hambidge, S. J. Jacobsen, L. Jackson, J. Nordin, and E. Weintraub. 2011. The Vaccine Safety Datalink: A model for monitoring immunization safety. Pediatrics 127(Suppl 1):S45-S53. Bate, A., M. Lindquist, I. R. Edwards, S. Olsson, R. Orre, A. Lansner, and R. M. De Freitas. 1998. A Bayesian neural network method for adverse drug reaction signal generation. The European Journal of Clinical Pharmacology 54(4):315-321. CDC (Centers for Disease Control and Prevention). 2012. Recommended immunization schedule for persons aged 0 through 6 years—United States, 2012. Atlanta, GA: Centers for Disease Control and Prevention. Chen, R. T., J. W. Glasser, P. H. Rhodes, R. L. Davis, W. E. Barlow, R. S. Thompson, J. P. Mullooly, S. B. Black, H. R. Shinefield, C. M. Vadheim, S. M. Marcy, J. I. Ward, R. P. Wise, S. G. Wassilak, and S. C. Hadler. 1997. Vaccine Safety Datalink project: A new tool for improving vaccine safety monitoring in the United States. The Vaccine Safety Datalink team. Pediatrics 99(6):765-773. DeStefano, F., and the Vaccine Safety Datalink Research Group. 2001. The Vaccine Safety Datalink project. Pharmacoepidemiol Drug Safety 10(5):403-406. DeStefano, F., E. Weintraub, and C. Price. 2012. Immunogenic stimulation from vaccines and risk of autism. Paper presented at Vaccine Safety Datalink Annual Meeting, Denver, CO. DuMouchel, W. 1999. Bayesian data mining in large frequency tables, with an application to the FDA spontaneous reporting system. Journal of the American Statistical Association 53(3):177-190. Eng, P. M., T. C. Mast, J. Loughlin, C. R. Clifford, J. Wong, and J. D. Seeger. 2012. Incidence of intussusception among infants in a large commercially insured population in the United States. Pediatric Infectious Disease Journal 31(3):287-291. Evans, S. J., P. C. Waller, and S. Davis. 2001. Use of proportional reporting ratios (PRRs) for signal generation from spontaneous adverse drug reaction reports. Pharmacoepidemiol Drug Safety 10(6):483-486.

OCR for page 161
198 THE CHILDHOOD IMMUNIZATION SCHEDULE AND SAFETY Fireman, B., J. Lee, N. Lewis, O. Bembom, M. van der Laan, and R. Baxter. 2009. Influenza vaccination and mortality: Differentiating vaccine effects from bias. American Journal of Epidemiology 170(5):650-656. Gargiullo, P. M., T. V. Murphy, and R. L. Davis. 2006. Is there a safe age for vaccinating infants with tetravalent rhesus-human reassortant rotavirus vaccine? The Journal of Infectious Diseases 194(12):1793-1794. Glanz, J. 2012. Alternative immunization schedules and the risk of pertussis. Paper presented at Vaccine Safety Datalink Annual Meeting, Denver, CO. Hambidge, S. J. 2012. Seizures in children with recommended versus delayed infant immuniza- tion schedules. Paper presented at Vaccine Safety Datalink Annual Meeting, Denver, CO. Hambidge, S. J., J. M. Glanz, E. K. France, D. McClure, S. Xu, K. Yamasaki, L. Jackson, J. P. Mullooly, K. M. Zangwill, S. M. Marcy, S. B. Black, E. M. Lewis, H. R. Shinefield, E. Belongia, J. Nordin, R. T. Chen, D. K. Shay, R. L. Davis, F. DeStefano, and the Vaccine Safety Datalink Team. 2006. Safety of trivalent inactivated influenza vaccine in children 6 to 23 months old. Journal of the American Medical Association 296(16):1990-1997. Hsiao, C. J., E. Hing, T. C. Socey, and B. Cai. 2011. Electronic health record systems and intent to apply for meaningful use incentives among office-based physician practices: United States, 2001-2011. NCHS Data Brief (79):1-8. Iqbal, S., and F. DeStefano. 2012. Immunological-stimulating content of vaccines and neuro- physiological outcomes in children. Paper presented at Vaccine Safety Datalink Annual Meeting, Denver, CO. Klein, N. P., B. Fireman, W. K. Yih, E. Lewis, M. Kulldorff, P. Ray, R. Baxter, S. Hambidge, J. Nordin, A. Naleway, E. A. Belongia, T. Lieu, J. Baggs, E. Weintraub, and Vaccine Safety Datalink. 2010. Measles-mumps-rubella-varicella combination vaccine and the risk of febrile seizures. Pediatrics 126(1):e1-e8. Kramarz, P., E. K. France, F. DeStefano, S. B. Black, H. Shinefield, J. I. Ward, E. J. Chang, R. T. Chen, D. Shatin, J. Hill, T. Lieu, and J. M. Ogren. 2001. Population-based study of rotavirus vaccination and intussusception. Pediatric Infectious Disease Journal 20(4):410-416. Kulldorff, M., R. L. Davis, M. Kolczak, E. Lewis, T. Lieu, and R. Platt. 2011. A maximized sequential probability ratio test for drug and vaccine safety surveillance. Sequential Analysis 30(1):58-78. LaRussa, P. S., K. M. Edwards, C. L. Dekker, N. P. Klein, N. A. Halsey, C. Marchant, R. Baxter, R. J. Engler, J. Kissner, and B. A. Slade. 2011. Understanding the role of human variation in vaccine adverse events: The Clinical Immunization Safety Assessment Net- work. Pediatrics 127(Suppl 1):S65-S73. Leonardi, M., K. Bromberg, R. Baxter, J. L. Gardner, S. Klopfer, O. Nicholson, M. Brockley, J. Trammel, V. Leamy, W. Williams, B. Kuter, and F. Schodel. 2011. Immunogenicity and safety of MMRV and PCV-7 administered concomitantly in healthy children. Pediatrics 128(6):e1387-e1394. Lieu, T. A., M. Kulldorff, R. L. Davis, E. M. Lewis, E. Weintraub, K. Yih, R. Yin, J. S. Brown, R. Platt, and for the Vaccine Safety Datalink Rapid Cycle Analysis Team. 2007. Real- time vaccine safety surveillance for the early detection of adverse events. Medical Care 45(10 Suppl 2):S89-S95. Murphy, T. V., P. M. Gargiullo, M. S. Massoudi, D. B. Nelson, A. O. Jumaan, C. A. Okoro, L. R. Zanardi, S. Setia, E. Fair, C. W. LeBaron, M. Wharton, J. R. Livengood, and the Rotavirus Intussusception Investigation Team. 2001. Intussusception among infants given an oral rotavirus vaccine. New England Journal of Medicine 344(8):564-572. Nguyen, M., R. Ball, K. Midthun, and T. A. Lieu. 2012. The Food and Drug Administration’s post-licensure rapid immunization safety monitoring program: Strengthening the federal vaccine safety enterprise. Pharmacoepidemiol Drug Safety 21(Suppl 1):S291-S297.

OCR for page 161
APPENDIX D 199 Norén, G. N., R. Sundberg, A. Bate, and I. R. Edwards. 2008. A statistical methodology for drug-drug interaction surveillance. Statistics in Medicine 27(16):3057-3070. Pittman, P. R. 2002. Aluminum-containing vaccine associated adverse events: Role of route of administration and gender. Vaccine 20(Suppl 3):S48-S50. Price, C. S., W. W. Thompson, B. Goodson, E. S. Weintraub, L. A. Croen, V. L. Hinrichsen, M. Marcy, A. Robertson, E. Eriksen, E. Lewis, P. Bernal, D. Shay, R. L. Davis, and F. DeStefano. 2010. Prenatal and infant exposure to thimerosal from vaccines and immu- noglobulins and risk of autism. Pediatrics 126(4):656-664. Rothman, A. L. 2011. Immunity to dengue virus: A tale of original antigenic sin and tropical cytokine storms. Nature Reviews Immunology 11(8):532-543. Rothman, K. J., Y. Young-Xu, and F. Arellano. 2006. Age dependence of the relation between reassortant rotavirus vaccine (RotaShield) and intussusception. The Journal of Infectious Diseases 193(6):898. Rowhani-Rahbar, A., N. Lewis, B. Fireman, R. Baxter, N. Klein, and other VSD colleagues. 2012. Effect of age on the risk of seizure following immunization with measles-containing vaccines: VSD study 233—preliminary results. Paper presented at Vaccine Safety Datalink Annual Meeting, Denver, CO. Sears, R. 2007. The vaccine book: Making the right decision for your child, 1st ed. New York: Little, Brown. Thakrar, B. T., S. B. Grundschober, and L. Doessegger. 2007. Detecting signals of drug-drug interactions in a spontaneous reports database. The British Journal of Clinical Pharma- cology 64(4):489-495. Tse, A., H. F. Tseng, S. K. Greene, C. Vellozzi, G. M. Lee, and the VSD Rapid Cycle Analysis Influenza Working Group. 2012. Signal identification and evaluation for risk of febrile seizures in children following trivalent inactivated influenza vaccine in the Vaccine Safety Datalink project, 2010-2011. Vaccine 30(11):2024-2031. Veirum, J. E., M. Sodemann, S. Biai, M. Jakobsen, M. L. Garly, K. Hedegaard, H. Jensen, and P. Aaby. 2005. Routine vaccinations associated with divergent effects on female and male mortality at the paediatric ward in Bissau, Guinea-Bissau. Vaccine 23(9):1197-1204. Yih, W. K., M. Kulldorff, B. H. Fireman, I. M. Shui, E. M. Lewis, N. P. Klein, J. Baggs, E. S. Weintraub, E. A. Belongia, A. Naleway, J. Gee, R. Platt, and T. A. Lieu. 2011. Active surveillance for adverse events: The experience of the Vaccine Safety Datalink project. Pediatrics 127(Suppl 1):S54-S64.

OCR for page 161