Study Background and Contextual Issues
Vaccines are regarded as among the greatest public health achievements of the twentieth century (CDC, 1999a). Their use has drastically reduced morbidity and mortality from infectious diseases throughout the world. Indeed, most people in industrialized countries now have little or no recollection of epidemics of polio or smallpox or of the occurrence of pertussis. However, although incidences of vaccine-preventable infectious diseases continue to be very low in the United States, there is always a risk that low population vaccination rates will contribute to outbreaks of infectious diseases that previously were held at bay (such as recent measles and pertussis outbreaks in the United Kingdom and the United States) (CDC, 1993; Jansen et al., 2003).
The current extremely low incidences of many childhood infectious diseases (such as polio, mumps, and rubella) and thus the low risk of long-term damage or death from these diseases in the United States and other industrialized countries have led to detailed studies of the risk-benefit balance of vaccines. In 1971, for example, concerns about the well-documented frequency of adverse reactions to the smallpox vaccine, the elimination of endemic smallpox in the Western Hemisphere, and the excellent progress made in the World Health Organization smallpox eradication program by that time led the U.S. government to suspend the routine vaccination of children against smallpox because the risk of adverse effects of the vaccine in children and their family contacts was deemed greater than the risk of the disease itself (CDC, 1971).
In the United States, the Department of Health and Human Services, through the Food and Drug Administration (FDA) and the Centers for Disease Control and Prevention (CDC), has responsibility for ensuring vaccine safety. Within CDC, the National Immunization Program (NIP) is responsible for assisting health departments with immunization programs, supporting the establishment of vaccine supply contracts, administering research and operational programs for the prevention and control of vaccine-preventable diseases, and monitoring the safety and efficacy of vaccines (CDC, 2001). The NIP, in conjunction with its colleagues at FDA, uses a variety of means to continually evaluate vaccine safety, including signal detection through reports from the Vaccine Adverse Event Reporting System (VAERS), ad hoc epidemiologic studies, state and community immunization registries, and laboratory surveillance. Another resource that has been used since 1991 to evaluate vaccine safety is the Vaccine Safety Datalink (VSD).
The VSD is a large linked database that was developed in 1991 through the collaborative efforts of CDC and several private managed care organizations (MCOs) (Chen et al., 1997). The VSD currently includes data from administrative records for more than 7 million members of eight MCOs (Davis, 2004). In the VSD, vaccination records, patient characteristics, and health outcomes are linked; this allows the VSD to serve as a unique and potentially powerful resource for the ongoing evaluation of vaccine safety (Davis, 2004). With its longitudinal data on reasonably well-defined cohorts, the VSD differs from VAERS, a passive surveillance system that depends on voluntary reporting. Both data sources have strengths and limitations, but they complement one another.
The opportunities offered by the VSD for thorough investigations of vaccine safety concerns and well-designed, planned, retrospective vaccine studies have led to heightened interest in the results of VSD studies and sometimes in the VSD data themselves. A few researchers interested in particular vaccine safety hypotheses also have shown interest in accessing and analyzing VSD data. The interest in the VSD shown by researchers, advocacy groups, members of Congress, and others has brought increasing attention to its use, its limitations, and the implications of studies that were conducted through the analysis of its data.
CHARGE TO THE COMMITTEE
The Institute of Medicine (IOM) Committee on the Review of the National Immunization Program’s Research Procedures and Data Sharing Program was convened at the request of the NIP to offer advice on two
issues related to the VSD. The NIP asked IOM to convene a panel of experts to address the following charge:1
(1a) review the design and the implementation to date of the new Vaccine Safety Datalink Data Sharing Program to assess compliance with the current standards of practice for data sharing in the scientific community and, (1b) make recommendations to the National Immunization Program and the National Center for Health Statistics for any needed modifications that would facilitate use, ensure appropriate utilization, and protect confidentiality and (2a) review the iterative approaches to conducting analysis that are characteristics of studies using the complex, automated Vaccine Safety Datalink system. Examples of recent studies to be examined are a completed screening study on thimerosal and vaccines (Verstraeten et al.) and cohort studies on asthma; (2b) review whether, when, and how preliminary findings about potential vaccine-related risks obtained from the Vaccine Safety Datalink system should be shared with other scientists, communicated to the public, and used to make policy or recommendations to CDC; and (2c) make recommendations to the National Immunization Program on the release of such preliminary findings in the future.
The charge was expanded to include the National Center for Health Statistics (NCHS) in part 1 of the charge because organizational responsibility for the VSD data sharing program was transferred from the NIP to NCHS in March 2004. The recommendations related to the VSD data sharing program apply to both the NIP and NCHS; the recommendations related to the release of preliminary findings apply to only the NIP.
In response to the NIP request, the IOM assembled a committee of experts in epidemiology, biostatistics, research design, research ethics, vaccine research, risk communication, and public input into the scientific process. (See Appendix A for committee biographies.) This report is the committee’s response to the charge.
Readers familiar with vaccine safety issues may be aware of the work of earlier IOM committees focused on vaccine safety. The Committee on the Review of the National Immunization Program’s Research Procedures and Data Sharing Program is separate and distinct from other IOM committees. It was convened solely to address the charge stated above, and it has examined issues of process related to the VSD, not issues of scientific validity pertaining to specific VSD studies.
The committee gathered information to address its charge through a variety of means. It held two information-gathering meetings that were open to the public. The first, on August 23-24, 2004, focused on the first part of its charge, the VSD data sharing program; its full agenda is in Appendix D. The second meeting, on October 21-22, 2004, focused on the second part of the committee’s charge, the release of preliminary findings from the VSD; its full agenda is in Appendix E. The committee also held a closed meeting on December 13-14, 2004.
Each of the open meetings included a session for comments from the public, and many persons did speak. Both meetings were Webcast in real time so that members of the public could listen to the proceedings and send questions to the committee by e-mail. The committee also received public submissions of material for its consideration at the meetings and by mail, e-mail, and fax throughout the course of the study. A list of the public submissions received by the committee is in Appendix F.
A Web site (http://www.iom.edu/nipdatasharing) and a listserv were created to provide information to the public about the committee’s work and to facilitate communication with the committee. Many of the speakers’ presentation slides from the two information-gathering meetings are available in electronic format on the project’s Web site.
Committee members and staff made informal visits to the NIP, NCHS, and one of the MCOs contributing data to the VSD to gain a greater understanding of the background and daily operations of the VSD data sharing program. The site visits provided additional background information for the committee. The committee developed a list of questions for the NIP and NCHS (submitted to the agencies after the August 23-24, 2004, committee meeting) that provided the context for the visits; the NIP and NCHS submitted a formal response to the committee’s list of questions (CDC, 2004d).
A list of materials reviewed by the committee (in the form in which they were reviewed), including all submissions of information from the public and many items not cited in this report, can be obtained from the National Academies Public Access Records Office at (202)334-3543 or http://www.national-academies.org/publicaccess.
When the committee was convened, the NIP asked it to produce two reports—one on each part of the charge. In the course of its deliberations, however, the committee found that the two parts of its charge overlapped substantially. It concluded that it could provide its best advice to the NIP and NCHS if it thought broadly about solutions that would address all the overlapping concerns and if it integrated its findings, conclusions, and recommendations into a single report. It sought to recommend the best
solutions for issues inherent in the full charge; it would not have been able to provide its best advice if it viewed the two parts of the charge separately. The NIP permitted the committee to provide a single report, and this report thus responds to the committee’s full charge.
CONTEXT OF THIS STUDY
To appreciate important elements of the societal context of this report, it is important to acknowledge the breadth and depth of concerns that are peripheral to this IOM study but related to the VSD. Concerns about public access to data, transparency of research activities, researchers’ conflicts of interest, and confidentiality of individuals’ information are an important part of the context of this study. The confluence of the concerns affects how the public, advocacy groups, researchers, the federal government, and MCOs approach vaccine safety and data sharing issues.
Over the last two decades, community engagement in the health enterprise generally and in vaccine safety activities in particular has come under increasing public scrutiny. Criticisms related to vaccine safety have been wide-ranging. Some people believe that the increase in the rate of autism is attributable to the use of thimerosal in vaccines (Fisher, 2004b; SafeMinds, 2004a), that CDC has used questionable research methods when examining that possibility (Bernard, 2004), that CDC is limiting access to the VSD to prevent the discovery of evidence about vaccine adverse reactions (Fisher, 2004b), and that oversight of vaccine safety activities should not be in the same CDC office that is responsible for promoting immunization (Copeland and Simpson, 2004; Fisher, 1999). NIP staff indicated to the committee that they have devoted much time to responding to those concerns (Bernier, 2004a). With greater attention to access to and use of VSD data, the MCOs participating in the VSD have indicated that they have spent considerable time on ensuring that proper procedures are in place to protect the confidentiality of VSD data (Wharton, 2004). Previous IOM committees that examined the evidence on particular vaccine safety questions also have been criticized by some groups (NVIC, 2004; SafeMinds, 2004b).
There has been increasing concern over the last few years about protecting the confidentiality of personal data, whether held by the U.S. government or by the private sector, that meet the definition of protected health information (HHS, 2003a). The Health Insurance Portability and Accountability Act (HIPAA) created new standards for the protection of the confidentiality of data meeting the definition (Pub. L. No. 104-191 ). In 2003, a privacy rule was issued by DHHS to implement HIPAA (HHS, 2003a). Organizations subject to the privacy rule must have standards in place to address the use and disclosure of health information on individu-
als (“covered entities”). They must also have standards to help people to understand their privacy rights and how their health information is used (HHS, 2003a). The year 2002 saw passage of the Confidential Information Protection and Statistical Efficiency Act (CIPSEA), which requires that information not be disclosed in an identifiable form for any nonstatistical purpose unless there is formal consent from the individual (Pub. L. No. 107-347 ). CIPSEA was enacted to establish uniform confidentiality protections and promote statistical efficiency by authorizing limited data sharing (Title V Pub. L. No. 107-347 ); when a federal agency collects data for statistical use only, it can protect the data, and they will be exempt from release under the Freedom of Information Act (FOIA).
The demand by consumers for more information about issues affecting their health and the health of their families is growing and evolving. As evidenced by the recent public outcries over the alleged withholding of pharmaceutical risk information by FDA and pharmaceutical manufacturers, consumers insist that there be more and more transparency in the research processes, practices, and policies that affect their health. Litigation may also be a motivating force for requesting access to data and for performing particular studies. This can be considered a new era of consumer advocacy and consumer access, and the institutions that historically have controlled health information are still determining how to respond to evolving consumer demands.
The study reported here occurred at a time of increased focus on the transparency of U.S. government-funded research activities, including open access to the published results of federal research, to research databases, and to the results of clinical trials. The Information Quality Act, enacted in December 2000, requires federal agencies to establish a process for ensuring the “quality, objectivity, utility, and integrity” of the data and information disseminated to the public by the federal government (Pub. L. No. 106-554 ; Copeland and Simpson, 2004). The Shelby Amendment (Pub. L. No. 105-277 ), enacted in October 1998, required all federal agencies to ensure that data resulting from a grant award be made available to the public through FOIA (Gough and Milloy, 2000; Phillips, 2002). Federal agencies also have instituted other measures to promote and standardize data release. For example, both CDC and the National Institutes of Health (NIH) recently released data sharing guidelines (CDC, 2003b; NIH, 2004a) outlining the processes whereby data should be shared with other researchers. On September 3, 2004, NIH released for public comment a proposal requiring that final peer-reviewed manuscripts containing results of NIH-funded research be available in a free, publicly accessible database (PubMed Central) 6 months after publication, or sooner if the publisher agrees (NIH, 2004b). On February 3, 2005, after receiving 6,000 public comments, NIH announced the new policy
that calls on scientists to voluntarily release to the public manuscripts from research supported by NIH as soon as possible, and within 12 months of final publication (NIH, 2005b). There is increasing public demand for the registration of current clinical trials to foster full transparency of clinical-trial research results. It is in that environment that the committee conducted its review and the committee’s report has been developed.
Some people have criticized the NIP for how the VSD data sharing program was developed and implemented, for the processes used to conduct a thimerosal screening study (Verstraeten et al., 2003a), and for how the findings from that study were released and shared with interested stakeholders. The committee heard many of those concerns during the public comment periods at its two meetings and duly noted the level of concern expressed by many of the attendees, some of whom stated their frustration with the systems and processes in place for VSD data sharing. Many of the same sentiments were expressed in e-mails to the committee.
The committee has reviewed and considered all documents and other information submitted by the NIP, NCHS, independent external researchers, other groups, and the public, and it appreciates the thoughtful comments it has received. The committee has addressed its charge by deliberating the relevant issues with an ear open to the voices of all interested parties. The committee’s report is not intended to resolve many of the specific points of contention around vaccine safety issues, but the processes for data sharing suggested by the committee should offer opportunities for greater public transparency and information sharing and consequently a means to address criticisms and enhance trust among segments of the public in which it has been eroded.
ISSUES FRAMING THE COMMITTEE’S DELIBERATIONS
Information-sharing for databases that have potentially important implications for public health policy and decision-making raises issues that go beyond the traditional norms or practices for analysis and communication of results in science. The objectives of providing public health agencies and the public with guidance for the policy process and of evaluating public health practices place additional responsibilities and demands on those entrusted with protecting the confidentiality of individually identifiable information in the database, agencies acting as gatekeepers, and individuals seeking access to the information. Procedures for access, analysis, and data sharing must foster public trust and confidence in conclusions and decisions based on findings. They should also promote confidence in the integrity and appropriateness of the data for addressing policy-relevant questions. Those goals are more likely to be achieved for vaccine safety issues and the use of the VSD if the proce-
dures are transparent, fair, credible, reliable, and justifiable (Ball et al., 1998; Calman, 2002; McComas, 2004b; McComas and Trumbo, 2001). Concomitantly, the integrity of the data must be monitored and ensured in ways that promote further sharing for the purposes of improving public health by private entities that have access to otherwise unavailable data and resources. Those principles and balancing factors framed the committee’s deliberations, its review of current practices, and its recommendations.
Concerns about access to VSD data and the results based on those data are tied closely to concerns about credibility, transparency, and trust regarding the NIP. Trust in the VSD—as a program designed, implemented, and maintained in the public interest—suffers when members of the public do not have confidence that systems for access are fair and transparent. Confidence in the NIP and its public health decisions that touch the lives of millions of Americans is tied directly to the perceived independence, transparency, and fairness of the VSD data sharing program.
PREVIOUS RELEASE OF PRELIMINARY FINDINGS FROM THE VACCINE SAFETY DATALINK
To appreciate the context of the committee’s review of whether, when, and how preliminary findings from the VSD should be shared with other scientists, communicated to the public, and used to make policy or recommendations to CDC, it is important to understand the concerns and circumstances surrounding previous releases of preliminary findings from the VSD. Many of the concerns related to the release of preliminary findings from the VSD stem from a study by Verstraeten et al. (2003a) that was intended as an initial screen of possible associations between thimerosal and neurodevelopmental disorders.
Some members of the general public have criticized the thimerosal screening study for changes in the original study protocol, changes in eligibility criteria, the selective official release of preliminary findings, and the inclusion of vaccine manufacturers’ representatives in a meeting intended to provide external expert review of the study (Bernard, 2004). One of the publicly expressed criticisms was that preliminary findings indicating no association between thimerosal exposure and neurodevelopmental disorders were released to advocates and at a conference presentation in May 2000, whereas preliminary findings that indicated a weak association were released a month later at the Simpsonwood meeting that included vaccine “insiders” (Bernard, 2004). Concerns about those differences in interpretation have been cited as the reason that a FOIA request was submitted to gain additional information about the study (SafeMinds,
2003). In a presentation to the committee, NIP staff described the decisions that were made during the course of the thimerosal study, the preliminary findings that were released to particular groups, and the timing of the releases (DeStefano, 2004).
The committee was not charged with reviewing the procedures used for that study, but it was asked to examine the preliminary findings issue by using it and other studies as illustrative examples of the iterative analysis approaches used for VSD studies. The concerns expressed by some members of the general public about such iterative analyses provide a part of the context for the committee’s recommendations. A detailed chronology of milestones of two VSD studies is provided later in this report (see Chapter 4).
HOW TRUST AFFECTS THE VACCINE SAFETY DATALINK
Importance of Trust in Perceptions of Vaccine Safety
Trust is essential to risk perceptions of the public and effective risk communication. Trust is easy to lose and difficult to win back (Poortinga and Pidgeon, 2004; Siegrist and Cvetkovich, 2001; Slovic, 1993), and a lack of trust can change how safety information is evaluated. “Negative” events (such as media reports that raise questions about vaccine safety) generally are likely to be weighed more than “positive” information (Cvetkovich et al., 2002; Siegrist and Cvetkovich, 2001). That implies that a single event questioning vaccine safety, even if invalid, can harm parents’ confidence in the safety of vaccines (Poortinga and Pidgeon, 2004). People reevaluate their risk perceptions under some circumstances, and confidence can erode even if data suggesting a risk are not overwhelming and are later refuted (Offit and Coffin, 2003).
Generally, Americans have confidence in CDC’s and FDA’s ability to provide safe and effective vaccines. A recent national survey found that the vast majority of parents (87%) understand the benefits of immunizations and rate immunization safety as relatively high (Gellin et al., 2000). However, a substantial proportion of parents (25%) do have beliefs that could erode their confidence in immunizations (Gellin et al., 2000). One of those beliefs is that children receive more vaccinations than are good for them. Furthermore, recent data from the National Immunization Survey showed that although most parents understand the importance of immunizations, the majority of parents do have some concerns about vaccine safety and have raised this issue with pediatricians (Bardenheier et al., 2004). Those concerns were not enough for most parents to refuse immunizations, but for some groups that was the case. Some American children remain underimmunized (Bardenheier et al., 2004; Gust et al., 2004). In
England, mass-media reports of a study that found a possible link between autism and measles-mumps-rubella vaccine led to decreased immunization rates in England (Offit and Coffin, 2003).
Even though rates of immunization among U.S. children are high, this does not necessarily mean that concerns about vaccine safety are limited to a small percentage of parents that hesitate to immunize their children. A silent majority generally have high confidence in CDC and in immunizations (e.g., Gellin et al., 2000) but also share some of the serious concerns of the more vocal and active groups questioning the safety of vaccines in the United States (Bardenheier et al., 2004; Gellin et al., 2000). Their lingering concerns suggest that building the public’s confidence and addressing attitudes and beliefs that might place immunization decisions at risk are crucial tasks for maintaining and improving immunization coverage in the United States (Gust et al., 2004).
Trust Relationships Relevant to the VSD
The committee heard that some people do not trust the NIP to portray vaccine safety risks accurately (Bernard, 2004; Fisher, 2004a). Taking steps to improve the independence, transparency, and fairness of VSD procedures while continuing to protect confidentiality will help to enhance a variety of trust relationships.
Four types of trust relationships are particularly relevant to the VSD, each with specific concerns and implications:
Researcher-participant trust relationships are affected by participants’ trust that their personal information will remain confidential (Weijer, 2004). For the VSD, if members of the MCOs that participate in the VSD do not trust that their medical information will remain confidential when analyzed by external (or internal) researchers, MCOs may question their involvement with the VSD since it affects their relationships with their members.
Researcher-other scientist trust relationships are affected by researchers’ trust in colleagues to report research results accurately and appropriately (Weijer, 2004). For the VSD, when researchers do not trust that other researchers conducted a study appropriately or reported the results accurately, they may ask to conduct an audit or reanalysis of a published study through the data sharing program.
Researcher-society trust relationships are affected by society’s trust in the researchers to portray research findings accurately and to be free of conflicts of interest relevant to the research (Weijer, 2004). For the VSD, if members of the public do not trust that final research results portray all study findings accurately, they may ask to see earlier versions of the study’s findings or to have broader access to VSD data.
Researcher-sponsor relationships also must be recognized. The relationship between the sponsor of research and the researcher is relevant to understanding any possible influence on the research—the framing of the research question, the methods used, the interpretation of the findings, and the dissemination of the findings.
Research with VSD data—and scientific research in general—operates best in an environment of trust among all those responsible for and affected by the research findings. The importance of these trust relationships is conveyed in On Being a Research Scientist (NAS, NAE, and IOM, 1996):
The scientific research enterprise, like other human activities, is built on a foundation of trust. Scientists trust that the results reported by others are valid. Society trusts that the results of research reflect an honest attempt by scientists to describe the world accurately and without bias. The level of trust that has characterized science and its relationship with society has contributed to a period of unparalleled scientific productivity. But this trust will endure only if the scientific community devotes itself to exemplifying and transmitting the values associated with ethical scientific conduct.
Trust in sources, trust in researchers, trust in policy, and trust in outcomes all are relevant to the VSD. All are interrelated and affect each other. All can be enhanced by greater trust in the VSD process—the first step in enhancing trust in the findings from the VSD and other vaccine safety activities.
In the course of its deliberations, the committee found that four overarching principles emerged. The principles can be described as common themes inherent in the committee’s recommendations and thus principles that should be considered in any modifications to the VSD data sharing program or any determinations about whether, when, and how to release VSD preliminary findings. The overarching principles apply to issues related both to the VSD data sharing program (the focus of the first part of the committee’s charge) and to whether, when, and how to release preliminary findings (the focus of the second part of the charge). The
committee’s recommendations can be understood best in the context of the four overarching principles.
Some principles are inherent in the scientific process (such as scientific integrity, protection of human subjects, and ethical conduct of research); such principles also are inherent to the VSD research process. The four overarching principles identified by the committee build on the principles that are inherent in the general scientific process. They represent important concerns for VSD research; their relative importance for other kinds of research will vary.
The four overarching principles that emerged from the committee’s recommendations are:
Independence. Ensure that potential biases and potential conflicts of interest are minimized, balanced, or otherwise managed in the design and implementation of all processes, practices, and policies related to the VSD.
Transparency. Ensure that all processes, practices, and policies related to the VSD are developed in the spirit of openness, clearly articulated, and easily available to interested persons or entities, and that any deviations from them are documented and justified.
Fairness. Ensure that all processes, practices, and policies related to the VSD are designed and implemented in a fair manner.
Protection of confidentiality. Ensure that the design and implementation of the VSD protect the confidentiality of individually identifiable information.
The committee determined that it could not adequately address issues of independence, transparency, fairness, and protection of confidentiality without examining how the VSD research process supports or hinders the application of those principles. Concerns that arise about the independence and transparency, in particular, of the general VSD research process spill over into people’s perceptions of the independence and transparency of the VSD data sharing program and of the determinations about whether, when, and how to release VSD preliminary findings. The committee believed that it had to consider how the VSD research plan, the setting of priorities among VSD studies, and the VSD peer-review process affect the VSD data sharing program and the release of preliminary findings if it wanted to provide the most appropriate and useful recommendations requested in its charge.